https://numerata.com/blog Notes on private AI infrastructure: training and deploying models on your own hardware, air-gapped training, fine-tuning, inference performance, and model-family guides. en-us Thu, 03 Sep 2026 12:00:00 +0000 https://numerata.com/blog/post-synthetic-data-for-fine-tuning https://numerata.com/blog/post-synthetic-data-for-fine-tuning Thu, 03 Sep 2026 12:00:00 +0000 Nobody has a labelled corpus of agent trajectories, including the teams shipping fine-tuned agents. Where synthetic training data comes from, why the verifier and not the generator sets the ceiling, how much you actually need, and what must never be generated. https://numerata.com/blog/post-tool-use-fine-tuning https://numerata.com/blog/post-tool-use-fine-tuning Wed, 02 Sep 2026 12:00:00 +0000 A frontier model has read every public API reference and none of yours. What tool-use fine-tuning changes, what one training record looks like, where the data comes from, and how to evaluate it beyond validation loss. https://numerata.com/blog/post-reinforcement-learning-fine-tuning-vs-sft https://numerata.com/blog/post-reinforcement-learning-fine-tuning-vs-sft Tue, 01 Sep 2026 12:00:00 +0000 Supervised fine-tuning teaches a model to imitate attempts that worked. Reinforcement learning fine-tuning teaches it to produce attempts that score well. Where SFT, RLHF, RLVR, and GRPO each win on multi-step tasks, and how to pick between them. https://numerata.com/blog/post-what-is-agentic-fine-tuning https://numerata.com/blog/post-what-is-agentic-fine-tuning Mon, 31 Aug 2026 12:00:00 +0000 Agentic fine-tuning trains a model on whole task trajectories, tool calls and observations included, and rewards it for finishing the job. What the data looks like, why it needs a live environment, and when it beats a better prompt. https://numerata.com/blog/post-what-is-gpu-orchestration https://numerata.com/blog/post-what-is-gpu-orchestration Thu, 20 Aug 2026 12:00:00 +0000 GPU orchestration is the software layer that pools many GPUs and schedules training and inference across them. What an orchestrator actually does, how it differs from provisioning and Kubernetes, and why utilization is the whole game on a fixed fleet. https://numerata.com/blog/post-what-is-a-model-in-a-box https://numerata.com/blog/post-what-is-a-model-in-a-box Mon, 17 Aug 2026 12:00:00 +0000 A model in a box is a self-contained AI system that trains, serves, and owns a model inside your own network. What's actually in the box, how it differs from an inference appliance and a cloud API, and when it beats renting a model. https://numerata.com/blog/post-air-gapped-soc2-coding-assistants https://numerata.com/blog/post-air-gapped-soc2-coding-assistants Thu, 13 Aug 2026 12:00:00 +0000 A six-step guide to deploying an AI coding assistant with no network path out: why air-gapping and SOC 2 are separate requirements, how each component runs offline, and the control-to-criterion map that turns isolation into audit evidence. https://numerata.com/blog/post-prevent-source-code-leakage-ai https://numerata.com/blog/post-prevent-source-code-leakage-ai Tue, 11 Aug 2026 12:00:00 +0000 An on-premises blueprint for AI developer tools: the nine paths code takes out of a network, the control that closes each one, how far air-gapping actually goes, and the six artifacts that prove the boundary holds. https://numerata.com/blog/post-how-python-garbage-collection-works https://numerata.com/blog/post-how-python-garbage-collection-works Mon, 10 Aug 2026 12:00:00 +0000 Reference counting, cycle detection, and the generational thresholds that actually trigger a collection. Plus how the free-threaded build counts allocations per thread, and what a negative count does to collection scheduling. https://numerata.com/blog/post-self-hosted-copilot-vs-public-ai https://numerata.com/blog/post-self-hosted-copilot-vs-public-ai Mon, 10 Aug 2026 12:00:00 +0000 A self-hosted AI coding assistant versus a public copilot on the three axes that actually differ: what leaves your network, where the milliseconds go, and how far you can customize. With a decision table and a reference architecture. https://numerata.com/blog/post-self-hosted-vs-api-cost https://numerata.com/blog/post-self-hosted-vs-api-cost Fri, 07 Aug 2026 12:00:00 +0000 The honest break-even math against a frontier API, with every assumption stated. One GPU pays for itself around 550M output tokens a month, and against the cheapest hosted models it may never pay off at all. https://numerata.com/blog/post-gpu-sizing-for-inference https://numerata.com/blog/post-gpu-sizing-for-inference Fri, 07 Aug 2026 12:00:00 +0000 The KV cache formula, why decode speed is capped by memory bandwidth, and what sub-50ms can honestly mean. With a sizing table from 8B to 70B. https://numerata.com/blog/post-ai-vendor-risk-review https://numerata.com/blog/post-ai-vendor-risk-review Fri, 07 Aug 2026 12:00:00 +0000 The 24 questions a review board asks about an on-prem AI deployment, what each one is really asking, and the artifact that closes it. Written to be forwarded internally. https://numerata.com/blog/post-fine-tuning-llms-for-finance https://numerata.com/blog/post-fine-tuning-llms-for-finance Fri, 07 Aug 2026 12:00:00 +0000 How to fine-tune open-source LLMs for financial NLP: sentiment classification, entity extraction, and financial QA, using LoRA, QLoRA, and open-source tools like Unsloth and Axolotl. https://numerata.com/blog/post-how-to-deploy-private-ai https://numerata.com/blog/post-how-to-deploy-private-ai Thu, 06 Aug 2026 12:00:00 +0000 Deploying private AI means running training and inference entirely on infrastructure you control. Here's the five-step process, from your isolation model to verifying zero data egress. https://numerata.com/blog/post-deepseek-model-family-guide https://numerata.com/blog/post-deepseek-model-family-guide Tue, 04 Aug 2026 12:00:00 +0000 DeepSeek's R1 triggered Nvidia's biggest single-day market cap loss ever by proving frontier reasoning didn't need a frontier budget. Here's how it's trained and when to use it. https://numerata.com/blog/post-gemma-model-family-guide https://numerata.com/blog/post-gemma-model-family-guide Tue, 04 Aug 2026 12:00:00 +0000 Google's Gemma is built from the same research as Gemini but released as open weights, tuned for the best capability per parameter at small sizes. Here's how it's trained and when to use it. https://numerata.com/blog/post-llama-model-family-guide https://numerata.com/blog/post-llama-model-family-guide Tue, 04 Aug 2026 12:00:00 +0000 Meta's Llama built the largest open-weight fine-tuning ecosystem, but its license was never truly open source and Llama 4 lost ground to Chinese rivals. Here's the full picture. https://numerata.com/blog/post-mistral-model-family-guide https://numerata.com/blog/post-mistral-model-family-guide Tue, 04 Aug 2026 12:00:00 +0000 Mistral AI's flagship went from a gated research license back to Apache 2.0 with Mistral Large 3, a 675B-parameter MoE. Here's who builds it, how it's trained, and when to use it. https://numerata.com/blog/post-kimi-model-family-guide https://numerata.com/blog/post-kimi-model-family-guide Tue, 04 Aug 2026 12:00:00 +0000 Moonshot AI's Kimi K2 trained a 1T-parameter open-weight model with zero loss spikes; K3 pushes that to 2.8T. Here's who builds it, how it's trained, and when to use it. https://numerata.com/blog/post-qwen-model-family-guide https://numerata.com/blog/post-qwen-model-family-guide Tue, 04 Aug 2026 12:00:00 +0000 Alibaba's Qwen family now runs from 0.6B up to Qwen3.5's 397B-parameter open-weight flagship. Here's who builds it, how it's trained, and which size fits your task. https://numerata.com/blog/post-ai-for-internal-tooling https://numerata.com/blog/post-ai-for-internal-tooling Mon, 03 Aug 2026 12:00:00 +0000 From code review to support copilots, here are six places private AI pays off in the tools teams build for themselves, without handing your codebase to a vendor. https://numerata.com/blog/post-ai-for-trading https://numerata.com/blog/post-ai-for-trading Mon, 03 Aug 2026 12:00:00 +0000 From signal generation to trade surveillance, here are six places private AI pays off on and around the desk, without leaking strategy or missing a latency budget. https://numerata.com/blog/post-ai-for-finance https://numerata.com/blog/post-ai-for-finance Mon, 03 Aug 2026 12:00:00 +0000 From transaction monitoring to trading signals, here are six places private, fine-tuned AI outperforms a public model API in finance. https://numerata.com/blog/post-what-is-an-inference-engine https://numerata.com/blog/post-what-is-an-inference-engine Mon, 03 Aug 2026 12:00:00 +0000 Training produces a model's weights. An inference engine is the separate piece of software that serves them, fast, at scale, under real request traffic. https://numerata.com/blog/post-what-is-a-small-language-model https://numerata.com/blog/post-what-is-a-small-language-model Mon, 03 Aug 2026 12:00:00 +0000 A small language model trades general breadth for speed, cost, and control. Here's what actually makes a model "small," and when that trade is worth making. https://numerata.com/blog/post-what-is-fine-tuning https://numerata.com/blog/post-what-is-fine-tuning Wed, 29 Jul 2026 12:00:00 +0000 Fine-tuning updates a pretrained model's weights to fit your task. Here's how it actually works, and how it differs from prompting and RAG. https://numerata.com/blog/post-torch-compile-warmup-overhead https://numerata.com/blog/post-torch-compile-warmup-overhead Wed, 29 Jul 2026 12:00:00 +0000 Most of torch.compile()'s speedup comes from one thing: CUDA graphs. Here's how NinetyFive gets the same performance without the multi-minute warmup tax. https://numerata.com/blog/post-air-gapped-llm-training https://numerata.com/blog/post-air-gapped-llm-training Wed, 29 Jul 2026 12:00:00 +0000 Most vendors' "private cloud" just means a VPC inside AWS or GCP. Air-gapped is a stricter, different claim, here's what it actually takes. https://numerata.com/blog/post-deployment-agnostic-ai-infra https://numerata.com/blog/post-deployment-agnostic-ai-infra Wed, 29 Jul 2026 12:00:00 +0000 Coupling your training and inference tools to a specific cloud, data center, or region is a decision you'll regret. Here's what deployment-agnostic infrastructure looks like instead.