Testing Fine-tuned Model Quality
Generic benchmarks don't predict production quality. Domain-specific evals, regression tests, and A/B testing reveal whether your fine-tuning actually worked.
All Tags
#fine-tuning
7 posts tagged with "fine-tuning"
How Much Data You Actually Need
1,000 high-quality examples often outperforms 100,000 noisy ones. Data quality dominates quantity for fine-tuning. Curation is the work.
Deploying and Serving Fine-tuned Models
Merge adapters for single-tenant deployments. Keep them separate for multi-tenant. The serving architecture depends on how many customizations you're running.
The Real Cost: Fine-tuning vs Prompting
Prompting has high per-call cost but zero upfront investment. Fine-tuning has low per-call cost but significant upfront investment. The crossover point matters.
What Actually Works with LoRA
LoRA tutorials make it look easy. Production LoRA requires learning rate adjustments, layer selection, rank tuning, and careful validation. Here's what actually works.
Running Fine-tuned Models in Production
Fine-tuning a model is the easy part. Running it in production with checkpoints, evals, rollback, and serving is the hard part. Here's the full picture.
When LoRA Makes Sense
Full fine-tuning updates billions of parameters. LoRA updates millions. The 0.1% of parameters can capture 80% of the adaptation. Know when that's enough.