Fine-Tuning Forge
Fine-tuning is usually a black-box decision made off a vendor slide: "just fine-tune it." This forge lets you scrub a cached LoRA training run instead — set how good your best prompt already is, slide the number of training examples, and watch the accuracy curve redraw in real time. Then it hands you the one number that actually decides it: the break-even, the exact example count where fine-tuning finally beats prompting by a meaningful margin. Below that line, a better prompt is cheaper and faster; above it, the data is worth it. Runs entirely in your browser (0 uploads, works offline).
How to use it
- Load a preset — Strong prompt, little data, Typical run, or Weak prompt, data-rich — or set your own.
- Drag the prompting baseline slider to where a good prompt already lands on your task.
- Slide training examples from 0 (prompt only) up to 5,000 and watch the saturating curve and the accuracy readout move.
- Read the break-even and open the X-ray to see why the curve flattens — then copy or share the number.
What this clears up (the fundamentals)
- Fine-tuning is a saturating curve, not a straight line — accuracy climbs toward a ceiling and each extra 100 examples buys less than the last. "More data" stops helping long before you notice.
- Prompting is the baseline you have to beat — the honest question is never "does fine-tuning work?" but "does it beat the prompt I could write for free, by enough to matter?"
- The break-even is the decision — one number: the example count where the curve crosses your prompt plus a real margin. Below it, prompt harder; above it, forge.
- Marginal gain is the tell — when the last 100 examples add a fraction of a point, you are paying full price for almost nothing. That shrinking number is the whole story.
Where it's used
An in-module elective for the AI-engineering track (AIE-110), positioned as a preview of the future fine-tuning deep-dive. It is a miatz build-lab concept playable: scrub the cached runs here to feel the economics, then learn to build the saturating-curve model and the break-even inversion yourself — no live GPU, no training bill, just the decision made explorable.
FAQ
When should I fine-tune instead of prompting?
When your best prompt has plateaued below the accuracy you need and you have enough labelled examples to clear the break-even this tool computes. If a sharper prompt, better few-shot examples, or retrieval can close the gap, that is almost always the cheaper first move. Fine-tuning wins when the task is narrow, the format is strict, and you have the data.
What does the break-even number mean?
It is the smallest number of training examples at which the fine-tuned model's accuracy exceeds your prompting baseline by more than the margin (2 points here). Fewer examples than that and prompting is the better bet; more and fine-tuning starts paying for itself.
Why does the curve flatten out?
Fine-tuning follows diminishing returns: the first examples teach the model the most, and each additional batch corrects rarer and rarer cases. The curve saturates toward a ceiling, which is why doubling your data rarely doubles your gain — and why "just add more examples" eventually stops working.
Is anything uploaded?
No. Every curve and break-even is computed in your browser from a seeded, closed-form model — nothing is transmitted, stored, or logged, and no model is called. Turn off your Wi-Fi and it still works.
Is this training a real model?
No. There is no live GPU and no training happening. You are scrubbing a pre-computed, illustrative curve that captures the shape of real LoRA fine-tuning economics so the break-even decision becomes something you can feel, not just be told.
Limits
The curves are precomputed and illustrative — a single seeded saturation model, not a benchmark of your specific base model, task, or dataset. Real runs vary with data quality, task difficulty, rank, learning rate, and eval set, and the accuracy ceiling differs per task. Treat the break-even as a way to build intuition and frame the tune-vs-prompt conversation, not as a procurement number to quote in a contract.
Related
Part of the Demystify Playgrounds. Explore the rest from the Playgrounds home.
Bookmark this page (Ctrl+D, or ⌘D on Mac) — it works offline the next time you need it.