Prompt engineering is the fastest way to steer a base model—until the behavior you need is entangled with long-tail domain jargon, brittle formatting, or policy edges that repeat across thousands of tokens. At that point you are not “bad at prompting”; you are asking the prior to absorb a distribution it was not trained to own. Before jumping to full fine-tunes, consider whether retrieval and tools already encode the world’s facts better than weights can.
Intermediate options
Lightweight adapters and preference optimization can align tone or format without rewriting the entire model. They still need a stable reward: ranked outputs, human labels, or proxies tied to business metrics. Those rewards should show up in your eval stack, not only in offline loss curves.
Decision records
Teams that skip documentation pay twice: once in repeated experiments, again in onboarding. Capture what you tried in prompt space, why it failed, and what moved user-visible metrics—same rigor as runbooks for production incidents.
Training also shifts who owns quality: data labeling and skew become first-class concerns, intersecting policy when datasets contain sensitive content. Plan governance before you freeze a dataset snapshot.
When tools should win over weights
If the behavior you need is lookup, calculation, or an authoritative API response, put it in a tool with schema validation before you spend weeks tuning logits. That keeps truth boundaries testable and reduces the surface where the model can improvise—especially alongside human approval for irreversible actions.
Cold start vs. long-horizon maintenance
Prompts are cheap to change today and expensive to reason about a year later when twelve people have edited them. Adapters and fine-tunes add training pipelines, but they can also stabilize behavior when prompts become unmaintainable spaghetti. Document trade-offs in the same place you track eval coverage so the next team knows why you chose a path.
Instruction tuning vs. knowledge injection
Fine-tuning for format and tone usually works; fine-tuning for facts without a curated dataset invites silent hallucinations. Prefer retrieval and tools for facts, and reserve training for behavior that must be consistent even when context windows are tight—especially on edge where RAG may be trimmed.
Dataset hygiene
Labeling pipelines are where bias creeps in: over-enthusiastic raters, inconsistent rubrics, and PII pasted into examples. Run periodic inter-rater reliability checks and scrub before snapshots. Connect dataset versions to model releases the same way you version prompts in CI.
When to stop tuning and fix the product
If every new fine-tune is chasing a symptom—like users asking the wrong question—step back to UX and discovery. Sometimes the fix is a better default question, a template, or a guardrail in the workflow—not another epoch.
The right abstraction is rarely “prompt or fine-tune”—it is usually prompt, tool, retrieve, then selectively train what is left.