Hardening is where you convert exploration stories into repeatable gates: deterministic validators for structured outputs, a growing golden set seeded from real failures, and logging hooks that make postmortems possible. It is the practical answer to velocity vs. verification—small continuous tax instead of a big pre-release panic.
Align with eval layers
Map your gates to the eval hierarchy: CI-friendly checks first, then human-labeled goldens for high-risk flows, then broader sampling. Avoid duplicating the same assertion in three places unless each layer catches a different failure class.
Contracts and rollback
Encode truth contracts as testable scenarios—especially “no evidence” paths. Pair model and index versions with rollback plans you will need again in ownership.
Red-team as a habit, not a launch gate
One-off adversarial reviews catch headline jailbreaks; sustained red-teaming catches the boring failures—ambiguous entities, mixed languages, and PDFs where the table and the footnote disagree. Feed those into the same eval corpus you use in CI so regressions cannot sneak in under “just a prompt tweak.”
When hardening finishes, you should be ready for pilot traffic with metrics that matter—not vanity demos.
Staging environments that mirror retrieval
A staging model without staging corpora produces false confidence. Version indexes and prompts together, and run smoke tests against both “happy” and “empty retrieval” fixtures—aligned with contract scenarios. This closes gaps between fast iteration and what actually ships.
Ownership preview
Assign a rotating “hardening owner” who triages flaky tests and approves golden-set additions—practice for the on-call culture you will need when traffic is real. If hardening is everyone’s side job, it becomes no one’s job the week before a launch.
Load and chaos before users arrive
Run soak tests on embedding and retrieval paths with production-scale batch sizes—even if generation is mocked. Failures under load often expose connection pools, timeouts, and backoff bugs that unit tests miss. Pair with latency budgets so you know whether to scale out or trim context before pilots.
Documentation that operators will read
Write runbook drafts during hardening, not after the first outage: how to disable a bad tool, how to pin a model version, where to find retrieval diagnostics. Link to logging fields by name so on-call does not grep blindly at 2 a.m.
Security hardening
Prompt injection and tool abuse belong in the same phase as functional tests. Add negative cases where users attempt data exfiltration via indirect prompts; align mitigations with policy on what assistants may retrieve or send externally.
Hardening is not about zero incidents; it is about knowing which incidents you refuse to have twice.