Two incidents, a framework, and the uncomfortable discovery that one layer of it — the authority ranking — was built, wired, and doing nothing at all.
Sangeethkumar · engineering leader, AI platform
Everything it knows about your world, at the moment it answers, is what you put in front of it.
We have twenty years of discipline for shipping code — version control, review, CI, staged rollout, rollback. We have almost none for the information the model reads. And in an AI product, that information decides the answer.
Twenty minutes after a clean promote, an unattended nightly job wrote straight to the live search index, crashed mid-rebuild, and destroyed more than three-quarters of it.
Recovery cost hours. Until it was restored later that morning, the live index held less than a quarter of its documents.
The safety checks existed. They just weren't on the path the robot took.
We had validation. A person ran it, when a person was involved. The scheduled job wasn't a person.
A gate that isn't wired into the robot is decoration.
Not a product. Not a diagram. A build job, a set of checks, an approval step, and a rollback button.
Agents propose context. Humans promote it.
And the promotion has to be the only door — not the recommended one. That distinction is the whole talk, and I learned it the expensive way.
Months later. A different scheduled job — added after the framework existed — cut the live index to well under half its size in a single unattended run.
Automated jobs build candidates only.
Automated jobs build candidates only, unless someone adds a new job.
A new job was added, in good faith, and there was no person and no mechanism whose responsibility it was to ask whether that job could write to production.
The rule existed in a document. It did not exist in anyone's job.
A flagged candidate is never promoted. Good — that's the gate working.
It also means that candidate never self-heals. The next cycle builds on the same broken base, gets flagged again, and the damage compounds.
Without reseeding from a known-good state each cycle, "blocked" slowly becomes "permanently broken, but safely."
Rank sources by authority — code beats docs, verified beats inferred. We built exactly that: a tier on every source, a confidence score, and a ranking step applied at every point results were returned.
The tier was hardcoded to the same value for every prose document. Every piece existed and was correctly wired. The field carried no information, so the sort did nothing at all.
Governance that carries no data is worse than none — it produces the feeling of safety without the fact of it.
Atlan, who have published the most careful framework on this, split the work into five phases, the last of which is governance and lifecycle. They write: "Misaligned ownership at Phase 4 is the single most common reason context products go stale in production." That's an organisational failure, not a technical one.
Retrieval quality has a dashboard. Model choice has a debate. Ownership of the knowledge layer has neither — so it stays unassigned, and surfaces months later as "the assistant got worse," with nobody able to say when or why.
Who gets paged when the context is wrong?
If the answer is a shrug, everything in this talk is theoretical.
And the second question, which is not comfortable either: list every path that can write to production, and show me which gate each one passes through. Not the documented paths. All of them. Twice now, the answer included a path that did not go through the gate.
shipscale.org · sangeethcloud@gmail.com