All notes

SDLC → CDLC: governing the context that feeds your LLM

Your AI demo last month was magic. Is it still magic today? Nobody touched the prompt. The context moved.

Be honest about that question, because for a lot of teams the answer is “…not quite.” The same feature that wowed everyone in the demo now misses things. No one changed the model. No one edited the prompt. So what broke?

The context did.

The part we don't talk about enough

We spent twenty years perfecting how we ship code. Version control, code review, CI, staged rollout, feature flags, rollback. The whole SDLC exists because we learned — expensively — that changes to production need a process between “someone made an edit” and “users are affected.”

In an AI product, the code is the easy part. What actually decides whether your system is useful or embarrassing is the context — the bundle of information you put in front of the model along with the user's question. The model knows nothing about your company. In that moment, everything it knows about your world is what you handed it.

The useful way to sort that bundle isn't by file type. Atlan's guide to context architecture splits it into five layers — system, session, memory, artifacts and on-demand — and the way I read those layers is by how long each piece is supposed to live. Some of it is permanent — your rules, your guardrails. Some lives for one conversation and is gone. Some is institutional memory that accumulates across months. Some is a versioned artifact you deliberately maintain, like a spec or a runbook. And some is fetched fresh for a single question and never stored at all.

That distinction matters because each lifetime has a different way of going wrong. Permanent rules rot quietly. Institutional memory contradicts itself. Fetched-fresh information is only as good as what it was fetched from. Sorting context by file format tells you nothing useful; sorting it by lifetime tells you where to look when it breaks.

So you go back in and adjust things. You reprocess the documents. You change how they're split up. You tweak which ones get picked. It works for a while, then drifts, and you're back to chasing it.

Here's what should bother you about that: every one of those adjustments changes how the system answers in production — and none of them went through version control, a review, or a test. We would never accept that for code. We accept it for context every day, mostly because we've never framed it as a change to production at all.

The harder question underneath

There's a second problem hiding below the staleness one, and it's the one that actually changes how you have to work.

LLMs are probabilistic. The same context can produce a brilliant answer today and a wrong one tomorrow. So how do you make a confident decision in a probabilistic era?

Not by trusting the model. By measuring it.

That distinction sounds obvious written down, and almost nobody's pipeline reflects it. We ship context changes on the strength of a spot check and a good feeling, then act surprised when quality moves in a direction we didn't intend and can't attribute.

CDLC: a Context-Driven Life Cycle

So here is what it actually is, in one line: CDLC is a release pipeline for the information your AI reads. Not a product, not a diagram — a build job, a set of checks, an approval step, and a rollback button. If you have shipped code through CI in the last decade, you already know the shape of it.

The idea underneath: stop hand-feeding context. Let automation do the work — but make it show its work.

Agents do the work. Measurement makes it legible. The human decides.

Honest status: that's the design. In my own system, not every part of it was enforced.

A real one

In June, a test run of the sync pipeline built a candidate about two-thirds smaller than the live knowledge base. The shrink check flagged it. Live was never touched, and nothing was promoted.

It was not the first change that size. A little over two weeks earlier, an unattended job had written straight to production and taken most of the knowledge base with it. Two months later, a new scheduled job that had never been put behind the gate did the same again. The gate protected the paths it stood on, and no others.

That run was started and watched by hand, but the flag didn't depend on anyone watching: the shrink check ran as part of the pipeline. That is the entire point. The safety property has to belong to the pipeline, not to someone's attention.

None of this is novel machinery. It is CI, code review, staged rollout and rollback — four things every team in this industry already knows how to run — pointed at the one artifact we somehow decided didn't need them.

Which is why it doesn't get built

If the machinery is this familiar, the interesting question is why so few teams have it. Atlan, who have written the most careful public framework on this, split the work into five phases, the last of which is governance and lifecycle. They write: “Misaligned ownership at Phase 4 is the single most common reason context products go stale in production.” That's an organisational failure, not a technical one.

That matches my experience exactly, and I'd go further: an unowned lifecycle doesn't announce itself. Retrieval quality has a dashboard. Model choice has a debate. Ownership of the knowledge layer has neither — so it stays unassigned, and the failure surfaces months later as “the assistant got worse” with no one able to say when or why.

The question that actually predicts whether a team has this problem isn't about their stack. It's: who gets paged when the context is wrong? If the answer is a shrug, everything above is theoretical.

The hard part was never technical. It was deciding that the information feeding the model is a thing someone owns.