Notes

Writing

Not tutorials. The things I got wrong first, and the measurements that told me so. Every post draws on real work — the technique and the lesson are kept; employer, product and people details are not.

Sep 2026 The five ways context fails Start here When an AI feature gives a wrong answer, the instinct is to blame the model. Sometimes the model is the problem; often it's the information you handed it. Five distinct failures, five different fixes, which is why “improve the prompt” so often does nothing. Sep 2026 SDLC → CDLC: governing the context that feeds your LLM Governance Your AI demo last month was magic. Is it still magic today? Nobody touched the prompt — the context moved. Twenty years of shipping discipline, applied to the thing that actually decides whether an AI is useful or embarrassing. Sep 2026 The harness told me my AI was worse than I thought Evaluation I built a system to measure whether the assistant I'd shipped was any good. It told me it wasn't — and that was the most useful thing that happened to the project all quarter. A green dashboard you trust too early is more dangerous than a red one you understand. Sep 2026 A gate that isn't wired into the robot is decoration Incident An unattended sync wrote straight to a live index, crashed mid-rebuild, and destroyed most of it, twenty minutes after a healthy version went live. The safety checks existed. They just weren't on the automated path. Then it happened a second time. Sep 2026 One retrieval rule fixed enumeration and broke explanations RAG "What are all the X?" returned three when the real answer was forty. The fix worked perfectly — and quietly degraded questions that wanted an explanation instead of a list. One set of rules for ordering results cannot serve every kind of question.
In the queue

Coming next

Drawn from six recurring themes across the systems I've built. Roughly one a week.

SoonYour eval is noisier than your modelEvaluationVerdict churn, error bars, and the sweep where every apparent gain turned out to be inside the noise band.
SoonTwenty-four evaluation runs, zero promotionsEvaluationDecide offline over answers you already judged. Spend the expensive ruler only to confirm.
SoonYour judge has a bias you haven't measuredEvaluationWhen fixing the system reads as a regression, suspect the ruler before the work.
SoonGovernance you built but never filled inGovernanceEvery column present, every call site wired, and the authority sort key a complete no-op.
SoonFailures that write no recordReliabilityEighteen hours of silent cron death, and why observability has to be designed, not added.