Topics

What I work on, given twice: once in the words the field uses, once in plain English. Every claim below points at something — a piece of writing, a deck, or a specific thing that happened. Nothing here is a keyword I don't have evidence for.

Evaluation Context engineering AI governance Agentic systems Retrieval & search Orchestration Models & model selection Distributed systems Reliability & observability Engineering leadership

Evaluation

Deciding whether a system that gives different answers each time actually got better — with a number you'd stake a release on.

LLM evaluation · eval harness · LLM-as-judge · gold sets · offline eval · benchmarking · variance and significance testing

Context engineering

Designing what information reaches the model — and treating that information as a product with owners, versions and tests, rather than a folder someone dumps files into.

context layer · context window · context governance · CDLC · RAG substrate · provenance and lineage · freshness · knowledge layer

AI governance

The controls that decide what an automated system is allowed to change, who approves it, and how you undo it when it turns out to be wrong.

runtime governance · guardrails · human-in-the-loop · audited rollback · gating · promotion · least-privilege agent tooling

Agentic systems

Systems where the model decides which tools to call and in what order, rather than following a fixed script.

AI agents · tool use · function calling · tool routing · agent loops · ReAct · MCP · autonomous pipelines

Fixed context versus an agent loop Handed a fixed context: a question is answered from whatever a retrieve-once step returned. In the agent loop: a question arrives with a first knowledge-base search already run automatically; the model then decides the next step, searches again, greps source or reads a source file, sees the results, and answers with sources once it can answer or the turn limit forces an answer. One turn can call several tools at once. Handed a fixed context Question Retrieve once Answer from whatevercame back Agent loop Question and an automatic search Model decides the next step search grep read results Search the knowledgebase Grep source Read a source file can answer, or the turn limit forces an answer Answer with sources One turn can call several tools at once. Fixed context versus an agent loop Handed a fixed context: a question is answered from whatever a retrieve-once step returned. In the agent loop: a question arrives with a first knowledge-base search already run automatically; the model then decides the next step, searches again, greps source or reads a source file, sees the results, and answers with sources once it can answer or the turn limit forces an answer. One turn can call several tools at once. Handed a fixed context Question Retrieve once Answer fromwhatevercame back Agent loop Question and an automatic search Model decides the next step search grep read results Search theknowledgebase Grepsource Read asource file can answer, or the turnlimit forces an answer Answer with sources One turn can call several tools at once.
Handed a fixed context, the model answers from whatever a single retrieval step returned. In the agent loop, a first knowledge-base search runs automatically; then the model decides each turn whether to search again, grep source or read a file, sees the results, and stops when it can answer or the turn limit forces an answer.
How the router picks a tool, and where it failed A question reaches a router that reads each tool's description, in catalogue order. When a description names what's asked, the specific tool returns the rows; when nothing names it — a routing gap — it falls through to a catch-all keyword search, listed last. The specific tool: tool bug, an argument read too narrowly. The tool's rows and the catch-all's report of the nothing it received both feed the compose step, which produces the answer. A gap that clusters by category is a missing tool, not a model problem. Tools listed after the catch-all are ones the router was already told it doesn't need. Question Router reads each tool'sdescription, in catalogue order a descriptionnames what'sasked nothingnames it ·routing gap Specific toolreturns the rows tool bug: an argumentread too narrowly Catch-allkeyword search,listed last reports the nothingit received Compose the answer from what came back Answer A gap that clusters by category is amissing tool, not a model problem. Tools listed after the catch-all are onesthe router was already told it doesn't need.
The chat assistant routes each question by reading tool descriptions. When no description named what was asked, it fell through to a catch-all keyword search and honestly reported the nothing it got back, even though the data was there. None of the failures was missing data; they were routing gaps and tool bugs.

Retrieval & search

Finding the right passages to put in front of a model — and knowing when the ranking that helps one kind of question quietly hurts another.

RAG · retrieval-augmented generation · vector search · pgvector · embeddings · chunking · hybrid search · reranking · cross-encoder · intent classification

Orchestration

Coordinating many moving parts — jobs, agents, stages — so that failures are contained and the sequence is recoverable.

multi-agent orchestration · pipelines · workflow engines · scheduling · job orchestration · state machines · checkpointing

Models & model selection

Choosing and swapping the model underneath a product — and proving the swap actually helped on real traffic, not just on the questions you happened to test.

model routing · model evaluation · foundation models · Claude · GPT · Gemini · prompt caching · cost/latency tradeoffs

Distributed systems

Systems spread across services, clusters and regions, where the interesting failures are the ones that only appear at the seams.

multi-tenant SaaS · replication · change data capture · event-driven · Pub/Sub · reconciliation · multi-cluster · data pipelines

Reliability & observability

Knowing when something is broken — especially the failures that produce no error, no alert, and no record that they happened.

incident RCA · SRE · monitoring · instrumentation · silent failures · production debugging · Kubernetes · Cloud Run

Engineering leadership

Running the function: what it can absorb, how to grow the people in it, and how to turn engineering reality into something leadership can decide on.

engineering management · capacity planning · headcount modelling · talent retention · platform-as-product · AI transformation · executive communication