What I work on, given twice: once in the words the field uses, once in plain English. Every claim below points at something — a piece of writing, a deck, or a specific thing that happened. Nothing here is a keyword I don't have evidence for.
Deciding whether a system that gives different answers each time actually got better — with a number you'd stake a release on.
LLM evaluation · eval harness · LLM-as-judge · gold sets · offline eval · benchmarking · variance and significance testing
Designing what information reaches the model — and treating that information as a product with owners, versions and tests, rather than a folder someone dumps files into.
context layer · context window · context governance · CDLC · RAG substrate · provenance and lineage · freshness · knowledge layer
The controls that decide what an automated system is allowed to change, who approves it, and how you undo it when it turns out to be wrong.
runtime governance · guardrails · human-in-the-loop · audited rollback · gating · promotion · least-privilege agent tooling
Systems where the model decides which tools to call and in what order, rather than following a fixed script.
AI agents · tool use · function calling · tool routing · agent loops · ReAct · MCP · autonomous pipelines
Finding the right passages to put in front of a model — and knowing when the ranking that helps one kind of question quietly hurts another.
RAG · retrieval-augmented generation · vector search · pgvector · embeddings · chunking · hybrid search · reranking · cross-encoder · intent classification
Coordinating many moving parts — jobs, agents, stages — so that failures are contained and the sequence is recoverable.
multi-agent orchestration · pipelines · workflow engines · scheduling · job orchestration · state machines · checkpointing
Choosing and swapping the model underneath a product — and proving the swap actually helped on real traffic, not just on the questions you happened to test.
model routing · model evaluation · foundation models · Claude · GPT · Gemini · prompt caching · cost/latency tradeoffs
Systems spread across services, clusters and regions, where the interesting failures are the ones that only appear at the seams.
multi-tenant SaaS · replication · change data capture · event-driven · Pub/Sub · reconciliation · multi-cluster · data pipelines
Knowing when something is broken — especially the failures that produce no error, no alert, and no record that they happened.
incident RCA · SRE · monitoring · instrumentation · silent failures · production debugging · Kubernetes · Cloud Run
Running the function: what it can absorb, how to grow the people in it, and how to turn engineering reality into something leadership can decide on.
engineering management · capacity planning · headcount modelling · talent retention · platform-as-product · AI transformation · executive communication