AI Knowledge Assistant

An internal assistant giving engineers source-grounded answers about a large multi-service platform.

Designed and built end to end — an internal assistant giving engineers source-grounded answers about a large multi-service platform. Agentic retrieval with tool use, a knowledge pipeline that keeps its knowledge current, and a category-balanced evaluation harness scoring correctness, completeness and faithfulness separately. Made answer quality a measured number rather than a feeling — and the honest number held back a wider rollout.

Production RAG · Agentic retrieval · LLM evaluation · Multi-agent orchestration

Python · FastAPI · Next.js · PostgreSQL · pgvector · Cloud Run · Claude / GPT / Gemini

AI Knowledge Assistant: components The knowledge assistant's parts: a Next.js web app and a Python/FastAPI agent loop on Cloud Run. Each turn, the model chooses whether to search the PostgreSQL/pgvector knowledge store or grep and read source code. A knowledge pipeline keeps the store current, and a category-balanced evaluation harness scores correctness, completeness and faithfulness separately. Engineer'squestion Cloud Run Web app · Next.js API and agent loop Python · FastAPI Models · Claude /GPT / Gemini Knowledge pipeline Context Governance Framework Knowledge store · PostgreSQL · pgvector Source code Source-grounded answer each turn, the modelchooses a tool search:hybrid search+ reranking grep, read keeps itcurrent Evaluation harness category-balanced questions · LLM judge correctness · completeness · faithfulness, scored separately runs questions through AI Knowledge Assistant: components The knowledge assistant's parts: a Next.js web app and a Python/FastAPI agent loop on Cloud Run. Each turn, the model chooses whether to search the PostgreSQL/pgvector knowledge store or grep and read source code. A knowledge pipeline keeps the store current, and a category-balanced evaluation harness scores correctness, completeness and faithfulness separately. Engineer's question Web app · Next.js Cloud Run API and agent loop Python · FastAPI Models Claude / GPT / Gemini each turn, the model chooses a tool Knowledge pipeline Context Governance Framework keeps it current Knowledge store · PostgreSQL · pgvector search: hybrid search + reranking Source code grep, read Source-grounded answer runs questions through Evaluation harness category-balanced questions · LLM judge correctness · completeness · faithfulness, scored separately
The knowledge assistant's parts: a Next.js web app and a Python/FastAPI agent loop on Cloud Run. Each turn, the model chooses whether to search the PostgreSQL/pgvector knowledge store or grep and read source code. A knowledge pipeline keeps the store current, and a category-balanced evaluation harness scores correctness, completeness and faithfulness separately.
Fixed context versus an agent loop Handed a fixed context: a question is answered from whatever a retrieve-once step returned. In the agent loop: a question arrives with a first knowledge-base search already run automatically; the model then decides the next step, searches again, greps source or reads a source file, sees the results, and answers with sources once it can answer or the turn limit forces an answer. One turn can call several tools at once. Handed a fixed context Question Retrieve once Answer from whatevercame back Agent loop Question and an automatic search Model decides the next step search grep read results Search the knowledgebase Grep source Read a source file can answer, or the turn limit forces an answer Answer with sources One turn can call several tools at once. Fixed context versus an agent loop Handed a fixed context: a question is answered from whatever a retrieve-once step returned. In the agent loop: a question arrives with a first knowledge-base search already run automatically; the model then decides the next step, searches again, greps source or reads a source file, sees the results, and answers with sources once it can answer or the turn limit forces an answer. One turn can call several tools at once. Handed a fixed context Question Retrieve once Answer fromwhatevercame back Agent loop Question and an automatic search Model decides the next step search grep read results Search theknowledgebase Grepsource Read asource file can answer, or the turnlimit forces an answer Answer with sources One turn can call several tools at once.
Handed a fixed context, the model answers from whatever a single retrieval step returned. In the agent loop, a first knowledge-base search runs automatically; then the model decides each turn whether to search again, grep source or read a file, sees the results, and stops when it can answer or the turn limit forces an answer.

What I build