How to build a context factory for analytical agents

The Context Factory

Presenters: Emma Wagner, Jules Guillot, Yochan Khoi — the three context engineers who built the factory

Abstract

Intro

Everyone now knows you need a "context layer" to make an AI agent useful on top of your data. Far fewer people talk about the harder problem: a context layer is never done. Data models change, definitions drift, new questions expose gaps, and yesterday's perfect answer is today's hallucination. The Context Factory is our answer to that problem — not just how we built a context layer for an analytical agent, but how we set up the machinery to keep improving it continuously, with as little human effort as possible.

This talk walks through the pillars we automated, and what each one buys us.

1. Modeling made for and by agents

Modeling by agents — an autonomous worker across the model lifecycle:

• Authoring new models
• Optimizing or updating existing ones
• Writing valid documentation
• Running a series of checks before promoting to production

Modeling for agents — shifting modeling principles for agent consumption:

• BI-oriented modeling grouped many concepts into a small set of complex tables, on one principle: for every query, there should be one straightforward table to use.
• That constraint no longer applies once the agent writes the queries. It already knows which tables exist and how to combine them, so a handful of well-scoped tables is an advantage, not a burden.
• Each model becomes a deliberate choice between two shapes:
• Performance (for large models)
• Entity-level (near raw data)

2. Knowledge is the product — how it's structured and maintained

The five building blocks of the graph, and why each is its own first-class object:

• Skills — runnable, step-by-step operational runbooks
• Topics — recursive domain knowledge that carries guardrails and query conventions
• Metrics — named, owned SQL with metadata and drill-down relationships
• Tables / Columns — the grounded semantic layer, with types and curated dependencies
• Knowledge graph — the typed edges that turn these from a folder of docs into a navigable structure

3. The other half of the game — retrieving the context

A great context layer is worthless if the agent can't find the right piece at the right time.

• Progressive disclosure — the main agent only ever sees top-level catalogs (skill & topic names); depth is fetched on demand.
• Sub-agent delegation — a dedicated context sub-agent navigates the graph node-by-node and returns a structured "Context Pack" (the route, the metric SQL or verified tables, the guardrails, the paths visited).
• Controlled fan-out — LLM-ranked skill search, bounded tool budgets, and small parallel waves instead of speculative crawling.

4. Auto-improve — closing the loop, with production-like velocity

The heart of the "factory."

• Automated PR generation — feedback (a Slack thumbs-down, a provide_feedback signal, an outside request) becomes a real pull request to the context repo, authored by our coding agent — no engineer in the middle.
• CI/CD guardrails — every auto-generated PR is validated the same way a human's would be: schema checks, BigQuery dry-runs on the SQL, ownership validation, and code review — so the agent can't quietly break production.
• Evaluation, offline and online — a versioned QA dataset and custom evaluators (expected behaviour, retrieval accuracy, skill selection) gate changes before merge, and a scheduled weekly production eval watches the live agent for regressions.

5. Education — making the company a contributor, not just a consumer

The factory only scales if everyone can feed it.

• In-app authoring surfaces (skill editor, automation builder, agent builder) that let non-engineers create and edit context safely.
• The "ask the agent to update itself" pattern: how a normal user requests a new skill or a fix in plain language and gets a reviewed PR back → handovers to coding and review agents.
• How we teach the building blocks so the context layer improves from the edges of the company, not just from the team that built it → FDE model.

Takeaway: A context layer isn't a dataset you ship once — it's a factory you build, with feedback, automation, validation, and evaluation wired into a loop. We'll show you the pieces, and which ones are worth automating first.