Context Engineering for Large-Scale Product Search

Large language models are increasingly being used as components of complex agentic systems rather than as standalone conversational assistants. In this setting, the challenge shifts from designing prompts to designing context. Instead of asking "How should we phrase the instruction?", the question becomes "What information does the model need to accomplish the task?"

In this talk, I will present how we approached context engineering for large-scale product search at Criteo by fine-tuning an LLM to generate high-quality retrieval context in the form of alternative search queries. Instead of constructing a training dataset from manually curated or LLM-generated query expansions, we leverage Criteo's unique advantage: its rich commerce data. This enables us to derive supervision directly from the behavior of the retrieval system itself, allowing the model to optimize for downstream retrieval quality rather than text similarity.

The talk will discuss the practical challenges encountered along the way, including constructing high-quality training datasets, designing reliable LLM-based evaluation pipelines, optimizing against downstream retrieval metrics, and understanding how the learned policy becomes tightly coupled to the underlying product embedding space.

Although the case study focuses on product search, the techniques apply broadly to agentic systems in which an LLM's primary role is not to generate the final answer, but to construct high-quality context for downstream components.