Lesson 13 / 27

Query Rewriting and Expansion

Turn messy user questions into good search queries.

Users do not write search queries

Real questions are short, vague or depend on earlier turns ("and what about contractors?"). Improve them before searching: (1) condense the chat history and the follow-up into a standalone question; (2) expand with synonyms and related terms; (3) decompose a complex question into several sub-questions and search for each (multi-query); (4) HyDE, where the model drafts a hypothetical answer and you search with that text. Each rewrite adds a model call and can drift from the user's intent, so keep the original query in the search too and measure whether the rewrite actually helps.

Ask better, rank better, send less

Rewrite queries, rerank candidates, diversify results and choose how many chunks to send.

Four levers: rewrite, rerank, diversify, trim.
Figure 4.1 — Rewrite, rerank, diversify and trim.

Condensing a follow-up (prompt template)

A template for turning a follow-up into a standalone question. Not run here.

Given the chat history and the follow-up question, rewrite the follow-up as a
single standalone question that includes all needed context. Do not answer it.

History:
User: How many leave days do employees get?
Assistant: 24 days per year.
Follow-up: and contractors?

Standalone question: How many paid leave days do contractors get per year?

Quick check: Why condense a follow-up question before retrieval?

  • To make the query longer for no reason
  • "and contractors?" alone has no searchable context
  • To avoid embeddings
  • To delete history
Answer

"and contractors?" alone has no searchable context — A standalone question carries the meaning the retriever needs.