Query Engines and Response Synthesis

Retrieve, then synthesise

index.as_query_engine() wraps retrieval plus a response synthesizer: it retrieves the top nodes, builds a prompt with them and asks the LLM for the answer. The returned Response includes source_nodes so you can show citations. Response modes control how context is combined: compact stuffs as much as fits, refine improves an answer node by node, tree_summarize summarises hierarchically (good for summaries of many nodes). You can add node postprocessors (similarity cutoff, reranker, metadata filters) between retrieval and synthesis, and use as_chat_engine() for multi-turn chat with memory.

A query engine with mock models, run

I ran this offline in a Python virtual environment with langchain-core 1.6.6, langchain-text-splitters 1.1.2 and llama-index-core 0.14.25. No API key or network call is needed because a fake model or a toy embedding stands in for the real one. MockLLM and MockEmbedding let the engine run offline, so the answer text is a placeholder, but the structure is real: the result is a Response object with one source node holding the policy text.

from llama_index.core import Document, VectorStoreIndex, Settings
from llama_index.core.embeddings import MockEmbedding
from llama_index.core.llms import MockLLM

Settings.llm = MockLLM(max_tokens=20)
Settings.embed_model = MockEmbedding(embed_dim=8)

index = VectorStoreIndex.from_documents([Document(text="Employees get 24 days of paid leave per year.")])
engine = index.as_query_engine(similarity_top_k=1)
response = engine.query("How many leave days?")
print(type(response).__name__)
print("source nodes:", len(response.source_nodes))
print(response.source_nodes[0].node.get_content())

Output:

Response
source nodes: 1
Employees get 24 days of paid leave per year.

Quick check: Why does the Response include source_nodes?

• To train the LLM
• So you can show which passages supported the answer
• To compress the index
• To remove citations