Lesson 18 / 29

Long Prompts: Order and Placement

Place documents and questions so the model uses them well.

The middle gets lost

Models often use information at the start and end of a long prompt more reliably than details in the middle. Practical layout for long inputs: put the documents first, then the question and instructions last (and repeat the key rule briefly if the prompt is very long); label each document with an ID and title; put the most relevant material at the edges; and trim irrelevant text rather than sending everything. Test with a needle-in-a-haystack style check on your own data: hide a known fact at different positions and see when the model misses it.

Test with a planted fact

Hide a known sentence at the start, middle and end of a long input and check that the model finds it each time.

Quick check: Where is it usually best to place the question in a long prompt?

  • After the documents, near the end
  • Hidden in the middle of the documents
  • Never
  • Before the system prompt
Answer

After the documents, near the end — Ending with the question and key rules makes them salient.