What is a context window?

In practice

Large context windows have made it tempting to paste entire documents into every request. It works, and it is slow and expensive, and accuracy often falls because the relevant passage is buried among thousands of irrelevant ones. Retrieving the right few passages usually beats supplying everything, even when everything would fit.

Where it fits in a build

Context limits shape architecture decisions early: whether you need retrieval, how conversation history is summarised, and how long documents are processed. A system designed as though context were free will be rebuilt the first time it meets a two-hundred-page tender.

Common mistakes

• Treating a large context window as a replacement for retrieval, then paying for it on every request.
• Letting conversation history grow without summarising it, so long sessions get slower, dearer and less accurate.
• Forgetting that the answer needs room too, so long inputs produce truncated outputs.

Related terms

• Large language model (LLM)
• Retrieval-augmented generation (RAG)
• Embedding

Working on something that involves this?

ASTACKRA designs and builds AI systems, automation and custom software for businesses that need technology shaped around their own workflow. If this term turned up in a proposal and you want a straight answer about whether it applies to your situation, ask us — no obligation, and we will tell you if the answer is no.