Bigger Context Windows Don't Mean Better Answers
Suppliers quote how much material a system can be handed at once as though that were the specification that mattered. Two published results show why it is the wrong thing to buy on: position inside the pile changes accuracy, and advertised capacity is not sustained capacity.
Take a document the system needs. Put it near the top of a long pile of material and ask a question about it. Then move that same document into the middle of the pile, change nothing else, and ask again. The answer gets worse.
That is the finding of Lost in the Middle, work led by Nelson Liu at Stanford, released in 2023 and published in the journal TACL in 2024. It tested two ordinary jobs: answering a question from a set of supplied documents, and looking up one value in a long list. Accuracy, the paper found, "significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models."
The figures, on the systems available at the time. Twenty documents supplied, the right one placed first: 75.8% correct. The same document moved to the middle: 53.8%. Moved to the end: 63.2%. That same system, asked the same questions with no documents at all, scored 56.1%.
Read the last two numbers together. Handing it the right document, buried in the middle, left it worse off than handing it nothing.
Those percentages come from 2023 systems and should not be read as current. The shape has held. Position is a variable that appears on no datasheet and in no proposal.
The number being quoted
The context window is how much material a system can be handed at once. It has grown enormously, and suppliers lead with it. A bigger one genuinely helps: fewer splits, fewer handoffs, fewer chances to lose something between steps. But it describes what a system can hold, not what it will use.
There is a second gap between the figure and the behaviour. In 2024 a team at NVIDIA built a harder test than the usual one, which is a single fact hidden in a long pile. Theirs asked systems to follow a chain across the material and to count things spread through it. Seventeen systems were run through it, all advertised as handling a given quantity. Only half still performed acceptably when actually given that much. Their paper is titled, pointedly, "What's the Real Context Size of Your Long-Context Language Models?"
That was 2024 and the systems have moved on since. What it establishes is the category of problem: the advertised figure is a maximum the system will accept, not a standard it will sustain.
The saving we once found buried in a client contract was not concealed. It was sitting in plain sight on page 247 of a 300-page document. Length is what hid it. Nobody had read that far.
What the engineers built instead
Nobody who builds these systems for a living solves the problem by pasting more in.
In October 2023 a Berkeley group led by Charles Packer published MemGPT, since renamed Letta and now a company. Its proposal was borrowed from how computers have managed memory for fifty years: a small working set, a large store outside it, and deliberate movement between the two. Bring in what this step needs. Put it back afterwards.
Google's team arrived at the same place from another direction. In December 2025 Hangfei Lin, a technical lead there, put it in one line: "Context is a compiled view over a richer stateful system." What the system sees is assembled fresh for each task out of the real sources: the record of what has happened, a longer-term store, and named files it fetches when it needs them. The view is built, not accumulated.
Manus, a company building systems that carry out multi-step work, published its production notes in July 2025. It uses ordinary file storage as memory held outside the window, "unlimited in size, persistent by nature, and directly operable by the agent itself", and every shortening it does is reversible. Cost is part of the reason. On July 2025 prices, material the system had already been given cost about a tenth as much to reuse as material sent fresh. Re-sending everything every time is expensive as well as unreliable.
We ran the negative version of this before either of those papers mattered to us. A capable model failed a real workflow because of context limits, no file access, and no memory between steps.
What to ask instead
None of this contradicts the case for analysing everything rather than sampling. Breadth across a whole engagement is a different question from what a system can use in a single pass.
Capacity is cheap to quote. The things that are expensive to fake sit one layer down, in how the material gets assembled at the moment of the task.
Two honest limits. For a short self-contained job, a big window and a single pass is both the right answer and the cheap one. And a small working set with a large store is more moving parts: more to build, more to maintain, more to get wrong.
This is a question about what a system can reach, rather than how you word the request. The instructions side of it is settled elsewhere, in why one of your documents should be shrinking while the other grows.
Capacity is a ceiling. It tells you what will fit, not what will be found. Nobody quotes the second one as a number, because it is not one.
Related Articles
What Context Engineering Actually Means
A label arrived in June 2025 for an old problem: deciding what an AI system can see at the moment it does a job. Underneath the term are four decisions, and most of what you will be sold is a combination of them. Here is the vocabulary, the evidence behind it, and the two questions to put to a supplier.
Your AI Rules Should Be Getting Shorter
Anthropic deleted more than 80% of the instructions it gives its own AI tool and lost nothing. A peer-reviewed paper found the opposite: detail beats summary. Both are right, because two very different things get called context, and only one of them should be growing.
The Hard Part Isn't the AI
How we built an agentic workflow that turns months of manual analysis into auditable, presentation-ready reports — and why understanding the domain was harder than building the AI.
The Hidden Costs in Your Vendor Proposals
We found £200,000 hidden in a vendor proposal - not through negotiation, but by understanding what normal looks like in the industry and spotting what was missing.