context_length_exceeded: Four Fixes, in the Order That Saves You Money
The context limit counts input plus the output you asked for. Work out which one blew the budget before you reach for a bigger model — getting the order wrong is expensive.
Measure before you change anything
Most people hitting this error do not know how many tokens they actually sent. Rough conversions (about 1.3 tokens per English word) are fine for order of magnitude, but to fix it read `usage.prompt_tokens` from the response, or run a tokenizer locally. Without that number every subsequent step is guesswork, and you will probably change the wrong thing.
Remember the output counts too
This is a common misunderstanding. The window holds your input plus what the model is about to generate, so an oversized `max_tokens` eats the space directly. A request whose input is nowhere near the limit will still fail if `max_tokens` is set generously. Check that field first — sometimes dropping it from 4096 to 1024 fixes the error and lowers your bill at the same time.
Conversations: history cannot grow forever
Multi-turn chat sends the entire history by default, so turn fifty costs many times turn one and eventually hits the ceiling. Two options: a sliding window (keep the last N turns) is simplest; summarising older history preserves more at the cost of an extra call. Which to pick depends on how much early context matters to the current answer — for most support-style use cases a sliding window is enough.
Long documents: splitting almost always beats upgrading
Pushing a hundred-page document through in one call means paying for a great deal of text the answer never used, even when the model can technically hold it. Better to segment and combine, or retrieve the relevant passages and send only those. Several small calls usually cost less than one enormous call, and when something fails you re-run one segment rather than the lot.
When a longer window is genuinely the answer
After the first three, some cases remain — a model that must see an entire codebase to reason across it, where any split loses the connections that matter. That is real, and rarer than most people assume. The problem with upgrading first is that it raises your costs without changing the fact that context keeps growing, so you arrive back at the same wall with a larger bill.
Comparing windows across models quickly
If you need to compare how several models handle the same input, an OpenAI-compatible endpoint makes it a one-string change — no SDK swap, no second key. Context limits and per-token prices for each model are listed at cocodot.co/pricing#models, with a machine-readable version at `GET https://cocodot.co/api/ai/models`.
The same text is a different token count on every model
Tokenizers differ between vendors, and between model generations from the same vendor. A prompt that fits comfortably on one model can overflow on another, and non-English text, code and JSON usually cost noticeably more tokens per character than plain English. When you switch models, re-measure instead of assuming the old count carries over, and leave a margin of safety rather than packing the window to the last token.
Agents and tools: the silent growth
In agent loops the biggest contributors are rarely your own prompt. They are tool results (a file read, a search dump, a long log) and the accumulating record of previous steps. Truncate or summarise tool output before it goes back into the conversation, return identifiers instead of full documents when the model only needs to refer to them, and clear finished sub-tasks. Coding assistants that report a size error or say they cannot compress further are telling you the same thing: start a fresh session, carry over only a short summary, and stop feeding whole files when a section will do.
Caching lowers cost, not size
Prompt caching can make a repeated prefix cheaper to process, but it does not shrink the window: cached tokens still count toward the limit. Do not treat caching as a fix for this error. It is a cost tool to use after the fit problem is solved, by keeping the stable part of your prompt (instructions, reference text) at the front and the changing part at the end.
What each fix costs you
| Approach | Cost impact | When to use it |
|---|---|---|
| Trim history and prompts | Lower | Always, and first |
| Split and combine | Slightly higher (more calls) | Long documents, batch work |
| Reduce max_tokens | Lower | When the output never needed to be that long |
| Move to a longer-context model | Significantly higher | Only after the first three |