context_length_exceeded: Four Fixes, in the Order That Saves You Money
The context limit counts input plus the output you asked for. Work out which one blew the budget before you reach for a bigger model — getting the order wrong is expensive.
Measure before you change anything
Most people hitting this error do not know how many tokens they actually sent. Rough conversions (about 1.3 tokens per English word) are fine for order of magnitude, but to fix it read `usage.prompt_tokens` from the response, or run a tokenizer locally. **Without that number every subsequent step is guesswork**, and you will probably change the wrong thing.
Remember the output counts too
This is the most common misunderstanding. The window holds your input **plus** what the model is about to generate, so an oversized `max_tokens` eats the space directly. A request whose input is nowhere near the limit will still fail if `max_tokens` is set generously. Check that field first — sometimes dropping it from 4096 to 1024 fixes the error and lowers your bill at the same time.
Conversations: history cannot grow forever
Multi-turn chat sends the entire history by default, so turn fifty costs many times turn one and eventually hits the ceiling. Two options: a **sliding window** (keep the last N turns) is simplest; **summarising** older history preserves more at the cost of an extra call. Which to pick depends on how much early context matters to the current answer — for most support-style use cases a sliding window is enough.
Long documents: splitting almost always beats upgrading
Pushing a hundred-page document through in one call means paying for a great deal of text the answer never used, even when the model can technically hold it. Better to segment and combine, or retrieve the relevant passages and send only those. **Several small calls usually cost less than one enormous call**, and when something fails you re-run one segment rather than the lot.
When a longer window is genuinely the answer
After the first three, some cases remain — a model that must see an entire codebase to reason across it, where any split loses the connections that matter. That is real, and rarer than most people assume. The problem with upgrading first is that it raises your costs without changing the fact that context keeps growing, so you arrive back at the same wall with a larger bill.
Comparing windows across models quickly
If you need to compare how several models handle the same input, an OpenAI-compatible endpoint makes it a one-string change — no SDK swap, no second key. Context limits and per-token prices for each model are listed at cocodot.co/pricing#models, with a machine-readable version at `GET https://cocodot.co/api/ai/models`.
What each fix costs you
| Approach | Cost impact | When to use it |
|---|---|---|
| Trim history and prompts | **Lower** | Always, and first |
| Split and combine | Slightly higher (more calls) | Long documents, batch work |
| Reduce max_tokens | Lower | When the output never needed to be that long |
| Move to a longer-context model | **Significantly higher** | Only after the first three |