Is an API Gateway Cheaper Than Going Direct? Official API vs Relay, Scenario by Scenario (2026)
Official API or relay? Prompt caching, batch discounts and long prompts decide it. Break-even arithmetic and a test you can run on your own traffic.
1. Start from the shape of your traffic, not the vendor's headline
Articles comparing gateways with direct provider access tend to quote one number per model and declare a winner. That number is the base input rate, and it is rarely what your bill is made of. A real month of usage splits into at least three lines: input read from the prompt cache, input processed fresh, and output. Each line has its own multiplier on the official API, and a relay may price the same lines differently. Before comparing anything, export a week of usage from whichever route you use today and total those three lines per model. If your logs record cache_read_input_tokens and cache_creation_input_tokens (the field names Anthropic's API returns in usage), you already have the split. If they show zero everywhere, possible reasons include prompts that are not cache-friendly or caching that is not switched on; sort that out before choosing a route.
2. How official prompt caching is priced
Anthropic's pricing page lists prompt caching as multipliers on the base input rate. A write to the 5-minute cache costs 1.25x; a write to the 1-hour cache costs 2x; a cache hit (which also refreshes the entry) costs 0.1x, except 0.05x on Claude Opus 5.5 and Claude Sonnet 5.5 and 0.025x on Claude Fable 5.1 and Claude Mythos 5.1. The same page notes that a 5-minute write pays for itself after one cache read and a 1-hour write after two. cocodot bills cache hits at half the input price and cache writes at 1.25x the input price (2x for the portion written to the 1-hour cache), the same structure as the official pricing. The write side therefore lines up; the hit side does not. On a cache-heavy workload the official hit multiplier is several times lower, and the complete calculation must include each route’s actual base rate. Pricing multipliers change, so re-read the official prompt caching table rather than copying these into a spreadsheet permanently.
3. The break-even arithmetic for cache-heavy agents
Take one steady-state turn of an agent: C tokens are read from cache and U tokens are new input. Ignoring output for a moment, the official cost is (h × C + U) × P, where h is the official hit multiplier and P the official input rate. A relay that bills hits at 0.5x costs (0.5 × C + U) × R, where R is its input rate. The relay costs less on input only when R / P is below (h × C + U) / (0.5 × C + U). With an illustrative 40,000 cached and 10,000 new tokens per turn and h = 0.1, that ratio is 14,000 / 30,000, roughly 0.47. With h = 0.05 it falls to 12,000 / 30,000, or 0.4. These token counts are arithmetic examples, not measured traffic. The point is the direction: the larger C is relative to U, the lower a relay's input rate would need to be to win on input, and on cache-dominated agents the official route usually comes out ahead. Then add output separately, because output is billed the same way on every turn.
function relayBreakEven(cachedTokens, newTokens, officialHitMultiplier, relayHitMultiplier = 0.5) {
const official = officialHitMultiplier * cachedTokens + newTokens;
const relay = relayHitMultiplier * cachedTokens + newTokens;
return official / relay; // relay input rate must be below this share of the official rate
}
console.log(relayBreakEven(40000, 10000, 0.1).toFixed(2));No overseas card needed: cocodot's OpenAI-compatible API — switch your base_url to call the models on sale.
Top up via Alipay / WeChat Pay (RMB), USDT / USDC · billed per token · switch your base_url to connect
4. Does prompt caching work through a relay? Test it yourself
Caching only saves money if the cache actually hits, and through a relay that depends on two things you cannot see from outside: whether it forwards your cache_control markers unchanged, and whether consecutive requests from your key reach the same upstream cache. Test it rather than asking. Build a request with a long, fixed system prompt well above the model's minimum cacheable length, mark it for caching, and send it three times within a few minutes. On the first call you should see cache creation tokens; on the second and third, cache read tokens should appear and uncached input should drop. If the second and third calls look like the first, the cache did not hit on that route. Repeat at a quieter and a busier hour, because routing can vary. Keep the raw usage objects from each call; they are the evidence you compare against the charge on your statement. The general caching setup is covered in /hub/prompt-caching-guide, and model-substitution checks in /hub/ai-relay-model-downgrade-verify.
5. Output-heavy and uncached work: compare rates model by model
Prompt caching prices the reused input portion. For long-form generation, translation, rewriting or report writing, compare output rates while still including all input, cache reads and cache writes. Different user prompts can still share system instructions or tool definitions, so inspect usage instead of assuming nothing is cached. This is where a relay's per-token rates carry the most weight. Listed Claude / GPT / Gemini rates are at or below the official list price; check the pricing page for each model's current rate. Compare the same model generation on both sides; a lower rate on an older model is not a saving if your quality bar needs the newer one. A useful self-check: divide your monthly input tokens by output tokens. A ratio in the tens with large cache reads is agent-shaped; a ratio near one or below is generation-shaped, and the rate comparison in this section decides it.
6. Discounts and multipliers that only exist on the official API
Three official pricing rules are easy to leave out of a comparison. First, the Batch API gives a 50 percent discount on both input and output for asynchronous requests, and the official page says it stacks with prompt caching. If your jobs can wait, that discount is only available through a route that exposes a batch endpoint, so check a relay's documentation before assuming parity. Second, data residency: on Claude 4.6 and later models, pinning inference_geo to us applies a 1.1x multiplier to every token category; the default global routing uses standard pricing. Third, long context: the official page states that Claude 4.6 and later models, except Claude Haiku 5.5, bill the full 1M-token context window at standard rates, while Claude Haiku 5.5 charges higher prices once a prompt passes 100,000 tokens, counting cache reads and writes toward that length. When you compare a relay, ask for its published tier table for the same model; if it cannot show one, plan around the higher figure.
7. The costs that are not on any price list
Several deciding factors never appear in a per-token comparison. Access policy: Anthropic's Supported Regions Policy, effective November 12, 2026, says its apps, API and Claude models offered through cloud platforms and authorized resellers are available only in the listed countries and regions, and it covers use by entities headquartered or majority-owned in unsupported regions. Read it for where you and your company are, whichever route you pick. Payment: if the official console declines your local card, the cheapest rate is irrelevant until you can pay. Ask your card issuer and the merchant to confirm the reason for the decline. Engineering: one OpenAI-compatible key across Claude, GPT and Gemini removes an adapter per vendor, while going direct means you use each vendor's own features and documentation without depending on how a relay implements them. Data path: a relay is an extra party that handles your prompts, so read its terms. Control: a prepaid balance is not a hard cap: requests already in flight when the balance runs out may still be billed; your statement is the record.
8. A decision checklist, and where cocodot fits
Run the checklist in order. 1) Total last week's usage into cached input, uncached input and output per model. 2) If cached input dominates, compute the break-even ratio from section 3 with the current official multiplier; for cache-dominated agents, expect the official route to win and use it if you can pay and your region is supported. 3) If uncached input and output dominate, compare the two pricing pages model by model. 4) Add the official-only levers from section 6 that apply to you. 5) Weigh the non-price factors in section 7. Splitting traffic is also an option: cache-heavy agents direct, generation and multi-vendor work through one OpenAI-compatible endpoint. cocodot's endpoint is https://cocodot.co/api/ai/v1 with official model names or cocodot model codes, and per-model rates are public at cocodot.co/pricing without sign-up. For a wider view of relay selection, see /hub/how-to-choose-ai-api-relay; for spend guardrails, see /hub/api-relay-runaway-risk.
Which side the arithmetic usually favors, by traffic shape
| Your traffic | What decides the cost | Where to look |
|---|---|---|
| Agent loops resending a long system prompt and tools | Cache-hit multiplier: official 0.1x or lower vs relay 0.5x of its own rate | Official pricing page — prompt caching table |
| Every prompt different (translation, classification) | Uncached input and output rate per model | Both pricing pages, model by model |
| Output-heavy generation | Output rate; caching never applies to output | Both pricing pages, output column |
| Overnight jobs that can wait | Official Batch API halves input and output | Whether your relay exposes a batch endpoint at all |
| Very long prompts | Whether the model has a long-context price tier | Official long-context pricing section |
| Card declined or unsupported country | Not a price question — access and policy first | Anthropic Supported Regions Policy |