How do I know if an API relay is serving the real model?

Run three checks: verify identity and push a very long context to see whether it truncates; run 3–5 hard prompts side by side against the official endpoint; then send 20–50 requests and record latency and failures. If all three line up, you're likely getting what you paid for. Or run probe.cocodot.co, which does it for you.

Can I trust a model that tells me which model it is?

Only as a hint. A system prompt can make a small model introduce itself as a flagship, so identity has to be corroborated by the context-window and capability checks. A model can claim to be Opus; it can't fake solving Opus-level problems.

Does a very cheap gateway always mean something is wrong?

Not necessarily, but it earns an extra round of testing. Flagship inference has a hard upstream cost, so pricing well below it usually comes from one of three places: silent substitution, resold credits of unclear origin, or collecting prepaid balances at a loss. Test with a small amount before you concentrate spend anywhere.

Is there a ready-made tool for this?

Yes — cocodot's checker is open source, with a hosted version at probe.cocodot.co. It takes any gateway's base_url and key, runs the probes, and doesn't store the key. Pointing it at cocodot itself is a supported and expected use.

AI APIUpdated 2026-08

Is Your API Gateway Really Serving Claude Opus? How to Verify in 5 Minutes (2026)

Some relays quietly route "Opus" or "GPT-5.5" traffic to a cheaper model. You don't have to take anyone's word for it — three checks (identity and context, capability benchmarks, latency and success rate) tell you what you're actually being served.

TL;DR: "Downgrading" is when a gateway advertises Claude Opus or a GPT flagship but routes your request to something cheaper — you pay flagship rates for a lighter model and, on easy prompts, never notice. You can settle it yourself in about five minutes with three checks. First, identity and context: ask the model what it is, then feed it a genuinely long document (tens of thousands of tokens) and see whether it takes the whole thing without truncating or erroring — a smaller substitute usually can't hold the context window it claims. Second, capability: keep three to five hard prompts (multi-step reasoning, long code generation and debugging, multi-step math) and run them against both the official endpoint and the gateway; the gap shows up on hard problems, not easy ones. Third, latency and success rate: fire 20–50 requests on one key and record time-to-first-token, failures and truncations — silent timeouts and burnt tokens quietly eat a budget. If you'd rather not build the harness, cocodot's downgrade checker is open source and hosted at probe.cocodot.co: paste any gateway's base_url and key, and it runs the probes for you without storing the key. It works on us too — and a provider's willingness to be tested tells you more than any no-downgrade promise.

1. What "downgrading" actually means

A gateway lists Claude Opus or a GPT flagship, charges you accordingly, and routes some or all of that traffic to a cheaper model. The economics are obvious: flagship inference is expensive, and the margin between what you pay and what a small model costs is large. It's worth being precise, though — not every substitution is dishonest. Reputable aggregators openly document fallback routing when a provider is at capacity, and publish which upstream served each request. The problem is the gateways that do it silently.

2. Check one: identity and context window

Ask the model to state what it is and which version. Treat the answer as weak evidence only — a system prompt can make any model claim to be Opus. The harder test is the context window: send a document of tens of thousands of tokens and see whether it is ingested completely, without truncation or an error. A cheaper substitute often can't hold the window the listing advertises, and long input is where that shows.

3. Check two: a small capability benchmark

Keep three to five prompts with real discrimination between model tiers: one long multi-step reasoning problem, one long code generation-and-debug task, one multi-step math problem. Send each to the official endpoint and to the gateway, and compare answer quality and consistency. Easy prompts prove nothing — every model gets those right. The gap opens on long reasoning chains and long code.

4. Check three: latency and success rate

Send 20–50 requests on a single key and record time-to-first-token, the overall success rate, and any unexplained failures or truncations. This matters most for production: intermittent timeouts and tokens charged on failed requests drain a budget without ever showing up as an obvious incident. A gateway with unstable success rates isn't safe to put behind live traffic, whatever model it is serving.

5. Use the open-source probe instead of building your own

cocodot published its downgrade checker as open source, with a hosted version at probe.cocodot.co. Paste in any OpenAI-compatible gateway's base_url and an API key, and it runs the probes and reports what it found; the key is used for that request only and is not stored. The method is public precisely so you can point it at us as readily as at anyone else.

6. Choose a provider that lets you test it

The practical decision rule is not "who promises not to downgrade" — everyone promises that. It's who publishes enough for you to check: model identity, context window, pricing, and a real endpoint you can benchmark before you commit volume. Fund a small amount, run the three checks, and scale only after your own numbers come back clean. That advice holds whichever gateway you end up choosing.

Three checks: what to run, and what a substituted model looks like

Check	How to run it	A real flagship	A downgraded model
1. Identity + context window	Ask what model it is; then send a very long document	Consistent identity, ingests the full context	Vague identity, truncates or errors on long input
2. Capability benchmark	3–5 hard prompts, same prompt against the official endpoint	Solves hard problems, stable style	Fine on easy prompts, falls apart on hard ones
3. Latency + success rate	20–50 requests on one key; record TTFT and failures	Acceptable TTFT, steady success rate	Random timeouts, truncation, tokens burnt on failures
Shortcut	probe.cocodot.co with any base_url + key (key not stored)	Runs all probes and reports	— (nothing to set up)