Can a gateway prove it does not store my prompts?

Not to the standard of a cryptographic proof. What it can do is show billing records that contain no content field, state retention in binding terms, and answer directly in writing. If your risk level needs more than that, the answer is a direct account with a data agreement.

Is the official API automatically safer?

It removes one hop and lets you sign an agreement directly, which matters for regulated data. It does not mean zero retention — the provider still has its own policy, and reading it is worth the ten minutes.

SecurityUpdated 2026-08

Does an API Gateway Log Your Prompts? What You Can Verify and What Nobody Can Promise

Split the path into three layers before worrying. Most people worry about the middle one, but the layer nobody controls is the last one — and no gateway can make promises on its behalf.

TL;DR: Three layers: (1) your client → (2) the gateway → (3) the upstream model provider. Most concern lands on layer 2, but the layer nobody in the middle controls is layer 3. Layer 2 you can actually verify — whether they store content, what appears in your billing detail, and whether they will put retention in writing. Layer 3 no gateway can promise anything about, because the gateway is itself a customer of that provider; retention there is set by the provider's own policy. So the decision rule is simple: if a leak of this data would cost you money or breach a contract, do not route it through any intermediary — go direct and sign a data agreement. For everything else, pick a provider that will state in writing what it stores. cocodot does not store prompt or response content — billing records carry the model name and token counts only, verifiable line by line — and equally, we cannot make commitments on an upstream provider's behalf. That last sentence is true of every gateway, including the ones that do not say it.

Work out which layer you are actually worried about

The question does a gateway log my prompts is usually a proxy for will this data end up somewhere it should not. Those are different questions with different answers per layer. Your own client is fully yours. The gateway is partly verifiable. The upstream provider is out of reach entirely when you go through an intermediary. Naming the layer first saves you from asking a question the answer cannot satisfy.

Four things you can check about the gateway

Billing detail. If billing shows only model and token counts, there is no content field to leak in that record. Ask to see a real line item. Written terms. A provider willing to state retention in its terms has made a commitment you can hold them to; one that only says it verbally has not. Response headers and docs. Some gateways expose upstream request identifiers, which tells you something about what is passed through. A direct question in writing. The answer, and how readily it is given, are both signal.

Why layer three is nobody's promise to make

A gateway buys capacity from the model provider like any other customer. It is bound by that provider's retention policy and it cannot extend terms it does not hold. Any gateway claiming the upstream provider retains nothing is either repeating something they cannot verify or hoping you will not check. The honest position is: we control our layer, here is what we do at it, and layer three is governed by the provider's published policy — which you can read yourself.

What cocodot does at our layer

We do not store prompt or response content. Billing records contain the model identifier, token counts, and timestamp — that is what makes per-call reconciliation possible, and there is no content column to expose. You can verify this against your own billing detail rather than taking our word for it. What we cannot do, and will not claim, is speak for what the upstream provider retains.

Data that should not go through any intermediary

Draw the line by consequence rather than by category. If a leak would breach a contract, trigger a regulatory obligation, or expose personal data you are responsible for, the extra hop is not worth the convenience — use a direct account with a signed data agreement, and accept the payment friction that comes with it. Everything below that line is a normal engineering tradeoff. Being clear about which side you are on is more useful than trying to make one setup serve both.

While you are here, bound the key risk too

Data retention is one exposure; a leaked key is the other, and it is more common. Separate keys per project, a spending cap per key, and quarterly rotation cover most of it. The two concerns share a habit: decide in advance what the worst case costs you, and cap it, rather than relying on nothing going wrong.

What can be verified at each layer

Layer	Who controls it	Can you verify it	How
Your client	You	Yes	Your own code and logs
The gateway	The gateway	Partly	Billing detail, written terms, request headers
Upstream provider	The provider	No, not through a gateway	Only via a direct account and agreement