cocodot
← Back to guides
Local card declined for overseas AI? cocodot: one card + one key
AI APIUpdated 2026-10

Batch AI API Jobs From China: Durable Queues, Recovery and Agent Acceptance Tests

Run repeatable content and agent workloads with durable job states, bounded concurrency, deduplicated delivery and cost reconciliation.

TL;DR: For recurring batch AI work, build recoverable jobs before increasing concurrency. Keep a business job separate from its model attempts: generating one approved product description may take several attempts, but the application should publish only one accepted result. An interrupted connection should leave an identifiable record rather than automatically restarting the entire import. Measure time to first response for interactive work, total duration for background work, validation outcomes and cost per delivered artifact. cocodot provides an OpenAI-compatible gateway at https://cocodot.co/api/ai/v1 with token-based billing. Durable queues, business deduplication, budget reservations and recovery logic in this guide are responsibilities of your own application. New users must complete identity verification and a first top-up before API calls; the trial credit is usable after that top-up. Here, batch processing means your workers making ordinary model calls; it does not imply a separate asynchronous Batch API or a special bulk rate. The goal is a repeatable operational test: stop a worker midway through a small run, restart it, and account for every job without losing accepted results or repeating customer-facing actions.

1. Assign one job identity and many attempt identities

Choose a stable business identity from inputs such as product identifier, target language and template version. Assign a separate attempt identifier whenever a worker makes a model call. Your job record holds input version, state and final artifact location. Each attempt holds model, start and end times, error category and available usage. A retry keeps the business identity and adds an attempt; it is not a new customer request. This lets you price a delivered description or support draft across all generation steps. Keep personal information and full prompts out of identifier strings. Store the references and versions needed for recovery instead. These identifiers are maintained by your application, not a claim that a custom request header makes the model call idempotent or prevents duplicate generation charges.

Suggested application records, not a platform job API
job: job_id, input_version, state, result_ref
attempt: attempt_id, job_id, requested_model, started_at, ended_at, usage, charge_state
delivery: job_id (unique), accepted_result_ref, delivered_at

queued -> claimed -> attempted -> validated -> delivered
interrupted attempt -> unresolved -> reconcile -> retry_or_review

2. Test delivery outcomes alongside response timing

Preserve the principle of measuring before launch, but distinguish interactive and background workloads. Interactive work needs an observed time to first response; background work needs total duration, queue age and accepted artifacts. Run a sanitized set of short, long, multilingual and malformed inputs with fixed acceptance conditions. Increase concurrency gradually and inspect slow tasks separately from the average. A normal HTTP response still needs field validation, factual checks and business review where appropriate. Retain the test date, model identifier and configuration so the next run is comparable. A small test tells you about that test, not guaranteed capacity or a long-term latency commitment. Set your own release criteria and confirm that workers stop admitting more work when the queue becomes unhealthy.

3. Treat an interrupted attempt as unresolved until checked

A client that did not receive a complete answer does not know from that fact alone whether processing occurred. Keep an unresolved attempt record with its start time and any available request reference. Reconcile the usage and charges you can inspect before deciding on another attempt under your policy. Leave accepted jobs complete and recover only eligible unfinished jobs rather than rerunning the whole batch. If an answer arrived but was not published, validate and persist that artifact before generating again. Separate invalid input, authorization, insufficient funds and temporary errors because their recovery actions differ. /hub/ai-api-errors-china-fix provides related troubleshooting categories. Never turn every timeout into an automatic zero-cost record or an unlimited retry instruction; uncertainty is a state that needs an owner and a resolution path.

No overseas card needed: cocodot's OpenAI-compatible API — switch your base_url to call the models on sale.

Top up via Alipay / WeChat Pay (RMB), USDT / USDC · billed per token · switch your base_url to connect

4. Use leases and conditional writes in your own queue

One suggested design is to give each claimed job an owner and an expiry, renewed according to your application policy while work continues. Only an expired claim without a valid result becomes eligible for recovery. At final persistence, enforce a unique business job identity or a conditional state update so duplicate workers cannot both publish a result. Apply separate deduplication to business tools, especially sending messages, changing inventory or publishing content. A model may generate twice during recovery while the business action must still happen once. This is an application design recommendation, not a guarantee from the gateway. Before launch, deliberately terminate a test worker between generation and persistence. Confirm that a replacement either uses the saved artifact or records a new attempt without repeating an already completed business action.

5. Count retry layers before raising the worker count

Give each job a total deadline and attempt limit. Cap active calls across the service and within each tenant, and separate long background jobs from short interactive work. When a temporary error justifies retrying, delay with jitter and respect applicable waiting guidance. Stop dispatch and address the cause for authorization, parameter or balance problems. Inspect the client library, workflow framework and worker for overlapping retry loops. One place should own the job deadline and cumulative allowance so nested loops cannot quietly multiply attempts. If higher concurrency fails to reduce queue age while individual durations rise, stop expanding and investigate. See /hub/api-429-rate-limit-concurrency-china for concurrency troubleshooting. There is no fixed capacity number in this guide; use measurements from your own permitted workload and release criteria.

6. Make the agent tool loop recoverable too

cocodot accepts function tools only. Search, code execution or business lookups run in your own controlled process, which returns tool results to the model. For each tool action, retain the job identity, tool name, validated arguments and business completion state. Recovery should inspect that state before executing the action again. Losing the last model response is not sufficient reason to resend a customer message or repeat an inventory update. Bound model steps, tool steps, elapsed time and total budget, preserving useful state for an operator when a limit is reached. Validate and limit tool output before adding it to the next context. /hub/agent-api-cost-runaway discusses related spending guardrails. For a product serving multiple customers, combine these job states with the backend boundaries described in /hub/saas-multi-model-api-integration.

7. Reconcile batches using settled, active and unknown work

Aggregate settled charges, active budget reservations and unresolved attempts under the batch identity. Join every generation and checker attempt to its business job before calculating the cost of delivery. Admission should consider all three categories, rather than only the charges from completed calls. When the cocodot balance runs out, new requests are refused and in-flight requests may still incur charges, with fallback estimates used when usage is missing. Balance depletion is therefore not an exact stop mechanism for every active job. If usage is missing in an application log, retain an unresolved entry and reconcile available evidence; visible text length is not a reliable substitute for a charge. A batch with no accepted outputs has produced no accepted delivery, even if its requests were inexpensive. Faster dispatch can generate rejected artifacts faster, so cost reconciliation must remain connected to validation outcomes.

8. Release after a complete interruption and recovery drill

Register, verify email and complete identity verification, then prepare the balance and create a key for your intended workload. Use the OpenAI-compatible base URL https://cocodot.co/api/ai/v1 with Authorization: Bearer authentication. Keys can be named and individually revoked, with at most 10 active keys per account. The trial credit is usable after your first top-up. Funding options are Alipay / WeChat Pay (RMB), USDT / USDC. Listed Claude / GPT / Gemini rates are at or below the official list price; check the pricing page for each model's current rate. Run a small set through claim, call, validation, persistence, restart and accounting before expanding. For persistent problems, https://cocodot.co/docs points to in-console support. Provide sanitized timing, model and error categories, keeping credentials and customer source text out of the support material.

Interruption pointSuggested application stateCheck before recovery
Before dispatchQueuedInput version and budget admission
After send, no complete responseUnresolved attemptAvailable usage and result evidence
Response received, not deliveredAwaiting validationArtifact matches the business job
Validated and publishedCompletePrevent duplicate business delivery

FAQ

Does batch processing mean sending every request at once?

Use your own queue to bound active calls and separate long background jobs from interactive work. Increase concurrency gradually while checking queue age, duration and accepted outputs. Release against your own measured requirements.

Should I restart a whole import after an API timeout?

Keep completed jobs complete. Preserve the interrupted attempt, inspect available usage and results, then recover only eligible unfinished work under a bounded policy. Repeating the entire import can create duplicate generation and duplicate delivery.

Is this a separate asynchronous Batch API?

This guide describes application workers making ordinary model requests. It does not claim a separate asynchronous Batch API, a bulk rate or platform-managed recovery. Implement job ownership, deduplication and retry policy in your application.

How can a team attribute costs to individual jobs?

Record each attempt and join generation, validation and retry charges under one business job identifier. Keep unresolved attempts visible and track accepted delivery separately from request count.

Can an agent simply repeat a tool call after a crash?

First inspect whether the business action completed. Validate arguments and deduplicate actions in your own service, with limits on steps, time and spending. A lost response does not establish that the underlying action failed.

About cocodot

cocodot is a payment and AI access service for developers and cross-border teams in mainland China. It provides US-BIN virtual cards issued by a licensed institution — used to pay for overseas subscriptions and ad accounts — and an OpenAI-compatible AI API gateway for calling Claude, GPT and Gemini from within mainland China. Both share one wallet, funded by Alipay / WeChat Pay (RMB), USDT / USDC and accounted in USD. Card: $9.9 to open, 3% to load, $1 per active card per month; spending: $0.60 settlement fee on purchases under $20; a corresponding fee applies when the issuer charges one.

Service scope, pricing and limits →
OpenAI-compatible API, pay in RMBGet API access