Batch AI API Jobs From China: Durable Queues, Recovery and Agent Acceptance Tests
Run repeatable content and agent workloads with durable job states, bounded concurrency, deduplicated delivery and cost reconciliation.
1. Assign one job identity and many attempt identities
Choose a stable business identity from inputs such as product identifier, target language and template version. Assign a separate attempt identifier whenever a worker makes a model call. Your job record holds input version, state and final artifact location. Each attempt holds model, start and end times, error category and available usage. A retry keeps the business identity and adds an attempt; it is not a new customer request. This lets you price a delivered description or support draft across all generation steps. Keep personal information and full prompts out of identifier strings. Store the references and versions needed for recovery instead. These identifiers are maintained by your application, not a claim that a custom request header makes the model call idempotent or prevents duplicate generation charges.
job: job_id, input_version, state, result_ref
attempt: attempt_id, job_id, requested_model, started_at, ended_at, usage, charge_state
delivery: job_id (unique), accepted_result_ref, delivered_at
queued -> claimed -> attempted -> validated -> delivered
interrupted attempt -> unresolved -> reconcile -> retry_or_review2. Test delivery outcomes alongside response timing
Preserve the principle of measuring before launch, but distinguish interactive and background workloads. Interactive work needs an observed time to first response; background work needs total duration, queue age and accepted artifacts. Run a sanitized set of short, long, multilingual and malformed inputs with fixed acceptance conditions. Increase concurrency gradually and inspect slow tasks separately from the average. A normal HTTP response still needs field validation, factual checks and business review where appropriate. Retain the test date, model identifier and configuration so the next run is comparable. A small test tells you about that test, not guaranteed capacity or a long-term latency commitment. Set your own release criteria and confirm that workers stop admitting more work when the queue becomes unhealthy.
3. Treat an interrupted attempt as unresolved until checked
A client that did not receive a complete answer does not know from that fact alone whether processing occurred. Keep an unresolved attempt record with its start time and any available request reference. Reconcile the usage and charges you can inspect before deciding on another attempt under your policy. Leave accepted jobs complete and recover only eligible unfinished jobs rather than rerunning the whole batch. If an answer arrived but was not published, validate and persist that artifact before generating again. Separate invalid input, authorization, insufficient funds and temporary errors because their recovery actions differ. /hub/ai-api-errors-china-fix provides related troubleshooting categories. Never turn every timeout into an automatic zero-cost record or an unlimited retry instruction; uncertainty is a state that needs an owner and a resolution path.
No overseas card needed: cocodot's OpenAI-compatible API — switch your base_url to call the models on sale.
Top up via Alipay / WeChat Pay (RMB), USDT / USDC · billed per token · switch your base_url to connect
4. Use leases and conditional writes in your own queue
One suggested design is to give each claimed job an owner and an expiry, renewed according to your application policy while work continues. Only an expired claim without a valid result becomes eligible for recovery. At final persistence, enforce a unique business job identity or a conditional state update so duplicate workers cannot both publish a result. Apply separate deduplication to business tools, especially sending messages, changing inventory or publishing content. A model may generate twice during recovery while the business action must still happen once. This is an application design recommendation, not a guarantee from the gateway. Before launch, deliberately terminate a test worker between generation and persistence. Confirm that a replacement either uses the saved artifact or records a new attempt without repeating an already completed business action.
5. Count retry layers before raising the worker count
Give each job a total deadline and attempt limit. Cap active calls across the service and within each tenant, and separate long background jobs from short interactive work. When a temporary error justifies retrying, delay with jitter and respect applicable waiting guidance. Stop dispatch and address the cause for authorization, parameter or balance problems. Inspect the client library, workflow framework and worker for overlapping retry loops. One place should own the job deadline and cumulative allowance so nested loops cannot quietly multiply attempts. If higher concurrency fails to reduce queue age while individual durations rise, stop expanding and investigate. See /hub/api-429-rate-limit-concurrency-china for concurrency troubleshooting. There is no fixed capacity number in this guide; use measurements from your own permitted workload and release criteria.
6. Make the agent tool loop recoverable too
cocodot accepts function tools only. Search, code execution or business lookups run in your own controlled process, which returns tool results to the model. For each tool action, retain the job identity, tool name, validated arguments and business completion state. Recovery should inspect that state before executing the action again. Losing the last model response is not sufficient reason to resend a customer message or repeat an inventory update. Bound model steps, tool steps, elapsed time and total budget, preserving useful state for an operator when a limit is reached. Validate and limit tool output before adding it to the next context. /hub/agent-api-cost-runaway discusses related spending guardrails. For a product serving multiple customers, combine these job states with the backend boundaries described in /hub/saas-multi-model-api-integration.
7. Reconcile batches using settled, active and unknown work
Aggregate settled charges, active budget reservations and unresolved attempts under the batch identity. Join every generation and checker attempt to its business job before calculating the cost of delivery. Admission should consider all three categories, rather than only the charges from completed calls. When the cocodot balance runs out, new requests are refused and in-flight requests may still incur charges, with fallback estimates used when usage is missing. Balance depletion is therefore not an exact stop mechanism for every active job. If usage is missing in an application log, retain an unresolved entry and reconcile available evidence; visible text length is not a reliable substitute for a charge. A batch with no accepted outputs has produced no accepted delivery, even if its requests were inexpensive. Faster dispatch can generate rejected artifacts faster, so cost reconciliation must remain connected to validation outcomes.
8. Release after a complete interruption and recovery drill
Register, verify email and complete identity verification, then prepare the balance and create a key for your intended workload. Use the OpenAI-compatible base URL https://cocodot.co/api/ai/v1 with Authorization: Bearer authentication. Keys can be named and individually revoked, with at most 10 active keys per account. The trial credit is usable after your first top-up. Funding options are Alipay / WeChat Pay (RMB), USDT / USDC. Listed Claude / GPT / Gemini rates are at or below the official list price; check the pricing page for each model's current rate. Run a small set through claim, call, validation, persistence, restart and accounting before expanding. For persistent problems, https://cocodot.co/docs points to in-console support. Provide sanitized timing, model and error categories, keeping credentials and customer source text out of the support material.
| Interruption point | Suggested application state | Check before recovery |
|---|---|---|
| Before dispatch | Queued | Input version and budget admission |
| After send, no complete response | Unresolved attempt | Available usage and result evidence |
| Response received, not delivered | Awaiting validation | Artifact matches the business job |
| Validated and published | Complete | Prevent duplicate business delivery |