Batch API
High-throughput asynchronous workloads
A Batch API for high-throughput, latency-tolerant workloads — evaluations, backfills, dataset annotation — is on the roadmap. It will follow the familiar asynchronous pattern: upload a JSONL file of requests, create a batch, poll for completion, download results.
Planned workflow
| Step | Call |
|---|---|
| 1. Upload input file | POST /v1/files (JSONL of requests) |
| 2. Create the batch | POST /v1/batches |
| 3. Poll status | GET /v1/batches/:id |
| 4. Fetch results | Download the output file on completion |
Coming soonThe Batch API is not yet available. Concurrent
/v1/chat/completions calls remain the way to run volume today — see Reliability and Error Handling for rate limits (default 600 req/min · 200K tokens/min per key) and backoff patterns.