Batch Processing

Batch processing (batching) allows you to send multiple message requests in a single batch and retrieve the results later (within up to 1 hour). The main goals are to reduce costs by up to 50% and increase throughput for analytical or offline workloads.

To use batch processing, several endpoints are available:

Create a message batch

https://api.aimlapi.com/batches

Get status or results of a specific message batch

https://api.aimlapi.com/batches?batch_id={batch_id}

Cancel a specific message batch

https://api.aimlapi.com/batches/cancel/{batch_id}


Create a message batch

Create a message batch

post

Create a batch of messages for asynchronous processing. All usage is charged at 50% of the standard API prices.

Authorizations
AuthorizationstringRequired

Bearer key

Body
Responses
post
/v1/batches
201Success

No content

Code Example (Python)
Response

Get status or results of a specific message batch

Get batch status or results

get

Get batch status if in progress, or stream results if completed in JSONL format.

Authorizations
AuthorizationstringRequired

Bearer key

Query parameters
batch_idstringRequired

The ID of the batch to get results for

Responses
200Success

No content

get
/v1/batches
200Success

No content

Code Example (Python)
Response

Cancel a specific message batch

Cancel a message batch

post

Cancel a message batch that is currently in progress. Requests that have already started processing will complete, but no new requests will be processed.

Authorizations
AuthorizationstringRequired

Bearer key

Path parameters
batch_idstringRequired

The ID of the batch to cancel

Responses
post
/v1/batches/cancel/{batch_id}
201Success

No content

Code Example (Python)
Response #1 (successfully cancelled)
Response #2 (if already finished)

Last updated

Was this helpful?