Batch API

Create, retrieve, cancel, and list async bulk jobs, and poll them to completion. The Batch API processes large request sets asynchronously at roughly half the cost: the input is a JSONL file uploaded via the Files API with purpose="batch", and output is fetched with file_content using batch.output_file_id. OpenAI only.

Parsed Objects

UniLM.BatchObject — Type

BatchObject

A batch job from the Batch API: id, status, endpoint, input_file_id, output_file_id, error_file_id, and request_counts; raw holds the unparsed JSON response.

UniLM.BatchList — Type

BatchList

A page of BatchObjects from list_batches; has_more signals that further pages are available.

Result Types

UniLM.BatchSuccess — Type

Successful create/retrieve/cancel result wrapping a BatchObject.

UniLM.BatchListSuccess — Type

Successful list_batches result wrapping a BatchList.

UniLM.BatchFailure — Type

Batch API error result: HTTP status and the raw response body.

UniLM.BatchCallError — Type

Local/transport error from a Batch API call (the request never completed).

Request Functions

UniLM.create_batch — Function

create_batch(input_file_id, endpoint; completion_window="24h", metadata=nothing, service=OPENAIServiceEndpoint)

Create a batch job. endpoint is e.g. "/v1/chat/completions", "/v1/responses", or "/v1/embeddings". input_file_id comes from upload_file(path, "batch").

Pass config::Union{Nothing,RequestConfig} to override the timeout budget for this call (a single bounded attempt; max_attempts does not apply).

UniLM.retrieve_batch — Function

retrieve_batch(id; service=OPENAIServiceEndpoint)

Pass config::Union{Nothing,RequestConfig} to override the timeout budget for this call (a single bounded attempt; max_attempts does not apply).

UniLM.cancel_batch — Function

cancel_batch(id; service=OPENAIServiceEndpoint)

Pass config::Union{Nothing,RequestConfig} to override the timeout budget for this call (a single bounded attempt; max_attempts does not apply).

UniLM.list_batches — Function

list_batches(; limit=nothing, after=nothing, service=OPENAIServiceEndpoint)

Pass config::Union{Nothing,RequestConfig} to override the timeout budget for this call (a single bounded attempt; max_attempts does not apply).

UniLM.poll_batch — Function

poll_batch(id; interval=10.0, timeout=86400.0, service=OPENAIServiceEndpoint)

Poll a batch until terminal (completed/failed/cancelled/expired) or timeout.

Pass config::Union{Nothing,RequestConfig} to override the timeout budget for this call (a single bounded attempt; max_attempts does not apply).

Usage

# Upload a JSONL request file, then create a batch job
upload = upload_file("requests.jsonl", "batch")
batch = create_batch(upload.response.id, "/v1/chat/completions")

# Poll until terminal (completed / failed / cancelled / expired)
done = poll_batch(batch.response.id)
if done isa BatchSuccess && done.response.status == "completed"
    output = file_content(done.response.output_file_id)
    output isa FileContentSuccess && save_file_content(output, "results.jsonl")
end

# List jobs, or cancel one
list_batches(limit=10)
cancel_batch(batch.response.id)