The batch API processes requests that do not need an immediate answer in exchange for a discount —usually around 50%— versus real-time requests. It pays off on everything that is not interactive: reports, data enrichment, mass classification. It is trading minutes (or hours) for money.
It is the third of the 7 levers to cut your OpenAI bill: the one with the most saving per unit of effort when you have work that can wait.
At a glance
| What it does | Processes batches of requests asynchronously |
| Typical saving | ~50% versus real time |
| In exchange for | Not being immediate (a window of minutes to hours) |
| When it pays off | Everything that is not interactive |
How it works
Instead of calling one request at a time and waiting for the answer right away, you send a whole batch and the provider processes it when it has capacity, within an agreed window. When it finishes, you collect all the results together. The discount is the trade-off for giving up immediacy.
How much it saves
The discount is around half the real-time price, so on the loads you can defer the cost per task drops at once without touching the model or the prompt. The larger the volume you move per batch, the more it shows on the bill.
Which tasks fit
Anything no user is waiting on fits: overnight reports, enriching a database, classifying or tagging in bulk, generating catalog descriptions, summarizing historical files. If the result can arrive in a few hours, it is a batch candidate.
When not to use it
It is no good for anything interactive: a chat, a live assistant, any flow where the user waits for the answer now. There latency rules and real time is mandatory. The rule is simple: if someone is waiting in front of the screen, it is not batch.
Put a number on it
Before moving anything to batch, measure how much of your spend is deferrable. Put your tokens and your volume into the token cost calculator and apply the discount to that part: you will see how much the monthly cost drops. All of this is part of the cost per task that e-ficient measures.
Frequently asked questions
How much does the batch API save? Usually around half the real-time price, in exchange for the answer not being immediate. It pays off on everything that is not interactive.
Which tasks can I send in a batch? Any that do not need an answer right away: reports, data enrichment, mass classification, bulk generation.
Does it combine with the other levers? Yes. Batch stacks with picking the model per task and with prompt caching: a discount on an already optimized price.
At e-ficient we measure the cost per task of companies that already run AI in production and tell you how much you would save with each lever at your real volume. The first audit is free if you start a plan with a three-month commitment and returns a diagnosis within 72 hours.