Three of the four advanced ways to cut AI API costs work on a hosted API: the Batch API, schema-enforced output and a hard token ceiling. The fourth, speculative decoding, needs a server you run. In one worked nightly job, the first three take the bill from $80.00 to $24.00.
Part 1 picked the model tier. Part 2 cut the tokens you send. This part cuts what comes back and sets when you ask for it.
TLDR
The Batch API halves both token prices for work that can wait. Anthropic says most batches finish within 1 hour, and a batch expires after 24.
A JSON schema forces short, valid output. Output tokens cost 5x input tokens on Claude Sonnet 5.5, so short output is the larger saving.
The API enforces
max_tokensonly. Your code must enforce turn and dollar limits per task.Batch plus schema takes the worked job from $80.00 to $24.00 a night.
Self-host only when output tokens dominate your bill and you already run the GPU.
Three of These Four Savings Need No Model Access
On a hosted API the provider owns the model. You control what you send, when you send it and how much comes back. You do not control how the model decodes.
Some cost research reads a model's internal activations, and a hosted API exposes none, so those methods are out of reach. The four techniques below are the ones you can act on.
The Same Nightly Job Costs $80.00 Padded and $24.00 Capped
Take a job that analyzes 10,000 log chunks overnight. The numbers below are assumptions, so replace them with the usage field from your own responses.
Before: real-time calls, a padded prose answer of about 400 output tokens, no cap. Cost: $80.00.
After: one batch, a schema that returns about 80 tokens of JSON, and a 150-token cap. Cost: $24.00, and $27.50 if every call hit the cap.
The prices are Claude Sonnet 5.5 at $2 input and $10 output per million tokens, or $1 and $5 in a batch, from Anthropic's pricing page on 2026-10-13.
Rule of thumb. The batch discount lands only on the share of the bill that can wait. If 70% of a $100 bill can wait, 50% off that share leaves $65, a 35% cut.
Step 1. Send Anything That Can Wait to the Batch API
Anthropic and OpenAI both charge 50% less for batch work on input and output tokens. Both give you up to 24 hours. Anthropic says most batches finish within 1 hour.
A batch holds up to 100,000 requests or 256 MB on Anthropic. Each request carries a custom_id so you can match results to inputs.
Batch every task whose result can wait an hour. Queue the non-urgent requests through the day and submit them at midnight. Log analysis, bulk enrichment, moderation backlogs and embeddings all fit.
Tip. A batch can take longer than 5 minutes, so Anthropic's docs suggest the 1-hour cache duration when requests share context. That ties Part 2's caching to Part 3's discount.
Anthropic announced its batch API on X: up to 10,000 queries per batch at launch, each batch processed within 24 hours, at 50% less than standard calls.
Step 2. Force Short, Valid Output With a Schema
Output is the expensive side. On Claude Sonnet 5.5, output costs $10 per million tokens against $2 for input. A model trained to sound helpful adds a preamble, a summary and a disclaimer to an answer that needed 80 tokens.
A JSON schema removes that padding. Anthropic's structured outputs are generally available on the Claude API. You pass the schema in output_config.format, and the reply is valid JSON with your keys and types. That also removes the retry loop after a malformed answer.
The batch docs list three unsupported parameters, and output_config is not one of them. Send 20 requests as a test batch before you submit 10,000.
Watch out. A cap below the schema's natural length truncates the JSON mid-object. Check
stop_reasonformax_tokenson every result, and raise the cap for those calls.
A second lever exists on some providers. OpenAI's logit_bias takes values from -100 to 100 per token ID, so you can ban filler tokens. Anthropic's Messages API has no such parameter, and token IDs differ by model. Treat it as a provider-specific extra.
Step 3. Cap Every Agent Before It Loops
An agent that hits a type error can retry the same broken fix dozens of times. Each retry pays for output tokens, and nobody sees the bill until it lands.
Set three limits. The API enforces the first. You write the other two.
Tokens per call. Set
max_tokenson every request.Turns per task. Count turns in your loop and stop at a fixed number.
Dollars per task. Sum
usageafter each turn and stop at a fixed spend.
When any limit trips, stop the task and hand the trace to a person. A stuck agent has no new information to spend.
Example. Forbes reports that Uber used up its 2026 AI budget in four months on Claude Code. Yahoo Finance reports one executive's two-hour coding session cost $1,200. TechCrunch reports Uber's fix, a cap of $1,500 per month per employee and per coding tool.
Uber set that cap per person. A per-task cap catches the stuck loop sooner. Many builders leave agents running unwatched all night. Token Anxiety describes that pull from the builder's side.
Step 4. Self-Host Only If One Test Passes
Run two checks. Are output tokens most of your bill? Do you already run the GPU server? Self-host only if both answers are yes.
Speculative decoding is the technique that earns a server. A small draft model proposes tokens, and the large model checks them in one pass. The DeepMind paper reports a 2 to 2.5x decoding speedup on Chinchilla 70B without compromising sample quality. Medusa is a variant that adds extra prediction heads, so no second model is needed.
This video shows the draft-and-verify loop on a screen.
If you do run models yourself, Ollama and LiteLLM are the two tools I list on my tools page. Ollama runs open weight models on your own machine. LiteLLM puts one OpenAI-shaped interface in front of many model APIs.
A self-hosted small model can also take over one narrow job.
Daniel Bourke's June 2025 newsletter walkthrough covers one self-host recipe: fine-tune a 0.5B model on a 32B model's labels and drop the input prompt, for 50x faster document annotation.
For a cost cut from a different workflow, my AI-First DevRel write-up lists a 90% cost cut among its headline results.
Four Questions Readers Ask About API Cost Cuts
Can I batch and use a schema together? The batch docs list stream, speed and max_tokens: 0 as unsupported, and output_config is not on that list. Test 20 requests first to confirm on your account.
What happens if a batch does not finish? Anthropic expires a batch that has not completed within 24 hours, and most finish within 1 hour. Resubmit the expired requests. Never batch work a user is waiting on.
Which providers support logit bias? OpenAI does. Anthropic's Messages API request has no logit_bias parameter. Check your own provider's docs, because token IDs also differ by model.
When does speculative decoding pay? Only on a server you control, and mostly when output tokens dominate the cost. On a hosted API you cannot switch it on.
The Appendix Holds the Batch Call, the Schema and the Guard
These snippets follow the Anthropic and OpenAI docs of 2026-10-13. Run each on 20 inputs before you scale.
The cost arithmetic behind the figures, with its real output:
IN_STD, OUT_STD = 2.00, 10.00 # Sonnet 5.5, USD per million tokens
IN_BATCH, OUT_BATCH = 1.00, 5.00
REQUESTS, IN_TOKENS = 10_000, 2_000
OUT_PADDED, OUT_JSON, CAP = 400, 80, 150 # assumptions, replace with your usage
def cost(i, o, out):
return REQUESTS * (IN_TOKENS * i + out * o) / 1e6
print(cost(IN_STD, OUT_STD, OUT_PADDED)) # 80.0
print(cost(IN_BATCH, OUT_BATCH, OUT_PADDED)) # 40.0
print(cost(IN_BATCH, OUT_BATCH, OUT_JSON)) # 24.0
print(cost(IN_BATCH, OUT_BATCH, CAP)) # 27.5A batch with a schema and a cap on Anthropic, using the Python SDK:
import anthropic
client = anthropic.Anthropic()
schema = {
"type": "object",
"properties": {"severity": {"type": "string"}, "cause": {"type": "string"}},
"required": ["severity", "cause"],
"additionalProperties": False,
}
batch = client.messages.batches.create(requests=[
{
"custom_id": f"log-{i}",
"params": {
"model": "claude-sonnet-5-5",
"max_tokens": 150,
"messages": [{"role": "user", "content": chunk}],
"output_config": {"format": {"type": "json_schema", "schema": schema}},
},
}
for i, chunk in enumerate(chunks)
])
print(batch.id)One line of an OpenAI batch file, with custom_id, method, url and body:
{"custom_id": "log-0", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "your-model", "messages": [{"role": "user", "content": "..."}], "max_tokens": 150}}A per-task guard that enforces the turn and dollar limits:
MAX_TURNS, MAX_USD = 12, 0.50
IN_PRICE, OUT_PRICE = 2.00 / 1e6, 10.00 / 1e6
spent, turns = 0.0, 0
while not task_done:
reply = client.messages.create(model="claude-sonnet-5-5", max_tokens=1024, messages=history)
spent += reply.usage.input_tokens * IN_PRICE + reply.usage.output_tokens * OUT_PRICE
turns += 1
if reply.stop_reason == "max_tokens":
escalate(history, "hit the per-call cap")
break
if turns >= MAX_TURNS or spent >= MAX_USD:
escalate(history, "hit the per-task limit")
break
history = next_history(history, reply)