AI Agent Cost: Why Agent Loops Break Token Budgets

Why the bill grows faster than the step count

Model APIs are stateless, so an agent loop sends the full context on every step: the system prompt, the tool definitions, the original task, and each tool call and tool result so far. OpenAI's pricing page says tokens are billed at the chosen model's input and output rates, and Anthropic's pricing page lists tool use as input tokens, including the tools parameter, tool_use blocks and tool_result blocks. Nothing in either price list discounts a repeated prefix unless you use caching.

That makes input per step grow by roughly the size of one step's additions, and total input is the sum of those growing contexts. The sum rises with the square of the step count, not linearly, so a loop that runs twice as long costs more than twice as much. A budget built from an average cost per call misses this, because the average itself climbs with run length.

A worked example with hypothetical sizes

The prices below are Claude Sonnet 5.5 on Anthropic's pricing page on 2026-10-18: $2 per million base input tokens, $10 per million output tokens, $0.10 per million for cache hits, and $2.50 per million for 5-minute cache writes. The loop sizes are assumptions chosen for the arithmetic, not measurements: a 5,000-token fixed prefix of system prompt and tool definitions, 1,500 tokens of tool result added per step, and 300 output tokens per step, which join the history. Each step therefore adds 1,800 tokens.

A naive estimate prices 20 steps as 20 calls of 6,500 input tokens: 130,000 input tokens, $0.26, plus 6,000 output tokens at $0.06, for $0.32. The real loop sends 5,000 tokens on step 1, 6,800 on step 2, and so on up to 39,200 on step 20. The input sum is 442,000 tokens, which is $0.884, and with output the run costs $0.944. The naive estimate is low by about 3 times.

At 40 steps the same arithmetic gives 1,604,000 input tokens, $3.208 of input and $0.12 of output, or $3.33. Twice the steps cost 3.5 times as much.

What caching changes, and what it does not

If each step lands inside the 5-minute cache window and the prefix is cached, step 1 writes 5,000 tokens, and each later step reads the prior context at $0.10 per million and writes its 1,800 new tokens at $2.50 per million. For the 20-step run that is 402,800 tokens read ($0.040), 39,200 tokens written ($0.098), and $0.06 of output, for $0.198 in total. For 40 steps it is $0.461. Caching cuts the cost by about 79 percent at 20 steps and 86 percent at 40, and the curve is flatter, but it still rises faster than the step count.

Caching does not help when a step takes longer than the cache window, when a tool result changes the early part of the prompt, or when you switch models mid-run. A cache miss bills the whole context at the uncached rate again. Track cache read tokens as their own line, because a run that quietly stops hitting the cache looks identical in step count and several times more expensive in dollars.

Costs that sit outside the token columns

Server-side tools add fees per use. Anthropic lists web search at $10 per 1,000 searches plus standard token costs, and says search results are counted as input tokens in later turns of the conversation. OpenAI lists web search at $10 per 1,000 calls for its main models, plus search content tokens at model rates. At $0.01 per search, 20 searches cost $0.20, as much as the whole cached 20-step loop above, and the retrieved text then rides along in every later step.

Tool definitions also cost input. Anthropic lists 286 added system-prompt tokens when tools are enabled on Claude Sonnet 5.5, before the tokens of your own schemas. Code execution on Anthropic is billed by container time when not paired with web search or web fetch, with a 5-minute minimum, and OpenAI bills hosted containers per session by memory size. Those are small per run and large per million runs, so log them as separate lines.

Controls that bound a run

Tracking tells you what a run cost after it ends. Bounding it needs limits enforced while it runs. Set a maximum step count and a per-run token ceiling in the agent loop itself, and stop with a clear error when either is hit. Truncate or summarize large tool results before they enter the history, since every token you add is re-billed on each later step.

• Record cost per run, not per call: a run ID on every request lets you total a loop and find the outliers.
• Alert on runs above a multiple of the median cost, such as 5 times, not on the average.
• Enforce a budget per key or per team at the gateway so a runaway loop cannot spend past it.
• Put the stable part of the prompt first and keep it unchanged so caching works.