AIForum.

AI Agent Cost Control: Token Budgets, Rate Limits, and Loop Detection

On this page

THE SHORT ANSWER

Agent cost control rests on three enforcement points: a run budget over provider calls, tokens, money, and wall clock; rate limiting with disciplined retry and full-jitter backoff; and loop detection that catches repeated action without progress. This guide explains how each control works, where they overlap, and how to verify that a limit actually terminates a run instead of merely reporting on it.
Quick facts

Controlling what an AI agent costs is less about choosing a cheaper model and more about enforcing limits inside the process that runs the agent. Three controls carry most of the weight: a run budget that caps provider calls, tokens, money, and wall clock; rate limiting with disciplined retry and backoff; and loop detection that notices when an agent is repeating itself without changing state. The Agentic Fabriq failure catalogue frames the same problem from the symptom side: a runaway agent first appears as a change in the shape of a graph — provider calls per run, 429 rate, or spend.

This article is for practitioners who run agents that call external APIs on a schedule or on user demand, and who want the mechanism rather than a vendor pitch. It covers what each control does, how the controls differ, and how to verify that they stop a run rather than merely reporting on it.

How agent spending actually runs away

Before adding controls, it helps to know which failure you have. Agentic Fabriq distinguishes three graph shapes that point at different causes:

  • A staircase — retries multiplying across layers.
  • A clean ramp — a loop with no terminating condition.
  • A flat plateau at a new level — pagination that never ends.

Retry amplification is the most counterintuitive of the three, because each layer looks reasonable in isolation. Three layers that each retry three times send twenty-seven requests for one logical call; the source notes that its offline exercise examples/fail-runaway-loops/retry_amplification.py prints exactly that number. The rule that follows is blunt: retry in one layer only.

Loops and endless pagination are different in kind. A loop is a control-flow problem — the agent keeps taking the same action because nothing in its state changes. Endless pagination is a termination problem — the agent keeps asking for the next page because the stopping condition was never expressed. Both consume budget, but they need different detection logic.

Run budgets: four quantities behind one gate

Agentic Fabriq’s central recommendation is that the control which actually stops a runaway is a run budget over four quantities at once: provider calls, tokens, money, and wall clock. A budget over only one of them leaks. A token cap does not stop an agent that makes thousands of cheap calls; a call cap does not stop an agent that sends one enormous prompt; neither stops an agent that hangs.

Two design details matter more than the numbers themselves.

Derive the limits from a measured healthy run. The source is explicit that budget values should come from observation, not taste. Run the agent on a representative task, record calls, tokens, spend, and duration, then set the ceiling as a multiple of that baseline. A budget invented at a whiteboard will either fire constantly on normal work or never fire at all.

A budget must halt, not warn. A counter that emits a metric is telemetry; a budget is an enforcement point that terminates the run and returns a clear error. If the run cannot be terminated mid-flight, the budget is advisory.

Money is the awkward quantity, because it requires a price per token that changes over time and differs per model. Where the agent runs against a self-hosted model, there is no per-token invoice at all, and wall clock plus hardware occupancy become the practical proxy. That substitution is a design decision rather than a measurement, and should be documented as such.

Rate limits and retry discipline

Rate limits and budgets protect different parties. A rate limit protects the provider — and, indirectly, your account’s standing — while a budget protects your spend. A provider returning HTTP 429 is telling you that you exceeded an allowance; a well-behaved client treats that as a scheduling instruction rather than an error to hammer through.

The retry policy Agentic Fabriq recommends uses full jitter:

sleep = random_between(0, min(cap, base * 2 ** attempt))

Two refinements accompany it. First, treat a Retry-After header as a floor you never go below, adding jitter above it rather than replacing it. Second, keep retries in a single layer of the stack; when a client library, an orchestration framework, and an application wrapper all retry, the multiplicative effect described above reappears.

Rate limiting also has a client-side half. Even without provider throttling, an agent that fans out parallel tool calls can saturate a downstream service that has nothing to do with the model. A concurrency limit on tool execution is a rate limit by another name.

Loop detection

Loop detection is the control with the least standardised definition. In practice it watches for repeated action without progress, using a small set of signals:

  • Repeated identical tool calls. Hash the tool name and its arguments; if the same hash appears several times in a short window, the agent is likely stuck.
  • No state change. If consecutive steps leave the working state unchanged, the agent is spending tokens without advancing.
  • Pagination cursors that repeat. A cursor seen before means the traversal is not moving forward.

These are heuristics, and heuristics need a fallback. A loop detector should raise a signal that the run controller can act on — abort, inject a different instruction, or hand off to a human — rather than silently rewriting the agent’s plan. It should also have a hard ceiling underneath it, because a detector tuned loosely enough to avoid false positives will miss some real loops.

Loop detection is also the control most sensitive to false positives. An agent legitimately polling a job status, or retrying a flaky read, can look identical to a stuck agent over a short window. Widening the window reduces false positives and increases the cost of each missed loop; there is no setting that removes the tradeoff.

Circuit breakers and idempotency keys are complements, not substitutes

Two adjacent controls are often confused with budgeting. A circuit breaker stops you calling a dependency that is already failing, which prevents an outage from turning into a retry storm. A stable idempotency key stops a retried write from becoming two rows, which prevents duplication when a retry is legitimate.

Agentic Fabriq’s formulation is worth keeping: neither substitutes for the other, and neither substitutes for the budget. A breaker does not cap spend on dependencies that are healthy; an idempotency key does not cap the number of attempts.

Where enforcement can live

Enforcement can sit inside the agent process, in a sidecar, or in a gateway in front of providers. In-process enforcement sees the richest context — tool names, arguments, intermediate state — which is what loop detection needs; a gateway sees only network traffic, which is enough for call and rate limits but not for state-aware checks.

RateGuard is one example of the in-process approach. Its README describes agent runtime enforcement covering token budgets, rate limits, circuit breakers, loop detection, and cryptographically delegable spend authority, running inside the application process, with SDKs for Go, Node.js, and Python. The Go module is published under the MIT licence, requires Go 1.24.0, and depends on github.com/redis/go-redis/v9 v9.17.0 and OpenTelemetry v1.39.0 [pkg.go.dev].

Two scope notes about that evidence. The retrieved package page documents the Go SDK; the Node.js and Python SDKs are described in the README but were not verified here. And the module has moved: v0.1.0 was published on 16 May 2026, while v0.2.0, published 6 July 2026, is the default version — so pin a version deliberately rather than relying on go get defaults.

Applying the same controls to local models and WordPress automation

Agents backed by local models do not generate a per-token invoice, which makes it tempting to skip budgeting. The control still applies, with different units: wall clock and hardware occupancy replace money, and a runaway loop on a local model still blocks the hardware for every other workload. Call and token ceilings remain meaningful because they bound the time to detection.

The same reasoning applies to automation platforms. A WordPress workflow that triggers an agent on a schedule or on a content event runs unattended, which is exactly the situation where a loop has nobody to notice it. The practical minimum for that case is a per-invocation cap on provider calls and wall clock, plus a log line recording which cap fired. These are applications of the principles above rather than measured results.

Verifying that a control works

None of these controls should be trusted on the basis of configuration alone. A short verification sequence is worth the effort:

  1. Run the agent against a simulated or sandboxed provider so failures cost nothing.
  2. Force each failure mode in turn — an injected retry storm, a deliberate loop, a pagination cursor that never advances.
  3. Confirm the run terminates, and confirm which control terminated it.
  4. Record the healthy-run baseline again after any change to prompts, tools, or models, since all four budget quantities shift with them.

The Agentic Fabriq examples are built to run offline against a simulated provider for exactly this reason, which makes them usable as a starting point for step two.

What to do next

Start with the budget, because it is the only control that bounds all four cost dimensions at once. Measure one healthy run, set ceilings from that measurement, and make the ceiling terminate the run. Then fix retry policy to a single layer with full jitter, and only then add loop detection, which needs the most tuning and produces the most false positives. If you are working through this on your own stack, the community is a reasonable place to compare baselines — share the task, the model, the measured calls and tokens per run, and which cap fired first.

FAQ

What is the difference between a rate limit and a run budget?

A rate limit controls how fast requests are sent and protects the provider and your account standing. A run budget controls how much a single run may consume in total — provider calls, tokens, money, and wall clock — and protects your spend. A run can stay under every rate limit and still blow a budget, and vice versa.

Why does retry logic multiply request volume?

Retries compound when more than one layer of the stack retries. Three layers that each retry three times produce 27 requests for one logical call, which is the amplification the Agentic Fabriq exercise examples/fail-runaway-loops/retry_amplification.py prints. The mitigation is to retry in exactly one layer.

Does a circuit breaker remove the need for a budget?

No. A circuit breaker only stops calls to a dependency that is already failing, so it does nothing about spend on healthy dependencies. Agentic Fabriq states that a breaker, an idempotency key, and a run budget are complements, and that neither of the first two substitutes for the budget.

How should the numbers in a budget be chosen?

Derive them from a measured healthy run rather than from intuition. Record provider calls, tokens, spend, and duration for a representative task, then set ceilings as a multiple of that observed baseline, and re-measure after changes to prompts, tools, or models.

References

  1. https://agenticfabriq.github.io/fail/runaway-loops/
  2. https://pkg.go.dev/github.com/varbees/rateguard/packages/sdk-go@v0.1.0

Related terms

Run budget
A per-run ceiling over provider calls, tokens, money, and wall clock that terminates the run when exceeded, rather than only reporting the overage.
Full jitter
A backoff strategy that picks a random delay between zero and an exponentially growing cap, spreading retries instead of synchronising them.
Circuit breaker
A control that stops calls to a dependency already known to be failing, preventing an outage from becoming a retry storm. It does not cap spend on healthy dependencies.
Idempotency key
A stable identifier attached to a write so that a retried request is recognised as the same operation and does not create a duplicate record.
HTTP 429
The status code a provider returns when a client exceeds a rate allowance; a correct client treats it as a scheduling instruction and respects any Retry-After value as a floor.

Leave a Reply

Your email address will not be published. Required fields are marked *