Agent cost is not only tokens. It includes model calls, tool APIs, search, storage, compute, queue time, retries, human review, and the cost of wrong actions. Control cost at admission, per run, per tool, per tenant, and per billing period.
After this lesson, you can create a budget envelope and stop a runaway run without corrupting state.
Define the Budget Envelope
max elapsed time
max agent turns
max model input/output units
max tool calls by type
max concurrent workers
max retries
max subtasks and depth
max estimated currency cost
daily tenant and system caps
Estimate before a run and update from actual usage. If provider prices or exchange rates change, version the estimate. Never promise an exact PKR cost from a stale model price.
Apply Admission Control
Reject or queue work when the tenant cap, concurrency cap, dependency circuit, or human-review capacity is exhausted. Reserve budget for cleanup and status notification so crossing a cap does not leave an unknown partial effect.
On stop:
- stop creating new tasks;
- cancel safe in-flight model calls;
- let committed tool actions report final state;
- checkpoint pending tasks;
- notify the owner with reason and resumable scope;
- require approval for additional budget.
Reduce Waste Before Changing Models
Use deterministic routing, smaller context, retrieval of only relevant records, cached approved facts with expiry, schema validation, batch operations, and early exit. Use a capable model only where evaluation shows it adds value. Cache keys must include tenant and version and must not leak private results.
Worked Example
A content-quality run covers 1,000 products. The system allows 20 workers, two attempts per item, six tool calls per task, and a versioned estimated budget. It first validates product records deterministically; 140 invalid records go directly to correction instead of consuming model calls.
A malformed instruction causes repeated evaluator failures. After two attempts per item, tasks enter review. The run-level error rate crosses its stop threshold, so new work pauses. The owner sees completed, pending, failed, estimated/actual usage, and the exact version. No recursive repair agents are spawned.
Failure Cases to Diagnose
- Only a monthly API budget: one run can consume it before detection.
- Workers enforce caps independently: use shared atomic counters or reservations.
- Retry cost ignored: include attempts and failure thresholds.
- Stopping kills a payment call blindly: reconcile consequential effects.
- Cache crosses tenants: include identity, authorization, and version.
- Cheapest model selected without quality test: optimize cost per accepted outcome.
- Human review is “free”: measure time and queue capacity.
🇵🇰 Pakistan Angle
Maintain foreign-currency and PKR views with the exchange assumption and date. Add card/payment-provider limits and tax/accounting treatment as business constraints. Notify the owner before caps, not after a foreign-currency bill becomes unaffordable.
Design offline-safe pause behavior for connectivity or power problems. A recovered worker must read central budget and checkpoint state before continuing; it cannot assume yesterday’s remaining allowance. Smaller Pakistani teams especially need a visible kill switch and simple approval for extra spend.
Hands-On Exercise
- Define per-run, per-tenant, daily, and monthly envelopes.
- Implement or diagram atomic reservations.
- Add error-rate and cost stop thresholds.
- Test concurrent cap crossing and safe cancellation.
- Calculate cost per accepted outcome including human time.
Done means: a run cannot exceed hard limits silently, and stopping preserves an auditable, resumable business state.
Completion Rubric
- Budgets cover time, turns, tools, workers, retries, and money.
- Shared admission control prevents race-condition overspend.
- Cleanup budget and checkpointing support safe stop.
- Costs use actual usage and versioned prices.
- Optimization measures accepted outcomes.
- Kill, alert, and extra-budget approval are tested.
Sources
Key takeaway: enforce shared, layered budgets and safe-stop behavior; optimize the total cost of accepted outcomes, not merely the price of a model call.