Module 05 · Production AI Tooling
Error Handling and Rate Limits in Production Tools
Open lesson + course map
On this lesson
Course outline
Module 1 · Claude Code Fundamentals
Module 2 · CLI Workflows
Module 3 · Model Context Protocol Basics
Module 4 · Building an MCP Server
Production tools must tell the host whether a failure is retryable, permanent, unauthorized, conflicted, or uncertain after a possible effect. Retrying everything creates duplicate actions and outage storms.
// concept
Define a Safe Error Model
{
"ok": false,
"error": {
"code": "RATE_LIMITED",
"retryable": true,
"retry_after_ms": 2000,
"correlation_id": "req_72"
}
}Do not return stack traces, tokens, SQL, internal paths, or private upstream payloads. Log safe detail under restricted access and give the client a correlation ID.
Classify validation, not found, conflict, unauthenticated, forbidden, rate-limited, temporary unavailable, timeout-unknown, and internal failure.
// concept
Control Load
Apply per-identity and per-tenant rate limits, concurrency caps, queue limits, timeouts, and request-size limits. Respect upstream Retry-After guidance. Use bounded exponential backoff with jitter for transient reads.
For writes, require idempotency. If a timeout occurs after submission, query authoritative state before retry. Use circuit breaking during broad upstream failure and dead-letter review for exhausted work.
// worked_example
Worked Example
A courier quote tool allows five concurrent requests per tenant and validates city/weight before the upstream call. When upstream returns 429, the server returns a controlled retry delay. The host retries twice with the same request identity.
A booking creation times out. The server queries by idempotency key, finds the booking, and returns it instead of creating another. Metrics show rate-limit count, queue age, p90 latency, and error class without customer address.
// failure_cases
Failure Cases to Diagnose
6 cases to diagnose
Model decides retry timing
enforce in host/tool code.
New idempotency key per attempt
preserve logical identity.
Global limit lets one tenant monopolize
add fair per-tenant quotas.
Queue grows without bound
cap and shed safely.
403 retried
authorization failures require correction.
Logs leak upstream response
redact and minimize.
// concept
Operator Note
Publish a small service-level contract for tool consumers: maximum input size, typical timeout, concurrency behavior, error codes, retry obligations, and idempotency support. Version that contract and exercise it in load tests. A client should never guess whether it may retry a write. During overload, reject early with a controlled response instead of accepting unlimited work that will expire unseen.
// pakistan_angle
Pakistan Angle
Expect intermittent courier, wallet, bank, and government endpoints. Do not replace an unavailable official service with scraping or invented status. Provide a case reference and honest pending state.
Foreign API cost is another quota. Limit tenant spend and convert estimates to PKR using a dated assumption; never let retries silently consume the client’s budget.
// hands_on
Hands-On Exercise
5 steps
Define the error taxonomy.
Add identity/tenant rates, concurrency, queue, and timeout.
Implement stable write idempotency.
Test 429, 403, timeout-after-effect, overload, and circuit open.
Verify redacted metrics and logs.
// completion_rubric
Completion Rubric
6 checks — tick as you verify
// sources
Sources
3 official sources — check every claim yourself
// check_yourself
Check yourself
4 questions · answers and options are taken word-for-word from this course
1 / 4 · diagnose
Your work shows this failure mode: “Model decides retry timing.” What does the lesson tell you to do about it?