API Keys, Secrets, Rate Limits, and Cost Controls

An API key is a credential, not a configuration value you may casually paste into code. A reliable AI application keeps credentials outside source control, limits what one key can affect, expects rate limits, caps retries and concurrency, and stops work when a budget or safety boundary is reached. These controls belong in the design before the first paid request, because a successful prototype can otherwise become an expensive or exposed failure.

This lesson uses dummy values and a simulated service. Never place a real key in a lesson submission, screenshot, chat message, issue, commit, browser bundle, mobile app, or analytics event.

Separate Public Configuration From Secrets

Public configuration includes a model identifier, timeout, maximum input size, and feature flag. A secret grants access: API keys, database passwords, signing keys, and OAuth client secrets are examples. Environment variables keep these values outside the code file, but they are not automatically secure. Other processes, deployment dashboards, shell history, crash reports, and careless logs may still expose them.

Use this minimum pattern:

• Commit .env.example with variable names and safe placeholders only.
• Add .env and provider-specific local secret files to .gitignore before the first commit.
• Read the real value from process.env or a managed secret store at runtime.
• Create separate keys for development and production when the provider supports it.
• Restrict access, ownership, and permissions to the smallest practical scope.
• Rotate and revoke a key immediately when exposure is suspected; deleting it from the latest commit is not enough because Git history retains earlier versions.

Frontend JavaScript runs on the user's device. Any key shipped to a browser can be extracted, even if the bundle is minified or the variable name says “private.” Browser and mobile clients should call your authenticated server, and the server should decide whether a model request is allowed.

Understand Limits as More Than Requests Per Minute

Providers can limit request count, input or output tokens, concurrent work, media duration, or other dimensions. Limits may vary by account, model, and usage tier. Read current dashboard values instead of copying a number from a tutorial. A 429 response can mean “slow down,” while other error details may indicate a quota or billing condition. Classify the error before retrying.

Retries need exponential backoff with jitter: wait, add randomness, then increase the wait for another transient failure. Jitter stops many workers from retrying at the same instant. Set a maximum attempt count and maximum elapsed time. Failed requests may still count against a rate limit, so an infinite retry loop makes the situation worse.

Also control concurrency. Ten queued tasks released together can exceed a limit even when the average hourly volume looks small. Use a queue, process a bounded number at once, and apply backpressure when new work arrives faster than it can be handled.

Worked Example: A Guarded, Simulated Request Runner

Create .env.example:

AI_PROVIDER_KEY=replace_in_your_local_environment
AI_MODEL=choose_from_your_provider_dashboard
MAX_INPUT_CHARS=2000
MAX_ATTEMPTS=4
DAILY_REQUEST_BUDGET=20

Create .gitignore:

.env
*.log
usage-ledger.json
node_modules/

The following JavaScript makes no external request. It lets you test limits without spending money or exposing a credential:

// guard-lab.mjs
const maxInputChars = Number(process.env.MAX_INPUT_CHARS ?? 2000);
const maxAttempts = Number(process.env.MAX_ATTEMPTS ?? 4);
const dailyRequestBudget = Number(process.env.DAILY_REQUEST_BUDGET ?? 20);

function assertPositiveInteger(name, value) {
  if (!Number.isInteger(value) || value < 1) {
    throw new Error(`${name} must be a positive integer`);
  }
}

assertPositiveInteger("MAX_INPUT_CHARS", maxInputChars);
assertPositiveInteger("MAX_ATTEMPTS", maxAttempts);
assertPositiveInteger("DAILY_REQUEST_BUDGET", dailyRequestBudget);

const input = "Classify this synthetic support message by urgency.";
if (input.length > maxInputChars) throw new Error("Input exceeds local size cap");

let callsToday = 0;
async function simulatedRemoteCall(attempt) {
  callsToday += 1;
  if (callsToday > dailyRequestBudget) throw new Error("Local daily budget reached");
  if (attempt < 3) return { ok: false, status: 429 };
  return { ok: true, status: 200, data: { urgency: "normal" } };
}

for (let attempt = 1; attempt <= maxAttempts; attempt += 1) {
  const result = await simulatedRemoteCall(attempt);
  console.log({ attempt, status: result.status }); // never log a key
  if (result.ok) {
    console.log(result.data);
    break;
  }
  if (result.status !== 429 || attempt === maxAttempts) {
    throw new Error(`Stopped after HTTP ${result.status}`);
  }
  const baseMs = 100 * 2 ** (attempt - 1);
  const jitterMs = Math.floor(Math.random() * 100);
  await new Promise((resolve) => setTimeout(resolve, baseMs + jitterMs));
}

Run it with safe, non-secret configuration:

$env:MAX_INPUT_CHARS="2000"
$env:MAX_ATTEMPTS="4"
$env:DAILY_REQUEST_BUDGET="20"
node guard-lab.mjs

The first two simulated calls return 429; the third succeeds. Change MAX_ATTEMPTS to 2 and confirm that the runner stops. Change MAX_INPUT_CHARS to 10 and confirm that it refuses the input before any call. These are testable gates, unlike a note saying “be careful with costs.”

Build Cost Controls in Layers

Money controls should not depend on a single estimate because pricing, tokenization, and tool charges can change. Use several layers:

1. Pre-request limits: cap input characters or measured tokens, requested output size, media length, batch size, tool count, and concurrency.
2. Request ledger: record timestamp, project, model identifier, request ID, status, and usage units returned by the provider. Do not record the secret or raw private prompt.
3. Provider controls: configure project budgets, alerts, permissions, and usage monitoring where available. An alert is not always a hard stop; verify the provider's current behavior.
4. Application stop: reject optional jobs after a local daily or monthly threshold. Keep essential and non-essential queues separate.
5. Review: compare the ledger with the provider dashboard. Investigate unexplained spikes instead of assuming they are legitimate growth.

Use the provider's live pricing page at planning time and store the date and assumptions beside your estimate. Do not hard-code a lesson's price into production logic.

Failure Cases and Correct Responses

• Key committed to Git: revoke it first, then clean history where appropriate and review access logs. A later .gitignore entry does not undo exposure.
• Key printed by debugging: remove or redact the log, rotate the key, and check every log sink and screenshot destination.
• Key used in browser code: move the request behind an authenticated server and issue a new key.
• Every error retried: retry only transient categories. Authentication, malformed input, and permission errors require correction, not backoff.
• Retry storm: add jitter, concurrency limits, maximum attempts, and a queue.
• Duplicate side effect: a retry that sends a message, creates an invoice, or records an order may run twice. Use an idempotency key or application record and require confirmation for high-impact actions.
• Budget based only on request count: input size, output size, media, and tools can make requests different. Reconcile provider-reported usage.
• One shared production key: attribution and revocation become difficult. Separate projects and environments where available.

🇵🇰 Pakistan Angle

For a Pakistani student, freelancer, or small agency, predictable spending matters more than demonstrating the largest possible prompt. Start with synthetic fixtures, small batches, short outputs, and a low application threshold. Check live provider availability, payment requirements, taxes, and prices directly before committing to a client quote; they can change and may differ by account.

Unstable connectivity can cause a client to give up while the server still completes a request. Assign a unique job ID, store status server-side, and let the client ask for the result rather than blindly submitting the same paid job again. On shared office systems, avoid permanent shell-history entries containing secrets, lock deployment dashboards, and remove access promptly when a collaborator leaves. Do not use CNIC images, bank statements, customer chats, or employee records as debugging fixtures.

Hands-On Exercise: Pass a Secret-and-Budget Review

Create a local safety-pack folder containing:

1. .env.example with names but no real values, plus a .gitignore that excludes the real secret file.
2. guard-lab.mjs with input, attempt, and daily-request caps.
3. THREAT-MODEL.md listing at least six leak paths: repository, browser bundle, logs, screenshots, shell history, and departed collaborators. Give each an owner and response.
4. BUDGET-RUNBOOK.md defining the stop threshold, retry categories, maximum attempts, concurrency, dashboard reconciliation, and who may approve a limit increase.
5. Terminal evidence showing one successful simulated run, one input-cap refusal, and one maximum-attempt refusal.

Use only a dummy value such as not-a-real-key if you test environment loading. Search the repository for common key variable names before committing, but never paste an actual key into the search command.

Completion Rubric

• Pass — secrecy: no credential exists in code, Git history, logs, screenshots, or submitted artifacts.
• Pass — bounded execution: input, attempts, concurrency policy, and daily work have explicit limits.
• Pass — error policy: transient errors retry with bounded jitter; authorization and validation errors stop.
• Pass — cost evidence: the runbook names provider usage records and a local ledger, not a guessed fixed price.
• Pass — incident readiness: the response begins with revoke/rotate and includes ownership and log review.
• Needs revision: the browser receives a provider key, retries are unlimited, or an alert is incorrectly described as a guaranteed hard cap.

Sources

• OpenAI: Production best practices
• OpenAI: Rate limits
• OpenAI: Error codes
• Node.js: Environment variables and .env files
• OWASP: Secrets Management Cheat Sheet

Key takeaway: Treat credentials, capacity, and spend as enforceable boundaries: keep secrets off clients and out of Git, classify errors, retry only transient failures with bounded jitter, and stop requests when measured limits are reached.