Build a Reliable Multi-Step Workflow

A reliable AI workflow is a state machine with explicit inputs, checkpoints, failure policies, and ownership—not a long prompt that asks a model to do everything. Separate deterministic work from model work, validate every boundary, save progress after meaningful stages, and require human approval before consequential output leaves the system. Then a timeout becomes a recoverable state instead of a mystery.

This lesson builds a local workflow for drafting a customer-facing product update from synthetic approved facts. The example intentionally uses a deterministic draft adapter so you can test orchestration without paying for calls. You can later replace that one adapter with a model request while keeping validation, state, evidence, approval, and publication controls unchanged.

Design the States Before the Happy Path

Use six stages:

received: assign a run ID and preserve the immutable input.
validated: check required fields, types, permissions, and limits.
drafted: call the model adapter with only approved facts.
checked: run deterministic claim, length, and format checks.
awaiting_approval: stop and display the exact proposed artifact.
approved or rejected: record a human decision; publication belongs to a separate, idempotent action.

Each state should have an entry condition and an artifact. Store the schema and workflow version beside the run. “Step 3 probably completed” is not a state. Never infer success only because a client connection closed or a worker stopped logging.

Decide which failures may retry. Network timeouts and 429 responses may be transient. Invalid input, failed authorization, unsupported claims, and human rejection are not. Side effects require an idempotency record before retry.

Worked Example: A Checkpointed Draft Pipeline

Create workflow.mjs:

import { mkdir, readFile, writeFile } from "node:fs/promises";
import { randomUUID } from "node:crypto";

const runsDir = "runs";
await mkdir(runsDir, { recursive: true });

const fixture = {
  product_id: "DEMO-42",
  product_name: "Reusable notebook cover",
  approved_facts: [
    "Available in blue and charcoal",
    "Fits the demo A5 notebook insert"
  ],
  audience: "existing demo-store customers",
  max_characters: 240
};

function validateInput(input) {
  const errors = [];
  if (!/^DEMO-\d+$/.test(input.product_id ?? "")) errors.push("invalid product_id");
  if (typeof input.product_name !== "string" || !input.product_name.trim()) errors.push("missing product_name");
  if (!Array.isArray(input.approved_facts) || input.approved_facts.length < 1) errors.push("approved_facts required");
  if (!Number.isInteger(input.max_characters) || input.max_characters > 500) errors.push("invalid max_characters");
  if (errors.length) throw new Error(errors.join("; "));
}

async function save(run) {
  run.updated_at = new Date().toISOString();
  await writeFile(`${runsDir}/${run.run_id}.json`, JSON.stringify(run, null, 2), "utf8");
}

// Replace only this adapter when connecting a model.
async function draftAdapter(input) {
  return `${input.product_name}: ${input.approved_facts.join(". ")}.`;
}

function checkDraft(input, draft) {
  const errors = [];
  if (draft.length > input.max_characters) errors.push("draft exceeds character limit");
  for (const fact of input.approved_facts) {
    if (!draft.includes(fact)) errors.push(`approved fact missing: ${fact}`);
  }
  const bannedClaims = ["guaranteed", "best in Pakistan", "limited stock"];
  for (const claim of bannedClaims) {
    if (draft.toLowerCase().includes(claim.toLowerCase())) errors.push(`unsupported claim: ${claim}`);
  }
  return errors;
}

async function start(input) {
  const run = {
    run_id: randomUUID(),
    workflow_version: "1.0.0",
    state: "received",
    created_at: new Date().toISOString(),
    input,
    history: [{ state: "received", at: new Date().toISOString() }]
  };
  await save(run);

  validateInput(input);
  run.state = "validated";
  run.history.push({ state: "validated", at: new Date().toISOString() });
  await save(run);

  run.draft = await draftAdapter(input);
  run.state = "drafted";
  run.history.push({ state: "drafted", at: new Date().toISOString() });
  await save(run);

  run.check_errors = checkDraft(input, run.draft);
  run.state = run.check_errors.length ? "rejected_by_checks" : "awaiting_approval";
  run.history.push({ state: run.state, at: new Date().toISOString() });
  await save(run);
  return run;
}

const result = await start(fixture);
console.log({ run_id: result.run_id, state: result.state, draft: result.draft });
console.log("No customer message was sent. Human approval is still required.");

Run node workflow.mjs, open the generated run file, and trace every state. The immutable input, draft, checks, and state history make the run reviewable. In a real system, do not keep sensitive input in a repository or unencrypted local folder; this lab uses synthetic facts only.

Add a separate approve.mjs that accepts a run ID and a human decision. It should refuse approval unless the current state is awaiting_approval, require a reviewer identifier and note, then update state to approved or rejected. Do not add a send or publish function yet. Approval and execution are different events.

Replace the Adapter Without Rewriting the Workflow

A model adapter should accept the validated object and return a narrow structured result. Its responsibilities are API request, timeout, bounded retry for transient failures, response parsing, and safe request metadata. It should not decide who is authorized, change approved facts, approve its own draft, or publish.

Use an interface such as:

async function modelDraftAdapter(input, context) {
  // context: run_id, deadline, model configuration, attempt number
  // return: { text, response_id, usage, model }
}

Save the response ID and usage beside the drafted checkpoint. If the worker crashes after the remote response but before saving, the run may be ambiguous. A robust queue records leases and attempt IDs, and the worker checks for an existing completed stage before repeating it. For high-value or side-effecting steps, use provider idempotency support where available plus your own durable business record.

Reliability Patterns That Matter

Timeouts: set a deadline for each remote stage and a larger workflow deadline. Do not let abandoned requests occupy workers indefinitely.
Bounded retries: retry only classified transient errors with jitter; keep attempt count and last safe error.
Checkpointing: persist after state changes, not only at the end.
Idempotency: a stage sees the same run and stage key and returns the prior result instead of repeating work.
Dead-letter path: after attempts are exhausted, move the run to review with enough safe metadata to diagnose it.
Concurrency control: prevent two workers from approving or publishing the same version.
Versioning: store workflow, prompt, schema, tool, and model configuration versions.
Observability: count runs by state, failure category, latency, and review outcome; never treat a single average as the whole distribution.
Human gate: show the exact final artifact, source facts, check results, recipient or destination, and material cost/action.

Failure Injection, Not Hope

Modify the lab one case at a time:

Set max_characters to 10; validation or checks must stop the workflow.
Add “guaranteed” inside the adapter output; the claim check must reject it.
Throw a simulated timeout in draftAdapter; the run must remain at its last durable state with a safe failure record.
Run approval twice; the second attempt must refuse a state that is no longer awaiting approval.
Remove an approved fact from the draft; the completeness check must fail.

Do not edit the generated run file to fake recovery. Add a resume command that reads the stored state, confirms versions, and continues only from an explicitly supported checkpoint.

🇵🇰 Pakistan Angle

Small Pakistani teams often operate across laptops, shared spreadsheets, WhatsApp approvals, and variable connectivity. A durable run ID and explicit state prevents “please resend” from silently creating duplicate work. Keep the formal approval in the controlled system even if a notification arrives through chat. Record the approved artifact and reviewer, not an ambiguous thumbs-up detached from the final version.

Design low-cost stages: validate and deduplicate locally, batch only compatible tasks, send the minimum authorized context, and stop optional work at a budget boundary. Preserve Urdu and Roman-Urdu text in UTF-8 and have an appropriate reader review customer-facing language. Do not place customer phone numbers, addresses, CNIC details, payment records, or private chats in workflow fixtures or public traces.

Hands-On Exercise: Demonstrate Recovery and Control

Extend the lab and deliver:

workflow.mjs with at least six named states, durable run files, workflow version, validation, draft adapter, deterministic checks, and a stop at approval.
approve.mjs with state precondition, reviewer ID, decision note, and duplicate-approval refusal.
resume.mjs that reads a run, permits only documented recoverable states, and never repeats a completed stage.
Five synthetic failure fixtures covering invalid input, unsupported claim, timeout, missing fact, and duplicate approval.
OPERATIONS.md defining retryable/non-retryable failures, maximum attempts, timeout, dead-letter owner, idempotency key, logging fields, and privacy exclusions.
A state-history record showing one clean run, one rejected-by-checks run, and one recovered run. No external message may be sent.

Completion Rubric

Pass — state clarity: every stage has an entry condition, artifact, and recorded transition.
Pass — recovery: the workflow resumes from a durable checkpoint and does not repeat completed work.
Pass — deterministic gates: invalid input, missing facts, banned claims, and duplicate approval fail without model judgment.
Pass — human authority: the exact artifact stops for named approval and approval does not itself publish.
Pass — operations: timeouts, retry categories, attempts, dead-letter ownership, and safe metrics are documented.
Needs revision: the workflow is one prompt, state exists only in memory, every error retries, or a model may approve/publish its own output.

Sources

Key takeaway: Build AI workflows as durable state machines: validate locally, isolate the model adapter, checkpoint every stage, retry only safe transient failures, test recovery, and keep approval and external action under independent human control.