Capstone: Ship an AI Workflow and Its…

Direct answer

Your capstone is complete when an authorized user can run one bounded AI workflow, inspect its evidence, observe safe failure, and review a package proving how it was built and tested. The goal is not the largest app. The goal is a small, deployed system whose scope, data, controls, evaluations, operations, and limitations another professional can understand.

Ship both the workflow and an evidence pack. A screen recording of one successful prompt is not proof of reliability. Your pack must include failed tests, corrections, threat controls, cost measurement, and a runbook alongside the demo.

Choose a bounded problem

Good capstone options include:

a cited assistant over approved public or synthetic documents;
a support-ticket triager that drafts but does not send replies;
a structured extractor that validates document fields and escalates uncertainty;
a content-quality reviewer that applies a defined rubric;
a read-only operations investigator that assembles an evidence packet.

Avoid a vague “business agent that does everything.” Choose one user, one trigger, one useful final artifact, and a clear non-goal. A capstone can be valuable without direct access to money, private accounts, or external write tools.

Write a one-sentence contract:

For [user], when [trigger] occurs, the workflow produces [artifact or decision support] using [approved evidence], while it never [important prohibited action] and escalates when [conditions].

Example: “For a manufacturer’s new operations assistant, when a worker asks an SOP question, the workflow returns a concise answer with current section citations, while it never invents safety steps and escalates missing, conflicting, or restricted evidence.”

Worked capstone: an SOP evidence assistant

Use synthetic SOPs for equipment checks, quality inspection, maintenance escalation, and shift handover. Do not include real factory secrets or safety instructions in a public repository. The architecture is:

authenticated web form
  -> input validation and request ID
  -> permission and active-version filters
  -> hybrid retrieval over approved SOP chunks
  -> evidence sufficiency/conflict gate
  -> cited answer or abstention
  -> trace, metrics, and user feedback

The assistant is read-only. It cannot start equipment, approve maintenance, or replace the named safety supervisor. Every answer shows SOP title, version, section, and effective date. If two active SOPs conflict, the system returns ESCALATED and names the document-owner role.

Build a labelled test set with routine questions, exact equipment codes, paraphrases, Roman Urdu phrasing, unanswerable questions, superseded SOPs, unauthorized sections, prompt injection inside a document, and a simulated retrieval outage. Measure retrieval recall, permission and stale-result failures, citation support, correct abstention, latency, and cost per completed answer.

Your deployment has staging and production configuration, server-side secrets, structured logs with no raw questions by default, a maximum request size, rate limits, a source-version monitor, and a kill switch. A small synthetic check runs regularly: it asks a known question and confirms the expected citation, not merely HTTP status 200.

Make the release gate executable

Store release criteria in code where possible. This provider-neutral JavaScript example fails a build when critical evidence falls below the agreed threshold:

const report = {
  retrievalRecall: 0.94,
  citationSupport: 0.98,
  permissionLeaks: 0,
  staleSourceAnswers: 0,
  correctAbstentions: 0.92,
  duplicateEffects: 0,
};

const gates = [
  [report.retrievalRecall >= 0.90, "retrieval recall"],
  [report.citationSupport >= 0.95, "citation support"],
  [report.permissionLeaks === 0, "permission isolation"],
  [report.staleSourceAnswers === 0, "freshness"],
  [report.correctAbstentions >= 0.90, "abstention"],
  [report.duplicateEffects === 0, "idempotency"],
];

const failed = gates.filter(([pass]) => !pass).map(([, name]) => name);
if (failed.length) throw new Error(`Release blocked: ${failed.join(", ")}`);

The sample numbers are illustrative, not universal standards. Set thresholds from consequence, baseline performance, and stakeholder review. Some risks—such as cross-user data leakage—should have zero tolerance, while quality metrics need a documented trade-off between answering and abstaining.

Build the evidence pack

Use a structure a reviewer can navigate:

evidence-pack/
  README.md
  scope-and-non-goals.md
  architecture.png
  data-and-source-register.md
  schemas-and-tool-contracts.md
  evals/
    dataset.jsonl
    rubric.md
    report.md
  red-team-report.md
  threat-model.md
  privacy-and-consent.md
  cost-log.csv
  cost-and-latency-review.md
  deployment-checklist.md
  runbook.md
  changelog.md
  demo-script.md
  case-study.md

README.md explains the user problem, setup, safe sample data, current status, and limitations. The source register names authority, audience, version, freshness, and permission. The eval report gives dataset composition, per-slice results, failures, and what changed. The threat model maps assets and controls. The cost log records dated usage, currency assumptions, and cost per successful task. The case study turns the problem, build decisions, measured evidence, failures, limits, and next experiment into one truthful portfolio narrative. The runbook covers provider outage, rate limits, stale index, security incident, and runaway cost.

Include at least three before-and-after failures. For example: Roman Urdu queries initially missed the correct SOP; a version filter allowed an expired chunk; a citation linked the right document but wrong section. Show the fix and regression ID. This proves engineering judgement better than hiding every mistake.

Demonstrate the difficult cases

A concise investor, employer, or client demo can follow this order:

State the user's problem and the system boundary.
Run one normal case and open the citation.
Run an unanswerable case and show useful abstention.
Run an unauthorized or stale-source case and show the control.
Open one trace with sanitized tool and state events.
Show the eval report and a fixed failure.
Show cost per completed task and the current operational limit.
End with limitations and the next measured improvement.

Do not claim a synthetic demo is a client deployment. Do not invent users, revenue, accuracy, testimonials, or income. Say exactly what was measured, on what dataset, at what version, and which parts remain mocked.

Portfolio story without hype

Describe the capstone with evidence:

Problem: the specific user delay, error, or knowledge gap.
Constraint: source quality, privacy, mobile access, cost, or authorization.
Decision: why a script, model-assisted workflow, or bounded agent was chosen.
Build: architecture, schemas, retrieval, approvals, and deployment.
Proof: eval results by slice, red-team outcomes, and operational metrics.
Limits: cases the system refuses or sends to a human.
Next experiment: one measurable improvement, not a promise of transformation.

This is useful for employment and freelance conversations because it demonstrates requirements thinking, implementation, testing, security, and communication. It does not guarantee work or earnings.

Failure cases that block completion

Only the happy-path demo works; no labelled test set exists.
Citations are decorative links rather than supporting excerpts.
A user-supplied role controls access to private documents.
The repository contains an API key, private record, or customer file.
The workflow can perform a consequential write with no approval or idempotency.
Logs cannot connect a failure to workflow and source versions.
The eval report gives one average and hides weak language or permission slices.
A deployed endpoint has no limits, monitoring, owner, or rollback.
The evidence pack claims real-client impact from synthetic data.
The runbook says “restart it” but cannot reconcile ambiguous side effects or stale state.

🇵🇰 Pakistan Angle

Choose a problem recognizable in Pakistan without using anyone's private data: a synthetic admissions circular assistant, a COD return triager, a branch-policy finder, an SME product-information reviewer, or a public service information navigator. Test English and Roman Urdu queries, mobile widths, reconnect behaviour, PKR cost reporting, and Pakistan Standard Time in operator views.

Keep the public demo light enough for mobile data and provide screenshots or a short recorded fallback if a live provider fails during presentation. Clearly label synthetic businesses and documents. If the workflow touches payments, health, employment, education decisions, or official rights, narrow it to information and evidence preparation, include human escalation, and seek current professional review before real use. The local context should improve requirements and testing—not become a marketing decoration.

Hands-on deliverable

Ship all of the following:

a deployed bounded workflow using synthetic or authorized data;
source code with safe setup instructions and no secrets;
an explicit scope contract, non-goals, and human escalation path;
architecture and data-flow diagrams;
versioned source register, prompts/configuration, schemas, and tool contracts;
a labelled eval dataset with routine, edge, multilingual, unanswerable, unauthorized, stale, and adversarial cases;
an automated report and release gate;
red-team report, threat model, privacy/consent note, cost log, cost review, case study, and runbook;
a sanitized trace, dashboard screenshot, and rollback evidence;
an eight-minute demo script that includes normal and safe-failure cases.

Completion rubric

Key takeaway: Your professional proof is not that AI produced an answer—it is that you scoped, built, tested, secured, deployed, and documented a workflow whose useful and unsafe behaviours are both visible.

Capstone: Ship an AI Workflow and Its Evidence Pack

Turn this lesson into one checked practice output