Module 02 · Agent Frameworks
Comparing Agent Frameworks: When to Build vs. Buy
Open lesson + course map
On this lesson
Course outline
Module 1 · Agent Fundamentals
Module 2 · Agent Frameworks
Module 3 · Swarm Orchestration
Module 4 · Operational Guardrails
Module 5 · Shipping a Real Agent System
An agent framework packages model calls, tools, state, routing, tracing, and sometimes deployment. A managed platform may reduce setup; a library may offer control; plain application code may be clearest for a small workflow. Compare evidence, not feature-count marketing.
After this lesson, you can run a framework evaluation using one fixed task and a weighted decision record.
// concept
Start With Requirements
List non-negotiables before products:
| Area | Questions |
|---|---|
| model/runtime | which providers and deployment regions are required? |
| tools | schemas, auth, approval, timeout, idempotency? |
| state | durable checkpoints, versioning, deletion, tenancy? |
| observability | traces, redaction, export, retention? |
| reliability | retry, resume, queue, concurrency, rollback? |
| security | SSO, roles, secrets, audit, data controls? |
| operations | hosting, upgrades, on-call, support, exit path? |
| economics | model, platform, storage, egress, and staff cost? |
Use current official documentation for candidates; capabilities and pricing change.
// concept
Prototype One Representative Task
Give every option the same task, tools, data, and evaluation set. Measure completion accuracy, invalid actions blocked, tool calls, latency, model tokens/cost where available, recovery after interruption, trace usefulness, and engineering time.
Do not compare a vendor demo with a production implementation. Include failure tests: tool timeout, malformed model output, duplicate event, missing permission, state resume, and sensitive-data redaction.
// concept
Decide Build, Library, or Managed
- Plain code: best when flow is small, deterministic, and needs precise control.
- Framework/library: useful when tested graph/state/tool abstractions remove repeated work.
- Managed platform: useful when hosted tracing, evaluation, security, and operations justify dependency and cost.
Document lock-in: stored state format, proprietary tools, trace export, model portability, and migration effort.
// worked_example
Worked Example
A Karachi software agency evaluates an invoice-intake assistant. It must extract allowed fields, match a purchase order, prepare an exception case, and never approve payment. The team prototypes plain TypeScript, one open-source graph library, and one managed service.
Weights are correctness 30%, safety 25%, operability 20%, cost 15%, and developer experience 10%. The plain implementation wins because the flow has four deterministic states and the managed platform adds little. The decision is revisited if workflows or teams grow. No framework is declared universally superior.
// failure_cases
Failure Cases to Diagnose
7 cases to diagnose
Choosing from a social-media benchmark
reproduce the actual task.
Ignoring failure recovery
test restart and duplicate delivery.
Token price treated as total cost
include engineering and operations.
Framework handles auth assumed
inspect where authorization is enforced.
No data export path
test state and trace portability.
One huge prototype
compare the smallest representative slice.
Version pinned nowhere
record versions and upgrade policy.
// pakistan_angle
Pakistan Angle
Include payment method availability, foreign-currency billing, tax/accounting treatment, support timezone, and data-transfer requirements in the decision. Do not recommend a tool only because a free tier exists today; calculate the expected PKR monthly cost under low, expected, and stress usage.
Connectivity and local hiring matter. A simpler system the team can test and recover may beat a sophisticated framework dependent on one specialist. For client delivery, disclose third-party processors and obtain authority before sending real business data.
// hands_on
Hands-On Exercise
5 steps
Write the requirement matrix and weights.
Select one representative task and fixed evaluation set.
Compare plain code, a library, and managed option.
Run six failure tests.
Publish a decision record with revisit conditions.
// completion_rubric
Completion Rubric
6 checks — tick as you verify
// sources
Sources
3 official sources — check every claim yourself
// check_yourself
Check yourself
4 questions · answers and options are taken word-for-word from this course
1 / 4 · diagnose
Your work shows this failure mode: “Ignoring failure recovery.” What does the lesson tell you to do about it?