Module 04 · AI Research Engine — Extraction Se Human Review Tak
4-Tier Research Architecture — Rules Se Review Tak
Open lesson + course map
On this lesson
Course outline
Module 1 · Market Systems and Safety — Pehle Boundaries Samjho
Module 2 · Python Bot Architecture — Ek Professional Bot Ka Skeleton
Module 3 · Market Data Pipeline — Read-Only Evidence Safely Fetch Karo
Module 4 · AI Research Engine — Extraction Se Human Review Tak
Module 5 · Strategy Research — Hypothesis Se Paper Test Tak
Module 6 · Paper Execution Engine — Synthetic Fills Only
Module 7 · Risk Controls — Estimation Error and Paper Limits
Module 8 · Database and Monitoring — Audit Logging and Model Evaluation
Module 9 · Deploying the Research Service — Read-Only and Measured
An AI research engine should make uncertainty easier to audit, not automate conviction. Use four tiers: deterministic extraction, optional model-assisted classification, independent verification, and human review. Every tier can abstain. A later tier may add evidence or reject an earlier result, but it cannot erase the record of what happened.
Tier 0 ingests only approved source documents and assigns immutable source IDs, capture times, hashes, and publication metadata. Tier 1 applies deterministic rules: date parsing, identifier matching, keyword extraction, and required-field checks. Tier 2 asks a model for structured candidate claims tied to source IDs. Tier 3 runs an independent check—another deterministic rule set or separately prompted model—against the same locked evidence. Tier 4 is the named human decision and abstention gate. The course calls this “four-tier research” after ingestion.
Define a case file with case_id, question version, evidence IDs, extraction version, prompt/model identifier when used, candidate claim, contradiction list, confidence category, and final status. Confidence is a review category such as LOW/MEDIUM/HIGH, not a calibrated probability unless you have separately validated it.
Models never browse freely inside the decision function. A retrieval step first builds an approved evidence packet. The model must cite only packet source IDs and return JSON matching a strict schema. Unsupported IDs, invented quotations, missing contradictions, or invalid JSON cause rejection. The system never converts prose directly into a paper action.
Use explicit statuses: INSUFFICIENT_EVIDENCE, CONFLICTING_EVIDENCE, READY_FOR_HUMAN_REVIEW, HUMAN_APPROVED_FOR_PAPER_TEST, and REJECTED. Approval means the hypothesis may enter a synthetic evaluation; it does not mean correct, profitable, legal, or suitable for real activity.
Define tier-level service objectives around integrity rather than speed: packet completeness, invalid source-ID rejection, abstention on missing rules, contradiction recall on a labelled fixture, and audit-record completeness. Measure model-free and model-assisted paths separately. If an optional model is unavailable, deterministic extraction and human review continue; the system must not invent a summary or reuse a result from a different packet hash.
// pakistan_angle
Pakistan Angle
Local reporting may mix English, Urdu, and Roman Urdu and may lack machine-readable archives. Preserve original text and translation separately, record the translator/model, and ask a bilingual reviewer for ambiguous claims. Never upload private WhatsApp messages, CNICs, or personal records as evidence.
// hands_on
Hands-On Exercise
Create one case from three supplied public documents, including one contradiction. Run deterministic extraction, produce model-shaped JSON manually or with an optional provider, validate source IDs, and complete a human review form. Then remove a critical source and prove the status becomes INSUFFICIENT_EVIDENCE.
// completion_rubric
Completion Rubric
5 checks — tick as you verify
// sources
Sources
3 official sources — check every claim yourself