Deterministic extraction handles facts that do not require generative interpretation: timestamps, known identifiers, exact phrases, document language, source type, and whether mandatory fields exist. It is cheap, testable, and reproducible. Use it to shrink the problem before asking a model anything.
Represent each document as immutable text plus metadata. Normalize Unicode for comparison while retaining original bytes. Extract dates only with an explicit locale and record both raw string and parsed UTC value. A parser should return AMBIGUOUS_DATE for 07/08/26 unless the source format is documented; guessing contaminates later features.
Rules should have IDs and versions. For example, R_REQUIRED_SOURCE_DATE_V1, R_TITLE_MATCH_V2, and R_CUTOFF_BEFORE_OBSERVATION_V1. Each result includes rule ID, pass/fail/unknown, evidence span offsets, and explanation. Do not hide unknown inside false.
Build cheap rejection gates: document outside time window, source host not approved, content hash duplicated, publication time after research cutoff, missing resolution rule, or language unsupported. These gates reduce cost and prevent future information reaching a model. They must be evaluated before feature construction.
Avoid fragile keyword certainty. A word match creates a candidate span, not a factual conclusion. Negation, quotation, and sarcasm can reverse meaning. Store surrounding context and ask later tiers to review it. For names, use a controlled alias table with provenance rather than approximate matching everything.
Evaluate each rule on a labelled fixture with precision, recall, unknown rate, and error examples, but do not optimize all thresholds on that same fixture. Keep a holdout set for regression review. A new version is acceptable only when intended fixes appear, unrelated cases do not regress unexpectedly, and every changed result has a machine-generated diff plus reviewer note.
Test strategy
Use table-driven tests with positive, negative, ambiguous, and adversarial examples. Include timezone offsets, Urdu text, repeated headings, quoted denials, malformed HTML, and content added after cutoff. Freeze parser libraries and record their versions. A rule update should produce a diff of changed case outcomes.
🇵🇰 Pakistan Angle
Pakistani dates, multilingual names, and transliterated places require careful aliasing. PKT is often used informally but UTC+05:00 or Asia/Karachi is safer in stored metadata. Preserve Urdu script; do not force Roman Urdu conversion for matching without recording the transformation and its limits.
Hands-On Exercise
Write five versioned rules and a table of twenty fixtures. Generate a JSON Lines rule trace for every case. Add a cutoff test proving a later document cannot enter an earlier paper decision. Compare two alias-table versions and explain every changed match.
Completion Rubric
- Original evidence and normalized comparison text are both retained.
- Rules return pass/fail/unknown with spans and version IDs.
- Ambiguous dates and language cases abstain rather than guess.
- Cutoff enforcement prevents lookahead.
- Rule upgrades produce reviewable diffs.
Sources
Key takeaway: Deterministic rules should remove ambiguity where possible and expose it explicitly everywhere else.