Filters define which records enter a research sample. If thresholds change after seeing results, the backtest becomes a story fitted to the answer. Write a filter specification first, version it, and make the pipeline produce both accepted and rejected records with reason codes.
Start with four independent dimensions. Time checks observation time, event cutoff, and allowed horizon. Status distinguishes open, closed, resolved, cancelled, and unknown. Spread checks whether both quote sides exist and whether their difference is within a predeclared descriptive threshold. Coverage measures how many expected observation windows were actually captured.
Filters must not overwrite. Produce a decision object:
{
"source_id": "fixture-27",
"filter_version": "eligibility_v2",
"eligible": false,
"reasons": ["STALE_QUOTE", "LOW_COVERAGE"],
"observed_at_utc": "2026-07-19T08:00:00Z",
"input_hash": "..."
}
Evaluate all reasons rather than stopping at the first. This supports a full exclusion matrix and reveals interactions. Keep unknown status as unknown; do not map it to open. Treat boundary times consistently, preferably half-open intervals such as [start, end). Test exactly-on-boundary records.
Spread is a data-quality and paper-cost input, not a signal. A 0.02 quoted spread does not guarantee a fill cost of 0.02. Coverage is also not volume: ten snapshots captured in one minute do not cover a ten-hour window. Define expected windows and count distinct covered buckets.
Report filters as a waterfall: discovered, schema-valid, within time, known status, quote-complete, coverage-eligible, final sample. Include multi-reason counts separately so readers do not mistake waterfall order for the only cause of exclusion.
Add a reconciliation assertion after every stage. If 250 normalized rows produce 180 eligible and 68 rejected, the pipeline must stop because two rows are unexplained. Store the exact filter configuration and code version beside the counts so a later threshold change cannot inherit the old report. Review a random accepted and rejected sample manually; this catches inverted comparisons and timezone mistakes that totals alone may miss.
🇵🇰 Pakistan Angle
Internet outages can reduce observation coverage systematically around local times. Report coverage by UTC and Pakistan-time hour to detect this operational bias. Do not “repair” gaps using later quotes. If a fixture covers mainly foreign topics, state that it does not represent Pakistani events.
Hands-On Exercise
Write a versioned filter function and twelve boundary fixtures: exact cutoff, one second before/after, unknown status, missing bid, missing ask, negative spread, very wide spread, duplicate timestamps, sparse windows, dense single window, and adequate coverage. Produce accepted and rejected CSVs plus a reason-count report.
Completion Rubric
- Filter rules and thresholds are fixed and versioned before evaluation.
- Every record is retained with all applicable reason codes.
- Time boundaries and unknown statuses have explicit tests.
- Spread and coverage are not described as profit signals.
- The published denominator reconciles to raw validated input.
Sources
Key takeaway: Versioned filters make exclusions visible and prevent thresholds from quietly following the desired result.