A baseline tests whether complexity adds value. Build a transparent paper rule that consumes validated snapshots and produces ABSTAIN, PAPER_YES, or PAPER_NO with fictional point limits. It is not a recommendation and has no market connection.
Example: enter a paper observation only when quote freshness, coverage, rule clarity, and spread pass frozen thresholds; direction follows whether midpoint is above or below a predeclared boundary; otherwise abstain. This deliberately simple rule exposes data plumbing, costs, and evaluation mistakes before adding AI.
Separate three functions: eligible(record, config), decide(record, config), and size_points(record, config). Eligibility returns all rejection reasons. Decision returns direction and hypothesis version. Size returns a fixed small number of fictional points capped globally; it never references money, account value, or a growth formula.
Evaluate against base-rate and “always abstain” controls. Report coverage—the proportion of eligible cases where a paper direction was emitted—alongside Brier score or classification metrics. A rule that acts on two easy cases can look accurate while being useless as a general forecast. Include all discovered and rejected denominators.
Simulate quoted spread, fixed latency, slippage stress, and stale-data rejection. Do not treat midpoint as a guaranteed fill. Run sensitivity grids chosen before results and show whether conclusions disappear under modest assumptions. If they do, the correct result is fragility.
Add property-based invariants: points never become negative, higher assumed cost cannot improve the same scenario, stale inputs cannot create a decision, and duplicate inputs cannot create duplicate events. Inspect the largest positive and negative paper contributions individually. If a few records dominate, report leave-one-family-out results rather than hiding dependence behind an aggregate score.
Freeze code and configuration before the comparison and store their hashes in each decision row. Review abstentions by reason and time period; a sudden drop in coverage can reveal a parser or data-source problem even when the metric on remaining cases improves. Reproduce one complete decision manually from the raw fixture.
🇵🇰 Pakistan Angle
Use fictional points with no PKR mapping. Avoid examples framed around earning or replacing income. The portfolio artifact demonstrates Python architecture, testing, and research integrity useful for engineering work; it is not a financial service or signal product.
Hands-On Exercise
Implement the three pure functions against a versioned synthetic fixture. Add boundary tests for every threshold. Produce a decision ledger with source hash, config hash, reasons, direction, points, and cutoff time. Compare baseline, stressed, and always-abstain results with full denominators.
Completion Rubric
- Eligibility, decision, and fictional sizing are pure and separate.
- Thresholds and stress assumptions are fixed before scoring.
- Coverage and rejected denominators accompany performance metrics.
- No midpoint is treated as a guaranteed fill.
- Code has no credentials, currencies, or write-capable client.
Sources
Key takeaway: A transparent baseline reveals whether data quality and assumptions support research before complexity is allowed.