This lab turns immutable snapshots into a local dashboard that answers operational questions: When was data last observed? Which records passed validation? How much is missing? How wide are quoted spreads? It does not recommend an action. The dashboard is successful when a reviewer can trace every displayed number back to a raw file and see uncertainty without opening the code.
Create three layers: raw/ stores untouched JSON, normalized/ stores validated rows with a schema version, and reports/ stores generated HTML. Never let the report read directly from the network. That separation makes the same report reproducible after an endpoint changes or your internet drops.
Use a normalized record like this:
{
"schema_version": "1.0",
"source_id": "public-market-id",
"question": "Sample question supplied by fixture",
"observed_at_utc": "2026-07-19T08:00:00Z",
"status": "open",
"bid": 0.54,
"ask": 0.58,
"quote_freshness_seconds": 45,
"missing_fields": [],
"raw_sha256": "fixture-hash"
}
Validate probability-like fields as numbers between zero and one, require UTC timestamps, and reject unknown schema versions. Compute midpoint only when both sides exist. Compute spread as ask - bid, but label it “quoted spread,” not transaction cost. A synthetic paper fill later needs an explicit slippage assumption; this dashboard has no fill logic.
The first panel should show data health: snapshot time, age, total records, valid records, quarantined records, and missing-field rate. The second can show descriptive distributions by topic and status. The third is an audit table with source ID, rule link, quote fields, freshness, and validation result. Use neutral colors for prices; reserve red/amber for errors and staleness, not “sell” and “buy.”
Add a visible banner: “Research dashboard · public/read-only data · no signals or execution.” Include dataset version and generated-at time on every page. If all records fail, render an error state with counts and the path to the error log. Never render yesterday’s report as fresh merely because the generator failed today.
Acceptance test
Change one fixture’s ask to 1.4. The row must be quarantined, the health panel must show one failure, and the report must not average that value. Remove a bid from another row: the midpoint and spread should display “unavailable,” while the record remains visible. Finally, rerun with identical input; the analytical values should match exactly.
🇵🇰 Pakistan Angle
Build the report as a lightweight static HTML file so it runs on an ordinary laptop and can be reviewed during unreliable connectivity. Use UTC internally and optionally display Pakistan Standard Time as a clearly labelled view. Keep English/Roman Urdu labels readable, but do not translate market rules in a way that changes their meaning.
Hands-On Exercise
Produce the three-layer folder, validate at least ten fixture records, and generate the three dashboard panels. Save a screenshot and an audit.json containing dataset hash, parser version, counts, and error categories. Ask another person to trace one card back to its raw record without your help.
Completion Rubric
- The report reads saved normalized data, never a live endpoint.
- Every metric traces to a raw hash and schema version.
- Invalid values are quarantined and missing values stay explicit.
- Safety banner, freshness, dataset version, and generation time are visible.
- The dashboard contains no action, signal, or profit language.
Sources
Key takeaway: A useful market dashboard makes provenance, freshness, and missingness easier to see than the headline price.