Module 03 · Signal Aggregation
Using Python to Aggregate and Score Signals Automatically
Open lesson + course map
On this lesson
Course outline
Module 1 · Prediction Markets 101
Module 2 · Market Research
Module 3 · Signal Aggregation
Module 4 · Paper Decision Logic
Module 5 · Risk and Responsible Research
Automate deterministic arithmetic on an approved fixture, not data collection or trading. Python should validate rows, group common origins, compute a transparent paper score, and preserve warnings. It must refuse malformed or future-dated evidence rather than silently coercing it.
// concept
Define the Fixture
Create signals.json using synthetic or approved public data:
[
{"claim_id":"c1","origin_id":"o1","weight":0.7,"direction":1,"approved":true},
{"claim_id":"c2","origin_id":"o2","weight":0.3,"direction":-1,"approved":true}
]direction is an educational encoding: 1 supports, -1 contradicts, 0 neutral. A weight is not a probability. In production-quality research, dimensions and provenance should remain separate; this compact fixture exists to teach validation and grouping.
// concept
Write a Safe Aggregator
import json
from pathlib import Path
rows = json.loads(Path("signals.json").read_text(encoding="utf-8"))
required = {"claim_id", "origin_id", "weight", "direction", "approved"}
origins = {}
for row in rows:
if set(row) != required:
raise ValueError(f"unexpected schema for {row.get('claim_id', 'unknown')}")
if row["approved"] is not True:
continue
if row["direction"] not in {-1, 0, 1}:
raise ValueError("direction must be -1, 0, or 1")
if not 0 <= row["weight"] <= 1:
raise ValueError("weight outside 0..1")
if row["origin_id"] in origins:
raise ValueError("duplicate origin requires manual dependency rule")
origins[row["origin_id"]] = row
denominator = sum(row["weight"] for row in origins.values())
score = None if denominator == 0 else sum(
row["weight"] * row["direction"] for row in origins.values()
) / denominator
print(json.dumps({"approved_origins": len(origins), "paper_score": score}, indent=2))The program rejects duplicate origins instead of double counting. It does not convert paper_score into odds, submit an order, or make a recommendation.
// concept
Add Reproducibility
Pin Python version/dependencies, hash the fixture, record cutoff UTC time, store configuration, and create tests for empty input, invalid weights, duplicate origins, unknown keys, and non-approved claims. Write output to a new versioned file rather than overwriting evidence.
For Brier score after resolution, use frozen probabilities that were created by a separate documented mapping/calibration step—not the raw directional score. Confirm outcomes and exclude/label unresolved or invalid contracts according to a predeclared rule.
// worked_example
Worked Example
Two articles share origin_id=o1. A naive script counts both and pushes the score positive. The safe program rejects duplication, forcing a reviewer to decide dependency. After grouping, the result is near neutral and marked sensitive to one source.
The automation improves consistency by refusing ambiguity, not by predicting better automatically.
// failure_cases
Failure Cases
7 cases to diagnose
Fetching live data and calculating on it without preserving raw input.
Hard-coding API secrets or wallet keys.
Treating missing, null, or failed fetch as zero.
Allowing unknown schema fields silently.
Double counting common origins.
Mapping a score to probability without calibration.
Adding an order-placement function “for later.”
// pakistan_angle
Pakistan Angle
Store timestamps in UTC and render PKT separately. Use Unicode UTF-8 and tests for Urdu text/identifiers, but keep numeric fields locale-independent. Never place CNIC, phone, account, wallet, or private messages in fixtures.
Learners can run this entirely offline. No exchange login, funding, VPN, or paid data is required.
// hands_on
Hands-On Exercise
Implement the script, add at least eight fixture rows and five automated tests, then introduce a duplicate, invalid weight, missing key, and empty dataset. Produce a versioned output with input hash and a plain-language limitation note.
// completion_rubric
Completion Rubric
3 grading bands
- Complete
schema, dependencies, failure states, tests, provenance, and offline-only boundaries are enforced.
- Needs revision
math works but malformed/missing data or reproducibility is weak.
- Not complete
the script includes credentials, live order placement, or uncalibrated recommendations.
// sources
Sources
4 official sources — check every claim yourself