AI Prediction Markets
0/15 complete

Module 03 · Signal Aggregation

Using Python to Aggregate and Score Signals Automatically

Automate deterministic arithmetic on an approved fixture, not data collection or trading. Python should validate rows, group common origins, compute a transparent paper score, and preserve warnings. It must refuse malformed or future-dated evidence rather than silently coercing it.

// concept

Define the Fixture

Create signals.json using synthetic or approved public data:

// json4 lines
[
  {"claim_id":"c1","origin_id":"o1","weight":0.7,"direction":1,"approved":true},
  {"claim_id":"c2","origin_id":"o2","weight":0.3,"direction":-1,"approved":true}
]

direction is an educational encoding: 1 supports, -1 contradicts, 0 neutral. A weight is not a probability. In production-quality research, dimensions and provenance should remain separate; this compact fixture exists to teach validation and grouping.

// concept

Write a Safe Aggregator

// python26 lines
import json
from pathlib import Path

rows = json.loads(Path("signals.json").read_text(encoding="utf-8"))
required = {"claim_id", "origin_id", "weight", "direction", "approved"}

origins = {}
for row in rows:
    if set(row) != required:
        raise ValueError(f"unexpected schema for {row.get('claim_id', 'unknown')}")
    if row["approved"] is not True:
        continue
    if row["direction"] not in {-1, 0, 1}:
        raise ValueError("direction must be -1, 0, or 1")
    if not 0 <= row["weight"] <= 1:
        raise ValueError("weight outside 0..1")
    if row["origin_id"] in origins:
        raise ValueError("duplicate origin requires manual dependency rule")
    origins[row["origin_id"]] = row

denominator = sum(row["weight"] for row in origins.values())
score = None if denominator == 0 else sum(
    row["weight"] * row["direction"] for row in origins.values()
) / denominator

print(json.dumps({"approved_origins": len(origins), "paper_score": score}, indent=2))

The program rejects duplicate origins instead of double counting. It does not convert paper_score into odds, submit an order, or make a recommendation.

// concept

Add Reproducibility

Pin Python version/dependencies, hash the fixture, record cutoff UTC time, store configuration, and create tests for empty input, invalid weights, duplicate origins, unknown keys, and non-approved claims. Write output to a new versioned file rather than overwriting evidence.

For Brier score after resolution, use frozen probabilities that were created by a separate documented mapping/calibration step—not the raw directional score. Confirm outcomes and exclude/label unresolved or invalid contracts according to a predeclared rule.

// worked_example

Worked Example

Two articles share origin_id=o1. A naive script counts both and pushes the score positive. The safe program rejects duplication, forcing a reviewer to decide dependency. After grouping, the result is near neutral and marked sensitive to one source.

The automation improves consistency by refusing ambiguity, not by predicting better automatically.

// failure_cases

Failure Cases

7 cases to diagnose

  • Fetching live data and calculating on it without preserving raw input.

  • Hard-coding API secrets or wallet keys.

  • Treating missing, null, or failed fetch as zero.

  • Allowing unknown schema fields silently.

  • Double counting common origins.

  • Mapping a score to probability without calibration.

  • Adding an order-placement function “for later.”

// pakistan_angle

Pakistan Angle

Store timestamps in UTC and render PKT separately. Use Unicode UTF-8 and tests for Urdu text/identifiers, but keep numeric fields locale-independent. Never place CNIC, phone, account, wallet, or private messages in fixtures.

Learners can run this entirely offline. No exchange login, funding, VPN, or paid data is required.

// hands_on

Hands-On Exercise

Implement the script, add at least eight fixture rows and five automated tests, then introduce a duplicate, invalid weight, missing key, and empty dataset. Produce a versioned output with input hash and a plain-language limitation note.

// completion_rubric

Completion Rubric

3 grading bands

  • Complete

    schema, dependencies, failure states, tests, provenance, and offline-only boundaries are enforced.

  • Needs revision

    math works but malformed/missing data or reproducibility is weak.

  • Not complete

    the script includes credentials, live order placement, or uncalibrated recommendations.

// sources

Sources