Module 02 · Python Bot Architecture — Ek Professional Bot Ka Skeleton
Safe Scheduler — Idempotent Research Jobs Automate Karo
Open lesson + course map
On this lesson
Course outline
Module 1 · Market Systems and Safety — Pehle Boundaries Samjho
Module 2 · Python Bot Architecture — Ek Professional Bot Ka Skeleton
Module 3 · Market Data Pipeline — Read-Only Evidence Safely Fetch Karo
Module 4 · AI Research Engine — Extraction Se Human Review Tak
Module 5 · Strategy Research — Hypothesis Se Paper Test Tak
Module 6 · Paper Execution Engine — Synthetic Fills Only
Module 7 · Risk Controls — Estimation Error and Paper Limits
Module 8 · Database and Monitoring — Audit Logging and Model Evaluation
Module 9 · Deploying the Research Service — Read-Only and Measured
A scheduler should automate evidence capture, not multiply mistakes. Jobs can overlap after a slow network call, rerun after a restart, or fire twice during deployment. Design every job as if duplicate delivery is normal. The goal is idempotency: repeating the same logical run does not create contradictory analytical events.
Derive a deterministic run_key from job name, scheduled UTC window, config hash, and dataset source. Before work begins, insert a run record with a unique constraint on that key. If it already exists in SUCCEEDED state, skip. If it is RUNNING but its lease is stale, record recovery and resume from immutable inputs. Never delete the first attempt.
Use APScheduler with an explicit timezone, max_instances=1, coalescing rules, and a misfire grace period chosen from the data freshness requirement. These options reduce overlap, but the database uniqueness rule remains the real protection. A second process or redeployment can bypass in-memory scheduling controls.
The job flow is:
- Acquire the run lease.
- Resolve and hash configuration.
- Fetch or load raw data with a bounded deadline.
- Validate and persist normalized records.
- Run deterministic research and paper simulation.
- Append metrics and report artifact hashes.
- Mark success only after every required artifact exists.
On failure, append a sanitized error category and mark FAILED; do not pretend an old report is current. Retry only errors classified transient, with a capped attempt count and jitter. Validation, forbidden-host, and schema-version errors require operator review.
Clock discipline
Schedule and store in UTC. The display may show Pakistan time. Avoid using “now” repeatedly inside calculations; capture one run timestamp and pass it through. In tests, inject a clock so misfire and stale-lease behavior is deterministic.
// pakistan_angle
Pakistan Angle
Power cuts and unstable connectivity make recovery a first-class feature. A missed job should be logged, not backfilled with invented snapshots. After connectivity returns, capture a new observation and preserve the gap. Set schedules that respect limited data bundles and use cached fixtures for practice.
// hands_on
Hands-On Exercise
Implement a scheduled fixture job and a runs table with a unique run_key. Trigger it twice concurrently; prove one succeeds and one reports duplicate without duplicated events. Simulate interruption after normalization, expire the lease, and recover using the same raw hash. Show the gap rather than manufacturing a missed quote.
// completion_rubric
Completion Rubric
5 checks — tick as you verify
// sources
Sources
3 official sources — check every claim yourself