Python Automation
Workflow automation, data pipelines, web scraping, REST APIs, data apps with Streamlit, Flask web services, and ML-powered tooling — all in Python.
All
DuckDB as an Analytical Read Layer: Querying a Gold Store Without a Database Server
DuckDB is an in-process analytical database that runs inside your Python application, reads Parquet files natively, and handles complex SQL including window functions, QUALIFY, and MEDIAN — with no server to manage. This tutorial covers the read-only connection pattern used in the flight disruption serving layer, thread-safety in a FastAPI threadpool, and the queries that power the reliability API.
FastAPI for ML Serving: Lifespan, Pydantic Validation, and Graceful Degradation
FastAPI's type annotations, Pydantic models, and lifespan context manager solve three common ML serving problems: loading expensive artifacts before requests start, validating inputs without manual checks, and returning useful errors when dependencies aren't ready. This tutorial walks through the patterns used in the flight disruption prediction API.
SQL Analytics You Can Actually Trust: Building Reliability Marts with dbt
dbt turns SQL SELECT statements into tested, documented, dependency-aware data models. This tutorial builds the flight reliability marts used by the dashboard — airport, route, and carrier delay rates — starting from a staging view and layering on not-null, uniqueness, relationship, and range tests that catch data quality problems before they reach the API.
Calibration, SHAP, and ONNX: Building a Delay Predictor That's Actually Honest
ROC-AUC tells you how well a model ranks flights by delay risk. It doesn't tell you whether a 40% prediction means 40% of flights actually get delayed. Those are different properties, and the second one matters when you're building a risk gauge. This tutorial covers isotonic probability calibration, SHAP explanations per prediction, ONNX export with a parity test, and why all three belong in the same ML lifecycle.
Making Data Leakage a Pipeline Failure, Not a Debugging Session
A rolling delay rate that includes same-day flights is leakage. A temporal split that shuffles randomly is leakage. In a flight delay prediction pipeline, leakage produces models that look great on held-out data and fail immediately in production. This tutorial covers the shared contract module, Spark window functions bounded to strictly prior days, schema assertions that catch banned features before training, and a temporal split that respects the direction of time.
Building a Multi-Stage ETL That Doesn't Lose Work Under Failure with NATS JetStream
A walkthrough of the backpressure-aware ingestion pipeline in EvidenceLens: durable consumers, manual acknowledgment, how a 60-second ack_wait caused a redelivery storm under load, and a backpressure loop that pauses consumption when the downstream indexer falls behind.