LightGBM

Calibration, SHAP, and ONNX: Building a Delay Predictor That's Actually Honest

ROC-AUC tells you how well a model ranks flights by delay risk. It doesn't tell you whether a 40% prediction means 40% of flights actually get delayed. Those are different properties, and the second one matters when you're building a risk gauge. This tutorial covers isotonic probability calibration, SHAP explanations per prediction, ONNX export with a parity test, and why all three belong in the same ML lifecycle.

Advanced

40 minutes

Aug 10, 2026

Python LightGBM SHAP ONNX +3 more

Python Automation

Making Data Leakage a Pipeline Failure, Not a Debugging Session

A rolling delay rate that includes same-day flights is leakage. A temporal split that shuffles randomly is leakage. In a flight delay prediction pipeline, leakage produces models that look great on held-out data and fail immediately in production. This tutorial covers the shared contract module, Spark window functions bounded to strictly prior days, schema assertions that catch banned features before training, and a temporal split that respects the direction of time.

Intermediate

25 minutes

Aug 10, 2026

Python Apache Spark PySpark Machine Learning +5 more

Python Automation Web Development

Flight Disruption Intelligence: A Full-Stack Delay Predictor on Public Data and Free Infrastructure

An end-to-end flight delay prediction platform spanning medallion lakehouse, calibrated LightGBM with SHAP explanations, Spark Structured Streaming, dbt reliability marts, FastAPI serving, and a SvelteKit dashboard — built entirely on public BTS/NOAA/OpenSky data, self-hostable for $0/year, with a contract module that enforces leakage prevention across every pipeline stage.

Python Apache Spark dbt LightGBM +12 more