Forecasting Metrics That Don’t Lie (Early Release Preorder)
📘 Forecasting Metrics That Don’t Lie
Evaluate and improve your forecasts with a rigorous, practitioner-focused guide to forecasting metrics — from MAE and RMSE to MASE, WRMSSE, CRPS, and modern probabilistic scoring.
Why this book?
Forecasting models are only as good as the metrics used to measure them. Yet many teams still rely on outdated or misleading metrics (hello, MAPE) that quietly bias decisions, reward the wrong behavior, and produce “accuracy” numbers that don’t survive contact with reality.
This book is a comprehensive, practical reference focused entirely on forecast evaluation. It blends clear explanations, Python-ready recipes, and real-world case studies spanning retail, finance, and energy — with a single goal:
help you measure what matters, and stop being fooled by metrics.
What you’ll learn
✔ How MAE, RMSE, MAPE, sMAPE, WAPE, MASE, RMSSE, and WRMSSE actually behave in real systems
✔ Why popular metrics systematically mislead portfolios, hierarchies, and low-demand series
✔ How to benchmark forecasts properly using naive baselines and scaled errors
✔ How to evaluate accuracy across hundreds or thousands of time series without statistical nonsense
✔ How to detect and control forecast bias before it turns into stockouts, waste, or budget drift
✔ How to evaluate probabilistic forecasts using CRPS and quantile loss (pinball)
✔ How forecasting competitions shaped (and sometimes distorted) metric best practices
✔ How to design forecasting KPIs that align with business cost structures
✔ Why “high accuracy” numbers often mean absolutely nothing without context
Who this is for
- Data scientists / ML engineers building forecasting models and needing evaluation that’s robust and defensible
- Retail, manufacturing, finance, and energy professionals who rely on forecasts for operational decisions
- Researchers, analysts, and students who want a single authoritative reference on forecast accuracy
No heavy math background required — concepts are explained clearly, with practical intuition and implementation guidance.
Living book: buy once, get all updates
This is an actively developed “living” book. You get all future updates automatically as new chapters and case studies are added.
Current coverage includes: core metric foundations and the most common failure modes (including MAPE pitfalls).
Planned expansions include: bias diagnostics, portfolio/hierarchical evaluation, scaled benchmarking, probabilistic scoring, and domain-specific case studies.
Why now?
Whether you’re tuning a model, running forecasting operations, or presenting results to executives, the wrong metric can sink an otherwise good forecasting system — and the right metric bundle can instantly make your forecasts more actionable.
This book gives you the clarity, tools, and confidence to evaluate forecasts like a professional.
🔥 Start mastering the metrics that drive better forecasting decisions.