---
title: "Energy Demand Forecasting"
subtitle: "Rolling-window backtested hourly electricity demand forecasting — PJM AEP region, 2004–2018"
author: "Alvin Alias"
date: today
format:
html:
theme: cosmo
toc: true
toc-location: left
number-sections: true
self-contained: true
code-fold: true
code-tools: true
execute:
echo: true
warning: false
message: false
---
```{python}
#| include: false
import sys; sys.path.insert(0, '..')
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import plotly.express as px
from pathlib import Path
figures = Path('../figures')
```
## Executive Summary
This report presents a rigorous evaluation of three hourly electricity demand forecasting models —
naive seasonal baseline, SARIMA, and Facebook Prophet — evaluated on rolling 7-day forecast windows
spanning the PJM AEP dataset.
**Headline result:** SARIMA achieved WAPE **7.9%** across **84** rolling-window evaluations
(7-day forecast horizon, 2005–2018) vs. **8.6%** for the naive seasonal baseline — a **7.8%**
improvement. The honest surprise: **Prophet (10.4% WAPE) underperformed even the naive
baseline.** Its smooth additive seasonalities cannot track the sharp hour-to-hour load
dynamics that a same-hour-last-week lag captures for free. SARIMA also won peak-hour WAPE
(8.5% vs. 9.0% naive vs. 10.7% Prophet) and was the most stable model in 2018, when both
other models' errors spiked.
**The retail transfer:** This exact framework — train on historical data, forecast the next period,
compare plan vs. actual, roll forward — is how Costco, Walmart, and Target run store-level demand
planning. Replace "MW" with "units" and the same `src/backtesting.py` module is directly reusable.
| Metric | Naive seasonal | SARIMA | Prophet |
|--------|---------------|--------|---------|
| WAPE | 8.6% | **7.9%** | 10.4% |
| MAE (MW) | 1,327 | **1,223** | 1,604 |
| Directional acc. | 86.0% | **87.2%** | 73.1% |
| Peak-hour WAPE | 9.0% | **8.5%** | 10.7% |
| Bias (MW) | +128 | **+55** | +135 |
| n windows | 84 | 84 | 84 |
---
## Data
**Source:** PJM Hourly Energy Consumption — AEP region (American Electric Power, Ohio/Indiana service area)
**Provider:** [Kaggle](https://www.kaggle.com/datasets/robikscube/hourly-energy-consumption)
**Coverage:** hourly load history, with exact date range and row count computed from the downloaded file
```{python}
#| label: tbl-data-summary
#| tbl-cap: "Dataset summary"
from src.features import load_pjm_data
df = load_pjm_data('../data/raw/AEP_hourly.csv')
pd.DataFrame({
"Metric": ["Total rows", "Date range", "Mean load (MW)", "Min load (MW)", "Max load (MW)", "Missing values"],
"Value": [f"{len(df):,}", f"{df.index.min().date()} → {df.index.max().date()}",
f"{df['load_mw'].mean():.0f}", f"{df['load_mw'].min():.0f}",
f"{df['load_mw'].max():.0f}", f"{df['load_mw'].isna().sum()}"],
})
```
---
## Exploratory Data Analysis
### Seasonal Patterns
Electricity demand exhibits three nested seasonalities: **daily** (peak at 5–8pm),
**weekly** (weekdays higher than weekends), and **annual** (summer cooling + winter heating double peak).
This multi-scale structure is the core challenge for forecasting.
```{python}
#| label: fig-load-patterns
#| fig-cap: "Hourly load patterns — daily, weekly, and seasonal"
from IPython.display import Image
Image('../figures/01_load_patterns.png')
```
### Seasonal Decomposition
```{python}
#| label: fig-decomposition
#| fig-cap: "STL decomposition — trend + seasonality + residual"
Image('../figures/01_decomposition.png')
```
### Stationarity (ADF Test)
```{python}
#| label: stationarity
from statsmodels.tsa.stattools import adfuller
# ADF on the most recent 3 years (autolag over the full 121k-row series is
# minutes of runtime for no extra insight); maxlag=48 covers two daily cycles
recent = df.loc['2015-08-01':, 'load_mw'].dropna()
result = adfuller(recent, maxlag=48, autolag=None)
print(f"ADF statistic: {result[0]:.4f} | p-value: {result[1]:.6f}")
print(f"Conclusion: {'Stationary (p < 0.05)' if result[1] < 0.05 else 'Non-stationary'}")
```
---
## Models
### Naive Seasonal Baseline
Predict the value from the same hour last week (168h lag). This is the hardest-to-beat
simple baseline for hourly energy demand because weekly seasonality is the dominant signal.
Any model that cannot beat the naive baseline is not useful.
### SARIMA
SARIMA(p,d,q)(P,D,Q,s) — statistical time-series model using autoregressive and moving-average
terms plus seasonal differencing.
**Parameter selection:** ADF test confirmed near-stationarity (d=0). ACF/PACF plots
suggested AR(1) + MA(1) with seasonal period s=24 (daily cycle).
```{python}
#| label: fig-acf-pacf
#| fig-cap: "ACF and PACF — daily resampled series"
Image('../figures/03_acf_pacf.png')
```
### Prophet
Facebook Prophet decomposes the time series into trend + seasonality + holiday components.
Unlike SARIMA, it handles non-stationarity automatically and supports multiple seasonality
periods (daily + weekly + annual) natively.
```{python}
#| label: fig-prophet-components
#| fig-cap: "Prophet component decomposition — trend, weekly, annual, holidays"
Image('../figures/04_prophet_components.png')
```
---
## Rolling-Window Backtesting
### Methodology
> "A single train/test split can hide how model performance changes over time.
> Rolling-window backtesting shows performance across seasons, years, and demand regimes —
> which is what you need before trusting a forecasting system in production."
**Setup:** 365-day training window → 7-day forecast → step forward 8 weeks → repeat across all data.
Yields 84 evaluation windows spanning 2005–2018 — every season and demand regime — each with a
fresh model fit. (SARIMA is re-fit on the most recent 90 days of each training window; it is a
short-memory model, and longer histories multiply fit time without improving a 7-day-ahead forecast.)
```{python}
#| label: fig-backtest-wape
#| fig-cap: "WAPE over time (8-week rolling mean) — drift and seasonal error regimes"
Image('../figures/05_backtest_wape_over_time.png')
```
### Drift Analysis
The WAPE-over-time chart highlights when model error rises during specific seasons, years, or load
regimes. This is the operational reason to use a rolling backtest: a model that looks good on one
holdout period may be less reliable during high-load seasons or unusual demand regimes.
---
## Results
```{python}
#| label: tbl-results
#| tbl-cap: "Model comparison — aggregated over all rolling windows"
from src.backtesting import BacktestEvaluator
bt = BacktestEvaluator()
results = pd.read_parquet('../data/processed/backtest_results.parquet')
bt.aggregate_metrics(results).round(4)
```
### What the numbers say
**SARIMA wins, and Prophet loses to the naive baseline.** SARIMA had the best WAPE in 43 of
84 windows (naive: 26, Prophet: 15) and leads on every aggregate metric — WAPE, MAE, RMSE,
directional accuracy, bias, and peak-hour WAPE. Prophet's smooth trend + seasonality
decomposition systematically misses sharp hour-to-hour transitions, which is most visible in
its directional accuracy (73% vs. 87% for SARIMA). The lesson worth stating plainly: **a
same-hour-last-week lag is a strong baseline for hourly load, and a popular library default
does not beat it.**
**Drift:** per-year WAPE shows SARIMA is also the most robust model late in the series — in
2018, naive error rose to 13.2% and Prophet's to 19.2%, while SARIMA held at 8.9%. Models
that re-fit on recent data (SARIMA's 90-day window) adapt to regime change; Prophet's
year-long seasonal curves lag behind it.
```{python}
#| label: fig-sample-forecast
#| fig-cap: "Sample 7-day forecast vs. actual — representative window"
Image('../figures/05_sample_forecast.png')
```
---
## Retail Demand Planning Transfer
The methodology in this project is directly transferable to retail demand planning:
| Component | Energy domain | Retail domain |
|-----------|--------------|---------------|
| Target variable | Load (MW) | Units sold / inventory turns |
| Granularity | Hourly | Daily / weekly |
| Training window | 365 days | 52–104 weeks |
| Forecast horizon | 7 days | Next week / next month |
| Rolling backtest | ✓ — same code | ✓ — same `src/backtesting.py` |
| Plan-vs-actual metric | WAPE / MAPE | WAPE / MAPE / bias |
| Regime changes | Seasonal peaks, economic shifts, weather-driven load changes | Promotions, stockouts, holidays, demand shocks |
The `src/backtesting.py` module from this project is imported directly by the
[Retail Returns Intelligence](https://github.com/aalias01/retail-returns-intelligence) project
for rolling 30-day plan-vs-actual evaluation of the return-prediction model.
---
## Limitations
1. **Single region:** AEP region only. Performance may differ for other grid operators
with different load profiles (industrial vs. residential mix, climate zone).
2. **No external regressors:** Weather data not included in this version. Temperature is
the strongest predictor of HVAC-driven demand; adding it would likely improve SARIMA/Prophet.
3. **No quantile forecasts:** Point forecasts only. Grid operators use P10/P50/P90 uncertainty
bands for reserve planning — a production system would add quantile regression or
conformal prediction intervals.
4. **SARIMA computation time:** Each window requires a fresh SARIMA fit (~10–30s).
This is feasible for backtesting but not suitable for real-time inference at high frequency.
In production, SARIMA would be replaced by a pre-trained LightGBM or TCN model.
---
*Report generated with [Quarto](https://quarto.org). Source:
[github.com/aalias01/energy-demand-forecasting](https://github.com/aalias01/energy-demand-forecasting)*
