Advanced Time Series Architectures & Production Forecasting

1. What This Notebook Covers#

This notebook is the final notebook in the Time Series section.

You have already learned:

1. Time Series Foundations & Exploratory Data Analysis
2. Classical Statistical Models: ARIMA & SARIMA
3. Modern Machine Learning for Time Series
4. Deep Learning for Time Series: RNNs, LSTMs & GRUs

Now we move from individual models toward advanced forecasting systems.

This notebook focuses on:

• Transformers for time series
• Prophet
• Advanced forecasting metrics
• Walk-forward validation
• TimeSeriesSplit
• Multi-step forecasting strategies
• Rolling retraining
• Production forecasting architecture
• Monitoring and model drift
• Choosing between statistical, machine-learning, deep-learning, and modern architectures
• A complete end-to-end forecasting project

The most important idea in this notebook is:

A strong forecasting system is not just a strong model. It is a combination of good data preparation, leakage-safe validation, appropriate forecasting strategy, reliable deployment, and continuous monitoring.

---

2. Learning Objectives

By the end of this notebook, you should be able to:

• Explain the intuition behind Transformers for time series.
• Understand attention and self-attention at a practical level.
• Describe when Transformers are useful for forecasting.
• Build a basic Transformer-style forecasting model.
• Explain the strengths and limitations of Prophet.
• Build a Prophet forecasting pipeline.
• Calculate MAE, RMSE, MAPE, and sMAPE.
• Understand why percentage-based metrics can be dangerous.
• Perform walk-forward validation.
• Use TimeSeriesSplit correctly.
• Compare one-step and multi-step forecasting.
• Implement recursive, direct, and multi-output forecasting.
• Design rolling retraining strategies.
• Understand production forecasting architecture.
• Monitor forecast quality and data/model drift.
• Select an appropriate model family for a business problem.
• Build an end-to-end production-oriented forecasting workflow.

---

3. Why Advanced Time Series Forecasting Is Different

In ordinary machine learning, we often assume that observations can be randomly shuffled.

For time series, this assumption is usually wrong.

Consider:

›January → February → March → April → May → June

If you train on June and test on March, information from the future can leak into the past.

Therefore:

›Time order matters.

A production forecasting system must answer several questions:

1. What information was available at prediction time?
2. How far into the future are we predicting?
3. How often will new observations arrive?
4. How often should the model be retrained?
5. How will forecast quality be measured?
6. What happens when the data distribution changes?
7. What happens when an external feature becomes unavailable?
8. How will predictions be served to downstream systems?

These questions are often more important than simply choosing the most sophisticated algorithm.

---

4. Transformer Models for Time Series

4.1 Why Transformers?#

Transformers became popular because of their ability to model relationships between different positions in a sequence using attention.

Traditional RNNs process sequences sequentially:

text
Ask AICopy
x1 → RNN → h1
 ↓
x2 → RNN → h2
 ↓
x3 → RNN → h3

The hidden state carries information forward.

Transformers instead allow sequence positions to directly attend to other positions.

Conceptually:

text
Ask AICopy
x1 ─┐
x2 ─┼──→ Attention ──→ representations
x3 ─┤
x4 ─┘

This can make it easier to model long-range relationships.

---

5. Self-Attention Intuition

Suppose we are forecasting today's demand.

The model may need to understand relationships between:

• yesterday's demand
• demand seven days ago
• demand four weeks ago
• recent promotions
• holidays
• recent trends

Attention allows the model to learn which historical positions are important for the current prediction.

A simplified attention calculation is:

$$Attention(Q,K,V) = softmax\left(\frac{QK^T}{\sqrt{d_k}}\right)V$$

Where:

• $Q$ = Query
• $K$ = Key
• $V$ = Value
• $d_k$ = dimensionality of the keys

You do not need to memorize the equation immediately.

The important intuition is:

The model learns how strongly one position should pay attention to other positions.

---

6. Transformer Components

A typical Transformer architecture contains:

text
Ask AICopy
Input sequence
 ↓
Embedding / Projection
 ↓
Positional information
 ↓
Self-Attention
 ↓
Feed-Forward Network
 ↓
Normalization
 ↓
Repeated Transformer blocks
 ↓
Forecasting head
 ↓
Prediction

For time series, numerical features must generally be projected into a representation space before entering the Transformer.

For example:

text
Ask AICopy
[temperature, sales, promotion]
 ↓
 Linear projection
 ↓
 embedding vector

---

7. Positional Information

Attention by itself does not inherently understand chronological order.

For example:

›Monday → Tuesday → Wednesday

must be distinguishable from:

›Wednesday → Tuesday → Monday

Therefore, sequence models commonly include positional information.

A simplified representation is:

$$Z = X + P$$

where:

• $X$ = input representation
• $P$ = positional encoding

Modern time-series architectures may use more specialized temporal embeddings rather than a simple sinusoidal encoding.

---

8. When Transformers Are Useful

Transformers can be useful when:

• sequences are long
• there are many interacting variables
• long-range dependencies matter
• large training datasets are available
• multiple forecasting horizons are required
• the problem contains complex temporal patterns

They are not automatically better than LSTMs, tree models, or statistical models.

A major practical consideration is data volume.

A Transformer trained on a tiny dataset may overfit badly.

---

9. Basic Transformer Forecasting Example

Below is a simplified PyTorch-style architecture.

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
import torch
import torch.nn as nn


class TimeSeriesTransformer(nn.Module):
 def __init__(
 self,
 input_size,
 d_model=64,
 nhead=4,
 num_layers=2,
 output_size=1
 ):
 super().__init__()

 self.input_projection = nn.Linear(input_size, d_model)

 encoder_layer = nn.TransformerEncoderLayer(
 d_model=d_model,
 nhead=nhead,
 batch_first=True
 )

 self.encoder = nn.TransformerEncoder(
 encoder_layer,
 num_layers=num_layers
 )

 self.output_layer = nn.Linear(
 d_model,
 output_size
 )

 def forward(self, x):
 x = self.input_projection(x)

 x = self.encoder(x)

 last_state = x[:, -1, :]

 return self.output_layer(last_state)

Input shape:

›(batch_size, timesteps, features)

For example:

›(32, 60, 5)

means:

• 32 sequences
• 60 time steps per sequence
• 5 features per time step

---

10. Transformer Design Considerations

Important hyperparameters include:

Sequence length#

How much history should the model see?

text
Ask AICopy
24 hours
7 days
30 days
90 days

Embedding dimension#

Larger representations increase model capacity but also computational cost.

Number of attention heads#

Multiple attention heads allow the model to learn different relationships.

Number of Transformer layers#

More layers increase representational capacity.

Dropout#

Useful for reducing overfitting.

Learning rate#

Often one of the most important training hyperparameters.

---

11. Transformer Limitations

Transformers also have disadvantages.

Computational cost#

Self-attention can become expensive for long sequences.

For standard attention, the attention matrix grows approximately with:

$$O(T^2)$$

where $T$ is the sequence length.

Data requirements#

Large models often benefit from large datasets.

Complexity#

A Transformer can be much harder to operate than a simple statistical model.

Interpretability#

Attention weights should not automatically be treated as complete explanations of model behavior.

---

12. Prophet

Prophet is a forecasting framework designed around decomposable time-series structure.

Conceptually:

$$y(t) = g(t) + s(t) + h(t) + \epsilon_t$$

where:

• $g(t)$ = trend
• $s(t)$ = seasonality
• $h(t)$ = holidays/events
• $\epsilon_t$ = error

Prophet is particularly convenient for business time series containing:

• strong seasonal patterns
• trend changes
• holiday effects
• daily/weekly/yearly seasonality
• missing observations

---

13. Prophet Data Format

Prophet expects two primary columns:

›ds
y

Example:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
import pandas as pd

df = pd.DataFrame({
 "ds": pd.date_range(
 "2025-01-01",
 periods=365,
 freq="D"
 ),
 "y": demand_values
})

Then:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
from prophet import Prophet

model = Prophet()

model.fit(df)

---

14. Forecasting With Prophet

Create future dates:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
future = model.make_future_dataframe(
 periods=30
)

Generate forecasts:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
forecast = model.predict(future)

Useful columns include:

text
Ask AICopy
ds
yhat
yhat_lower
yhat_upper

Where:

• yhat = forecast
• yhat_lower = lower uncertainty bound
• yhat_upper = upper uncertainty bound

---

15. Prophet Seasonality

Prophet can model common seasonal patterns.

For example:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
model = Prophet(
 yearly_seasonality=True,
 weekly_seasonality=True,
 daily_seasonality=False
)

Additional seasonalities can be added:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
model.add_seasonality(
 name="monthly",
 period=30.5,
 fourier_order=5
)

The exact seasonal structure should be validated rather than blindly enabled.

---

16. Prophet With Holidays

Business forecasting often depends on holidays.

Example:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
holidays = pd.DataFrame({
 "holiday": ["holiday_a", "holiday_b"],
 "ds": pd.to_datetime([
 "2025-01-01",
 "2025-12-25"
 ])
})

model = Prophet(
 holidays=holidays
)

This can help when demand changes systematically around known events.

---

17. Prophet Limitations

Prophet is not a universal solution.

It may be less suitable when:

• extremely complex nonlinear relationships dominate
• very high-frequency interactions matter
• many external variables drive the target
• the problem requires sophisticated sequence representation
• the dataset is extremely small and noisy

Always compare it against a baseline and other candidate models.

---

18. Forecasting Metrics

A forecasting model needs quantitative evaluation.

The most common metrics include:

• MAE
• RMSE
• MAPE
• sMAPE

No single metric is perfect.

---

19. Mean Absolute Error (MAE)

MAE is:

$$MAE = \frac{1}{n} \sum_{i=1}^{n} |y_i-\hat{y}_i|$$

It measures average absolute error.

Example:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
from sklearn.metrics import mean_absolute_error

mae = mean_absolute_error(
 y_true,
 y_pred
)

Advantages:

• easy to interpret
• same units as the target
• less sensitive to extreme errors than RMSE

---

20. Root Mean Squared Error (RMSE)

RMSE is:

$$RMSE = \sqrt{ \frac{1}{n} \sum_{i=1}^{n} (y_i-\hat{y}_i)^2 }$$

Python:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
from sklearn.metrics import mean_squared_error
import numpy as np

rmse = np.sqrt(
 mean_squared_error(
 y_true,
 y_pred
 )
)

RMSE penalizes large errors more strongly.

This is useful when large forecasting mistakes are particularly costly.

---

21. Mean Absolute Percentage Error (MAPE)

MAPE is:

$$MAPE = \frac{100}{n} \sum_{i=1}^{n} \left| \frac{y_i-\hat{y}_i}{y_i} \right|$$

A basic implementation:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
def mape(y_true, y_pred):
 y_true = np.asarray(y_true)
 y_pred = np.asarray(y_pred)

 mask = y_true != 0

 return np.mean(
 np.abs(
 (y_true[mask] - y_pred[mask])
 / y_true[mask]
 )
 ) * 100

---

22. Why MAPE Can Be Dangerous

MAPE becomes problematic when actual values are zero or close to zero.

For example:

Mathematical Formulation
Copy
Actual = 0
Prediction = 10

The percentage error is undefined.

Even when values are merely small, MAPE can become extremely large.

Therefore:

Never choose MAPE automatically just because it is expressed as a percentage.

---

23. Symmetric MAPE (sMAPE)

One common formulation is:

$$sMAPE = \frac{100}{n} \sum_{i=1}^{n} \frac{ 2|y_i-\hat{y}_i| }{ |y_i|+|\hat{y}_i| }$$

Implementation:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
def smape(y_true, y_pred):
 y_true = np.asarray(y_true)
 y_pred = np.asarray(y_pred)

 denominator = (
 np.abs(y_true)
 + np.abs(y_pred)
 )

 mask = denominator != 0

 return np.mean(
 2 * np.abs(
 y_true[mask] - y_pred[mask]
 ) / denominator[mask]
 ) * 100

sMAPE reduces some of the problems associated with ordinary MAPE, although it also has limitations and multiple formulations exist.

---

24. Metric Selection

A practical strategy is:

The most important rule is:

Choose the metric based on the decision the forecast supports.

---

25. Always Compare Against a Baseline

Suppose your model produces:

Mathematical Formulation
Copy
MAE = 120

Is that good?

Not necessarily.

Compare it against a naive model.

For example:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
naive_prediction = y_test.shift(1)

Or:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
naive_prediction = y_train.iloc[-1]

If the naive model has:

Mathematical Formulation
Copy
MAE = 100

your sophisticated model is actually worse.

A useful forecasting system should demonstrate improvement over meaningful baselines.

---

26. Walk-Forward Validation

Random cross-validation is generally inappropriate for forecasting.

Instead, we can simulate how the model would operate in the real world.

Suppose we have:

text
Ask AICopy
Train:
Jan Feb Mar Apr

Validate:
May

Then:

text
Ask AICopy
Train:
Jan Feb Mar Apr May

Validate:
Jun

Then:

text
Ask AICopy
Train:
Jan Feb Mar Apr May Jun

Validate:
Jul

This is walk-forward validation.

---

27. Expanding Window

An expanding-window strategy keeps all historical data.

text
Ask AICopy
Fold 1:
Train: [1 2 3 4]
Test: [5]

Fold 2:
Train: [1 2 3 4 5]
Test: [6]

Fold 3:
Train: [1 2 3 4 5 6]
Test: [7]

Advantages:

• uses all available historical information
• closely resembles many production systems

Disadvantage:

• older data may become less relevant after structural changes

---

28. Rolling Window

A rolling window uses only recent observations.

text
Ask AICopy
Fold 1:
Train: [1 2 3 4]
Test: [5]

Fold 2:
Train: [3 4 5 6]
Test: [7]

This can be useful when:

• the process changes over time
• recent behavior is more predictive
• old data becomes stale

The window length becomes an important hyperparameter.

---

29. TimeSeriesSplit

Scikit-learn provides TimeSeriesSplit.

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
from sklearn.model_selection import TimeSeriesSplit

tscv = TimeSeriesSplit(
 n_splits=5
)

for train_idx, test_idx in tscv.split(X):
 X_train = X.iloc[train_idx]
 X_test = X.iloc[test_idx]

 y_train = y.iloc[train_idx]
 y_test = y.iloc[test_idx]

This preserves temporal ordering.

However, you must still ensure that:

• feature engineering is leakage-safe
• preprocessing is fitted only on training data
• the test horizon matches the real business use case

---

30. Gap Between Train and Test

Sometimes information near the forecast boundary should be excluded.

For example:

text
Ask AICopy
Training data
 ↓
 GAP
 ↓
Validation data

This can be useful when:

• labels become available with delay
• features have publication delays
• there is temporal dependence that requires a separation period

Scikit-learn's TimeSeriesSplit supports a gap parameter.

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
tscv = TimeSeriesSplit(
 n_splits=5,
 gap=2
)

---

31. Multi-Step Forecasting

Many real-world problems require more than one prediction.

For example:

›Forecast next 24 hours

or:

›Forecast next 30 days

This creates a multi-step forecasting problem.

There are several strategies.

---

32. Strategy 1: Recursive Forecasting

Train a one-step model:

›t → t+1

Then use its prediction as an input for the next prediction:

text
Ask AICopy
t → t+1
 ↓
 t+2
 ↓
 t+3

Example:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
for step in range(horizon):
 prediction = model.predict(current_features)
 predictions.append(prediction)

 current_features = update_features(
 current_features,
 prediction
 )

Advantages:

• simple
• only one model required

Disadvantages:

• prediction errors accumulate
• later predictions depend on earlier predictions

---

33. Strategy 2: Direct Forecasting

Train a separate model for each horizon.

text
Ask AICopy
Model 1 → t+1
Model 2 → t+2
Model 3 → t+3
...
Model H → t+H

Advantages:

• each horizon can learn different patterns
• avoids recursive error accumulation

Disadvantages:

• more models
• more maintenance
• potentially more training cost

---

34. Strategy 3: Multi-Output Forecasting

Train one model to predict multiple future values.

text
Ask AICopy
Input:
[history]

Output:
[t+1, t+2, t+3, ..., t+H]

For example:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
model = SomeMultiOutputModel(
 output_size=horizon
)

Advantages:

• one model
• predictions can be learned jointly

Disadvantages:

• architecture can be more complex
• output horizon becomes part of the model design

---

35. Choosing a Multi-Step Strategy

The correct choice depends on:

• forecast horizon
• computational budget
• data size
• error tolerance
• operational complexity

---

36. Rolling Retraining

A production model should not necessarily be trained once and forgotten.

Suppose the model was trained in January.

By June:

• customer behavior may change
• prices may change
• competitors may change
• seasonality may shift
• data collection may change

Therefore, models may need periodic retraining.

---

37. Retraining Strategies

Fixed schedule#

Retrain:

text
Ask AICopy
daily
weekly
monthly

Data-triggered#

Retrain when:

›data distribution changes

Performance-triggered#

Retrain when:

›forecast error exceeds threshold

Hybrid#

Use a regular schedule plus monitoring-based triggers.

---

38. Example Rolling Retraining Pipeline

Conceptually:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
def retrain_model(data, cutoff):
 train_data = data[data["date"] <= cutoff]

 model = train_model(train_data)

 return model

Production loop:

text
Ask AICopy
New data arrives
 ↓
Validate data
 ↓
Update feature pipeline
 ↓
Evaluate current model
 ↓
Retrain if required
 ↓
Validate new model
 ↓
Deploy
 ↓
Monitor

---

39. Retraining Does Not Mean Automatic Promotion

A dangerous production pattern is:

text
Ask AICopy
Train model
 ↓
Immediately deploy

Instead:

text
Ask AICopy
Train candidate
 ↓
Validate
 ↓
Compare against production model
 ↓
Check business constraints
 ↓
Deploy only if acceptable

For example:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
if candidate_mae < production_mae:
 promote_candidate()
else:
 keep_production_model()

In a real system, promotion criteria should usually include more than one metric.

---

40. Production Forecasting Architecture

A simplified architecture can look like:

text
Ask AICopy
Data Sources
 ↓
Data Ingestion
 ↓
Data Validation
 ↓
Feature Engineering
 ↓
Training Pipeline
 ↓
Model Registry
 ↓
Model Validation
 ↓
Deployment
 ↓
Forecast Service
 ↓
Business Applications

Monitoring runs across the system.

text
Ask AICopy
 ┌───────────────┐
 │ Monitoring │
 └───────┬───────┘
 │
Data → Features → Model → Forecasts
 │
 Performance

---

41. Forecast Service

A forecasting API might expose:

›POST /forecast

Input:

json
Ask AICopy
{
 "series_id": "product_123",
 "horizon": 30
}

Output:

json
Ask AICopy
{
 "series_id": "product_123",
 "forecast": [
 102.4,
 104.1,
 103.8
 ]
}

A real API would typically also include:

• model version
• forecast timestamp
• horizon
• prediction intervals where available
• data freshness
• feature/model metadata

---

42. Batch vs Real-Time Forecasting

Batch forecasting#

Predictions are generated periodically.

Example:

text
Ask AICopy
Every morning at 06:00
 ↓
Generate 30-day forecasts
 ↓
Store forecasts

Useful for:

• inventory planning
• staffing
• financial planning
• demand planning

Real-time forecasting#

Predictions are generated when requested.

Useful when:

• new observations arrive frequently
• decisions require immediate forecasts

Batch forecasting is often simpler and cheaper when real-time prediction is unnecessary.

---

43. Monitoring Data Quality

Before monitoring model accuracy, monitor the input data.

Important checks include:

• missing timestamps
• duplicate records
• unexpected null values
• impossible values
• feature range changes
• frequency changes
• delayed data
• schema changes

Example:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
def validate_data(df):
 assert df["timestamp"].notna().all()
 assert df["target"].notna().all()
 assert df["timestamp"].is_monotonic_increasing

Production validation should normally produce structured validation results rather than relying only on assertions.

---

44. Data Drift

Data drift occurs when the distribution of inputs changes.

For example:

Mathematical Formulation
Copy
Training:
Average demand = 100

Production:
Average demand = 150

The model may now be operating outside its original training distribution.

Possible monitoring signals include:

• mean/variance changes
• quantile changes
• categorical distribution changes
• population stability measures
• statistical distance measures

---

45. Concept Drift

Concept drift is different.

The relationship between features and target changes.

For example:

text
Ask AICopy
Before:
Temperature ↑ → Demand ↑

After:
Temperature ↑ → Demand ↓

The input distribution might look normal, but the relationship changed.

This can be much more damaging to forecasting systems.

---

46. Forecast Error Monitoring

When actual outcomes become available:

text
Ask AICopy
Forecast
 ↓
Wait for actual
 ↓
Calculate error
 ↓
Track over time

Example:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
error = y_actual - y_forecast

mae = np.mean(
 np.abs(error)
)

Track metrics by:

• date
• product
• region
• customer segment
• forecast horizon

---

47. Horizon-Specific Error

A model can perform well for short horizons and poorly for long horizons.

Example:

This is extremely useful operational information.

Instead of reporting one overall metric, analyze performance across the forecast horizon.

---

48. Prediction Intervals

Point forecasts answer:

›What is the expected value?

Prediction intervals answer:

›How uncertain is the prediction?

For example:

›Forecast: 100
80% interval: [85, 118]

Prediction intervals are particularly valuable for:

• inventory decisions
• capacity planning
• risk management
• staffing
• financial forecasting

A forecast without uncertainty information can create false confidence.

---

49. Backtesting

Backtesting simulates historical forecasting decisions.

Example:

text
Ask AICopy
Historical data
 ↓
Choose cutoff
 ↓
Train using past
 ↓
Forecast future
 ↓
Compare with actual
 ↓
Move cutoff forward
 ↓
Repeat

This is one of the most important techniques for evaluating a forecasting system realistically.

---

50. Model Comparison Framework

Suppose we have:

text
Ask AICopy
Naive
ARIMA
SARIMA
Random Forest
XGBoost
LightGBM
LSTM
GRU
Transformer
Prophet

Do not compare them using one arbitrary split.

Instead:

text
Ask AICopy
1. Define forecast horizon
2. Define evaluation windows
3. Run walk-forward validation
4. Calculate multiple metrics
5. Analyze stability
6. Compare computational cost
7. Compare operational complexity
8. Select the best practical model

---

51. Model Selection Is a Business Decision

The most accurate model is not always the best production model.

Suppose:

A 0.3 improvement in MAE may not justify dramatically higher operational complexity.

Always consider:

• accuracy
• latency
• cost
• maintainability
• explainability
• data requirements
• reliability
• retraining complexity

---

52. Practical Model Selection Guide

This is a starting guide, not a universal rule.

---

53. A Strong Forecasting Workflow

A robust workflow looks like:

text
Ask AICopy
1. Understand the business problem
 ↓
2. Define forecast horizon
 ↓
3. Understand data availability
 ↓
4. Clean and validate time series
 ↓
5. Perform EDA
 ↓
6. Build naive baseline
 ↓
7. Build statistical baseline
 ↓
8. Build ML baseline
 ↓
9. Try deep learning if justified
 ↓
10. Try advanced architectures if justified
 ↓
11. Backtest using walk-forward validation
 ↓
12. Analyze errors
 ↓
13. Select production candidate
 ↓
14. Deploy
 ↓
15. Monitor
 ↓
16. Retrain when necessary

---

54. End-to-End Project

Project: Production Demand Forecasting System#

Imagine a company needs to forecast daily product demand for the next 30 days.

Dataset:

text
Ask AICopy
date
product_id
region
sales
price
promotion
temperature
holiday

Goal:

›Forecast sales for the next 30 days.

---

55. Step 1: Load Data

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
import pandas as pd

df = pd.read_csv(
 "demand.csv",
 parse_dates=["date"]
)

df = df.sort_values(
 ["product_id", "date"]
)

---

56. Step 2: Validate Data

Check:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
print(df.info())
print(df.isna().sum())
print(df.duplicated().sum())

Check date continuity for each product:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
for product_id, group in df.groupby("product_id"):
 dates = group["date"].sort_values()

 print(
 product_id,
 dates.min(),
 dates.max(),
 len(dates)
 )

---

57. Step 3: Create Time Features

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
df["day_of_week"] = df["date"].dt.dayofweek
df["month"] = df["date"].dt.month
df["day_of_month"] = df["date"].dt.day
df["week_of_year"] = df["date"].dt.isocalendar().week.astype(int)

Cyclical encoding:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
import numpy as np

df["dow_sin"] = np.sin(
 2 * np.pi * df["day_of_week"] / 7
)

df["dow_cos"] = np.cos(
 2 * np.pi * df["day_of_week"] / 7
)

---

58. Step 4: Create Lag Features

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
df["lag_1"] = (
 df.groupby("product_id")["sales"]
 .shift(1)
)

df["lag_7"] = (
 df.groupby("product_id")["sales"]
 .shift(7)
)

df["lag_28"] = (
 df.groupby("product_id")["sales"]
 .shift(28)
)

---

59. Step 5: Rolling Features

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
df["rolling_7"] = (
 df.groupby("product_id")["sales"]
 .transform(
 lambda x: x.shift(1).rolling(7).mean()
 )
)

df["rolling_28"] = (
 df.groupby("product_id")["sales"]
 .transform(
 lambda x: x.shift(1).rolling(28).mean()
 )
)

Notice the shift(1).

This is important.

Without it, today's target could accidentally enter today's feature.

---

60. Step 6: Chronological Split

Never randomly split this forecasting dataset.

For example:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
train = df[
 df["date"] < "2025-10-01"
]

validation = df[
 (df["date"] >= "2025-10-01") &
 (df["date"] < "2025-11-01")
]

test = df[
 df["date"] >= "2025-11-01"
]

The exact dates should be selected based on the actual business forecasting scenario.

---

61. Step 7: Build a Baseline

Seasonal naive forecasting:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
test["prediction"] = (
 test.groupby("product_id")["sales"]
 .shift(7)
)

In a real implementation, ensure the lagged value comes from information genuinely available at forecast creation time.

---

62. Step 8: Train an ML Model

Example:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
from xgboost import XGBRegressor

features = [
 "lag_1",
 "lag_7",
 "lag_28",
 "rolling_7",
 "rolling_28",
 "price",
 "promotion",
 "temperature",
 "holiday",
 "dow_sin",
 "dow_cos"
]

model = XGBRegressor(
 n_estimators=500,
 max_depth=6,
 learning_rate=0.05,
 objective="reg:squarederror"
)

model.fit(
 train[features],
 train["sales"]
)

---

63. Step 9: Evaluate

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
predictions = model.predict(
 test[features]
)

mae = mean_absolute_error(
 test["sales"],
 predictions
)

rmse = np.sqrt(
 mean_squared_error(
 test["sales"],
 predictions
 )
)

print("MAE:", mae)
print("RMSE:", rmse)

Also calculate:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
print("sMAPE:", smape(
 test["sales"],
 predictions
))

---

64. Step 10: Walk-Forward Evaluation

Instead of trusting one test period:

text
Ask AICopy
Backtest 1 → January
Backtest 2 → February
Backtest 3 → March
Backtest 4 → April
Backtest 5 → May

Calculate metrics for each period.

Example:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
results = []

for cutoff in cutoffs:

 train_data = df[
 df["date"] <= cutoff
 ]

 validation_data = df[
 df["date"] > cutoff
 ].head(30)

 model.fit(
 train_data[features],
 train_data["sales"]
 )

 pred = model.predict(
 validation_data[features]
 )

 results.append({
 "cutoff": cutoff,
 "mae": mean_absolute_error(
 validation_data["sales"],
 pred
 )
 })

results_df = pd.DataFrame(results)

The exact implementation should respect product-level horizons and feature availability.

---

65. Step 11: Compare Candidate Models

Create a comparison table:

Do not fill this table with assumed numbers.

Run the experiments and record actual results.

---

66. Step 12: Select the Production Model

A reasonable decision process:

text
Ask AICopy
Does it beat the baseline?
 ↓
 Yes
 ↓
Is performance stable across backtests?
 ↓
 Yes
 ↓
Does it meet latency/cost requirements?
 ↓
 Yes
 ↓
Can it be reliably retrained?
 ↓
 Yes
 ↓
Production candidate

---

67. Step 13: Deploy

A simplified service architecture:

text
Ask AICopy
 ┌──────────────┐
 │ Data Sources │
 └──────┬───────┘
 ↓
 ┌──────────────┐
 │ Data Pipeline│
 └──────┬───────┘
 ↓
 ┌──────────────┐
 │ Feature Store│
 └──────┬───────┘
 ↓
 ┌──────────────┐
 │ Model │
 └──────┬───────┘
 ↓
 ┌──────────────┐
 │ Forecast API │
 └──────┬───────┘
 ↓
 Business Systems

---

68. Step 14: Monitor

Monitor at least four areas.

Data#

• missing values
• freshness
• schema
• distribution

Model#

• prediction distribution
• latency
• failures
• model version

Forecast quality#

• MAE
• RMSE
• sMAPE
• horizon-specific error

Business#

• stockouts
• overstock
• revenue impact
• capacity utilization

---

69. Production Model Registry

Every production model should have identifiable metadata.

Example:

text
Ask AICopy
model_name:
demand_forecaster

version:
3.2.1

training_data_end:
2026-08-31

features:
lag_1, lag_7, lag_28, ...

forecast_horizon:
30 days

validation_mae:
8.7

status:
production

Model versioning makes debugging and rollback much easier.

---

70. Reproducibility

A forecasting pipeline should record:

• dataset version
• feature definitions
• preprocessing configuration
• model hyperparameters
• training period
• validation period
• software versions
• model artifact
• evaluation results

Without reproducibility, production failures become difficult to investigate.

---

71. Common Production Mistakes

Mistake 1: Random train-test split#

Why it fails:

›Future information can influence training.

Mistake 2: Leakage in rolling features#

Incorrect:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
df["rolling"] = df["sales"].rolling(7).mean()

Potential issue:

›Today's sales → today's feature

Better:

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
df["rolling"] = (
 df["sales"]
 .shift(1)
 .rolling(7)
 .mean()
)

---

72. Common Mistake: Optimizing Only One Split

One lucky validation period can produce a misleading conclusion.

Better:

text
Ask AICopy
Multiple historical forecast origins
 ↓
Walk-forward backtesting
 ↓
Average + variability

---

73. Common Mistake: Using the Most Complex Model

A Transformer is not automatically better than XGBoost.

An LSTM is not automatically better than SARIMA.

A useful model is one that provides sufficient accuracy while remaining:

• reliable
• affordable
• maintainable
• understandable enough for its use case

---

74. Common Mistake: Ignoring Data Availability

Suppose your feature is:

›monthly economic indicator

but the value becomes available 20 days after month-end.

If your model uses it as though it were available immediately, your backtest is unrealistic.

Always model:

›What was actually known at forecast creation time?

---

75. Common Mistake: Ignoring Forecast Horizon

A model may be excellent at:

›next hour

but poor at:

›next 30 days

Evaluate the model using the same horizon required in production.

---

76. Common Mistake: Ignoring Business Cost

Suppose:

Mathematical Formulation
Copy
Underforecast cost = $100
Overforecast cost = $10

Then symmetric error metrics may not fully represent the business objective.

The forecasting system may need cost-aware evaluation.

---

77. Advanced Evaluation

A mature forecasting system can track:

Overall metrics#

text
Ask AICopy
MAE
RMSE
sMAPE

Segment metrics#

text
Ask AICopy
Product
Region
Customer
Channel

Horizon metrics#

text
Ask AICopy
1 day
7 days
14 days
30 days

Time metrics#

text
Ask AICopy
Week
Month
Quarter
Season

This gives a much clearer picture of where the model succeeds and fails.

---

78. Forecast Bias

Error magnitude is not the only concern.

A model can systematically overforecast or underforecast.

Define:

$$Bias = \frac{1}{n} \sum_{i=1}^{n} (\hat{y}_i-y_i)$$

A consistently positive bias means the model tends to overpredict.

A consistently negative bias means it tends to underpredict.

This can have major business consequences.

---

79. Error Distribution

Do not inspect only the average.

Plot:

›Actual - Prediction

Look for:

• systematic bias
• outliers
• seasonality in errors
• changing variance
• errors around holidays
• errors during regime changes

The residual/error series can reveal problems that a single metric hides.

---

80. Forecasting Under Structural Breaks

Sometimes the underlying process changes suddenly.

Examples:

text
Ask AICopy
New competitor
Regulatory change
Pandemic
Product launch
Pricing change
Supply disruption

Historical data may no longer represent the future.

Possible responses:

• retrain more frequently
• reduce training window
• add event features
• use change-point detection
• use robust models
• manually intervene when necessary

---

81. Choosing the Training Window

More historical data is not always better.

Compare:

text
Ask AICopy
Last 30 days
Last 90 days
Last 180 days
Last 365 days
All history

Use walk-forward validation to determine which window performs best.

This is especially important when the data-generating process evolves over time.

---

82. Global vs Local Models

Suppose there are 10,000 products.

One approach:

text
Ask AICopy
Product A → Model A
Product B → Model B
...
Product 10,000 → Model 10,000

This is a local-model strategy.

Another approach:

text
Ask AICopy
All products
 ↓
One global model

Global models can share patterns across related time series.

Tree-based global models and deep-learning architectures can be especially useful in this setting.

---

83. Hierarchical Forecasting

Organizations often have hierarchies:

text
Ask AICopy
Company
 ├── Region
 │ ├── Store
 │ │ └── Product
 │
 └── Online
 └── Product

Forecasts at different levels may need to remain coherent.

For example:

Mathematical Formulation
Copy
Region forecast
=
sum of store forecasts

This introduces the concept of forecast reconciliation.

---

84. Probabilistic Forecasting

Instead of predicting:

Mathematical Formulation
Copy
sales = 100

we can predict a distribution:

›P(sales)

This can provide:

• prediction intervals
• quantiles
• uncertainty estimates
• risk-aware decisions

For example:

Mathematical Formulation
Copy
P10 = 80
P50 = 100
P90 = 130

This can be much more useful for inventory and capacity decisions.

---

85. Final Architecture

A mature forecasting platform can look like:

text
Ask AICopy
 DATA SOURCES
 │
 ↓
 ┌──────────────────┐
 │ Ingestion │
 └────────┬─────────┘
 ↓
 ┌──────────────────┐
 │ Data Validation │
 └────────┬─────────┘
 ↓
 ┌──────────────────┐
 │ Feature Pipeline │
 └────────┬─────────┘
 ↓
 ┌──────────────────┐
 │ Backtesting │
 └────────┬─────────┘
 ↓
 ┌──────────────────┐
 │ Model Training │
 └────────┬─────────┘
 ↓
 ┌──────────────────┐
 │ Model Validation│
 └────────┬─────────┘
 ↓
 ┌──────────────────┐
 │ Model Registry │
 └────────┬─────────┘
 ↓
 ┌──────────────────┐
 │ Deployment │
 └────────┬─────────┘
 ↓
 ┌──────────────────┐
 │ Forecast Service │
 └────────┬─────────┘
 ↓
 ┌──────────────────┐
 │ Business Systems │
 └──────────────────┘

Monitoring continuously observes:
- data quality
- drift
- forecast accuracy
- latency
- business impact

---

86. Exercises

Exercise 1: Transformer#

Build a Transformer forecasting model.

Requirements:

• input window = 30
• predict next 7 observations
• use at least 3 features
• compare against an LSTM

Questions:

1. Which model performs better?
2. How does sequence length affect performance?
3. Does the Transformer overfit?
4. How does training time compare?

---

Exercise 2: Prophet#

Build a Prophet model for a seasonal dataset.

Compare:

text
Ask AICopy
Prophet
Seasonal Naive
ARIMA

Evaluate using:

• MAE
• RMSE
• sMAPE

---

Exercise 3: Walk-Forward Validation#

Implement walk-forward validation manually.

Requirements:

• at least 5 forecast origins
• fixed forecast horizon
• calculate MAE for every fold
• calculate mean and standard deviation

Questions:

1. Is performance stable?
2. Which period is hardest?
3. Did model performance change over time?

---

Exercise 4: Multi-Step Forecasting#

Compare:

text
Ask AICopy
Recursive
Direct
Multi-output

for a 14-step forecast.

Measure:

• MAE
• RMSE
• computation time

---

Exercise 5: Retraining Strategy#

Simulate:

text
Ask AICopy
Daily retraining
Weekly retraining
Monthly retraining

Compare:

• accuracy
• training cost
• stability

Determine which strategy provides the best trade-off.

---

Exercise 6: Drift Detection#

Create an artificial distribution shift.

For example:

Mathematical Formulation
Copy
Training mean = 100
Production mean = 150

Measure how model performance changes.

Then experiment with:

• retraining
• shorter training windows
• additional features

---

87. Final Capstone Project

Build a complete forecasting system.

Problem#

Forecast demand for multiple products and regions.

Requirements#

Data#

Include:

• timestamp
• target
• product
• region
• price
• promotions
• holidays
• external variables

EDA#

Perform:

• trend analysis
• seasonality analysis
• missing-value analysis
• outlier analysis
• ACF/PACF analysis

Baselines#

Build:

• naive
• seasonal naive

Classical model#

Build:

• ARIMA or SARIMA

ML model#

Build:

• XGBoost or LightGBM

Deep-learning model#

Build:

• LSTM or GRU

Advanced model#

Build:

• Transformer

Alternative model#

Build:

• Prophet where appropriate

Evaluation#

Use:

• walk-forward validation
• MAE
• RMSE
• sMAPE
• horizon-specific metrics

Analysis#

Compare:

• accuracy
• stability
• training time
• inference time
• complexity
• data requirements

Production#

Design:

• training pipeline
• model registry
• forecasting API
• monitoring
• retraining strategy
• rollback strategy

---

88. Final Checklist

Before calling a forecasting system production-ready, verify:

Data#

• Timestamps are valid
• Frequency is understood
• Missing observations are handled
• Duplicate records are handled
• Data freshness is monitored
• Feature availability is realistic

Features#

• Lag features are leakage-safe
• Rolling features are leakage-safe
• Calendar features are correct
• External variables respect publication delays

Validation#

• Chronological split is used
• Walk-forward validation is used
• Forecast horizon matches production
• Baselines are included
• Multiple backtest windows are evaluated

Model#

• Candidate models are compared fairly
• Overfitting is checked
• Model complexity is justified
• Prediction uncertainty is considered when necessary

Production#

• Model versioning exists
• Data validation exists
• Monitoring exists
• Retraining strategy exists
• Rollback strategy exists
• Forecast failures are handled

Business#

• Forecast metrics align with business objectives
• Segment-level performance is understood
• Forecast bias is monitored
• Operational costs are considered

---

89. Time Series Section Complete

You have now covered the complete progression:

text
Ask AICopy
Foundations
 ↓
ARIMA / SARIMA
 ↓
Machine Learning
 ↓
RNN / LSTM / GRU
 ↓
Transformers / Prophet
 ↓
Backtesting
 ↓
Production
 ↓
Monitoring
 ↓
Retraining

The key lesson is not:

"Use the most advanced model."

The key lesson is:

"Build the simplest forecasting system that reliably solves the real problem, then add complexity only when the evidence justifies it."

A strong time-series practitioner should be comfortable moving between statistical models, machine learning, deep learning, and production engineering rather than treating one model family as universally superior.

---

90. Suggested Next Step

The Time Series section is now complete.

A natural next section in a broader Machine Learning / AI course would be one of:

1. Natural Language Processing (NLP)
2. Computer Vision
3. Recommendation Systems
4. Generative AI and Large Language Models
5. MLOps and Production Machine Learning
6. Reinforcement Learning

If this course is intended to progress toward modern AI engineering, NLP → Transformers → LLMs is a particularly natural progression.