Capabilities

Evaluation & monitoring

Trust a forecaster by testing it honestly and watching it over time. APDTFlow backtests across history and tracks accuracy and interval coverage against its calibration baseline.

← All features

Evaluation & monitoring

Score honestly, then watch for drift in production.

Rolling-origin backtesting

Simulate production forecasting over history — the realistic way to score a model.

historical_forecasts() replays the model across your history with a moving origin, so the score reflects how it would have done in production rather than on one lucky split. Tune where it starts, how often it forecasts (stride), and whether it retrains each fold.

model.historical_forecasts(df, start=0.8, stride=7)

python

bt = model.historical_forecasts(df, target_col='y', start=0.8,
                                stride=7, metrics=['mae', 'mase', 'smape'])

see example: backtesting_demo.py →

A full metric suite

MAE, RMSE, MAPE, R², MASE, sMAPE, CRPS and interval coverage, out of the box.

Reach for the metric that fits the job — scale-free MASE to compare across series, sMAPE/MAPE for percentage error, CRPS and coverage to judge interval quality — all available through score() and the backtest results.

model.score(test_df, metric='mase')

Drift & coverage monitoring

Compare recent accuracy and interval coverage against the calibration baseline.

Once deployed, score_recent() checks fresh data against the calibration baseline. A rising error ratio, or coverage slipping under target, is your cue to recalibrate or refit before forecasts go stale.

model.score_recent(recent_df)

python

s = model.score_recent(recent_df, target_col='y')
s['mae_ratio']     # recent error vs the calibration baseline
s['coverage']      # vs expected coverage — recalibrate if it drops

← Previous Inputs & features Next → Production & architectures