Before trusting a forecasting score, print the training and test boundaries. A random split can let a model learn from observations later than the period it is being evaluated on. For an ordered sequence, scikit-learn’s TimeSeriesSplit provides successive earlier training windows and later test windows. Inspecting its indices is a simple first check that your evaluation resembles the order in which predictions would actually be made.
This tutorial uses synthetic row positions, not customer records or measured forecasting results. It is for educational purposes: test carefully before production. The site owner accepts no responsibility for misuse, damage, data loss or production issues.
Check that rows represent the timeline you intend
The official TimeSeriesSplit reference explains that ordinary cross-validation can train on future data and evaluate on past data. It also states that equally spaced samples are needed for comparable fold metrics. The splitter works with row positions; it does not inspect or repair your timestamps.
Sort the dataset chronologically and check for duplicate timestamps, missing intervals and mismatched feature and target rows. If your data are hourly, two rows normally represent two hours only when that spacing is actually present. Do not silently fill missing target values just to create regular spacing. Decide how missing observations should be handled for the real prediction task.
Run a small boundary inspection
Use an isolated Python environment with scikit-learn installed. Save the following as a new scratch script and run it there. It only prints synthetic indices and does not read or modify production files.
from sklearn.model_selection import TimeSeriesSplit
rows = list(range(12))
splitter = TimeSeriesSplit(n_splits=3, test_size=2, gap=2)
for fold, (train, test) in enumerate(splitter.split(rows), start=1):
excluded = list(range(int(train[-1]) + 1, int(test[0])))
assert train[-1] < test[0]
print(f'Fold {fold}')
print('train:', train.tolist())
print('gap:', excluded)
print('test:', test.tolist())Read the first fold before fitting any estimator. Its training indices should be 0 through 3, the excluded gap indices 4 and 5, and the test indices 6 and 7. Later folds advance the test window and expand the training window. This behaviour matches the documented example with twelve samples, three splits, a two-sample test size and a two-sample gap.
Understand what each parameter does
n_splits controls the number of evaluation folds. test_size sets each test window’s size in samples. gap excludes samples at the end of the training set before the test window. It is not an automatic rule about business days or publication delays.
By default, successive training sets accumulate earlier observations. If you need a bounded historical window, investigate max_train_size and inspect the boundaries again. A rolling window and an expanding window answer different questions about how much history a deployed system can use.
Do not choose the gap merely because a larger number sounds safer. Describe when the target becomes available and what each feature contains. Then select a separation consistent with that information timing. A two-row gap in this demonstration is illustrative, not a recommended setting for every forecast.
Keep preprocessing and feature creation honest
A chronological split is necessary for many time-based tasks, but it is not sufficient to prevent every form of leakage. A feature computed using future observations remains problematic even when the row indices are correctly separated. Check rolling calculations, aggregate tables and target-derived columns against the actual prediction time.
Fit learned preprocessing on each fold’s training rows rather than on the whole dataset before splitting. For missing-value preprocessing, see using SimpleImputer inside a pipeline without test-data leakage. The pipeline helps keep learned transformations within the training process; it cannot correct a source column that already contains future information.
Troubleshooting checklist and FAQ
- If the splitter rejects your settings, check whether there are enough rows for the requested test windows, gap and initial training set. Reduce the demonstration sizes only if that still represents a useful evaluation.
- If equal row counts span different calendar durations, inspect missing timestamps and irregular sampling before comparing scores.
- If scores look unexpectedly strong, check feature availability and target construction rather than assuming the splitter guarantees realism.
- Record boundary dates, row counts and settings alongside later model metrics so that a reviewer can reconstruct the experiment.
Does TimeSeriesSplit sort my data? No. Supply the rows in the intended time order.
Does the gap remove those rows forever? No. It separates training from testing in that fold; later expanding windows may include previously excluded rows.
Is this a forecasting model? No. It generates indices for evaluation. Validate those boundaries first, then add an estimator and a metric appropriate to the prediction task.
