To fill missing numeric features without leaking test information, split your rows first and put SimpleImputer inside the same scikit-learn pipeline as your estimator. Fit that pipeline only on the training rows. When you call predict on held-out rows, it applies the previously learned replacement values instead of learning new ones from the test set.
This matters even when filling gaps feels like housekeeping rather than machine learning. A median calculated across the whole dataset incorporates information from the rows you intend to hold back. The official scikit-learn common pitfalls documentation explicitly identifies imputation among preprocessing operations vulnerable to leakage.
Recognise the tempting but incorrect order
A problematic workflow is to load every row, run SimpleImputer().fit_transform(X), and then create training and test subsets. The imputer has already seen the distribution of the future test rows. Moving the split lower in the script does not undo that exposure.
The safe order is split, fit training preprocessing, fit the estimator, and evaluate held-out predictions. A pipeline expresses the last three operations consistently. It also helps avoid a different mistake: filling training gaps one way and using a new median calculated from live inputs when making predictions.
Build a small numeric pipeline
Use an isolated Python environment with NumPy and scikit-learn installed. The example below uses invented measurements to demonstrate API usage, not a real service-performance dataset. Its two columns represent request volume and queue depth; the target represents response time. These few rows are not sufficient to validate a deployable model.
import numpy as np
from sklearn.impute import SimpleImputer
from sklearn.linear_model import Ridge
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
X = np.array([
[10., 1.], [12., np.nan], [14., 2.],
[16., 2.], [18., 3.], [20., np.nan],
[22., 4.], [24., 5.], [26., np.nan],
[28., 6.], [30., 7.], [32., 8.],
])
y = np.array([90., 95., 103., 108., 120., 125.,
138., 145., 150., 164., 177., 188.])
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.25, random_state=42
)
model = make_pipeline(
SimpleImputer(strategy="median"),
StandardScaler(),
Ridge(alpha=1.0),
)
model.fit(X_train, y_train)
predictions = model.predict(X_test)
print("Training medians:",
model.named_steps["simpleimputer"].statistics_)
print("Held-out predictions:", predictions)
The imputer learns its medians during model.fit. The scaler then learns from the imputed training matrix. During prediction, both reuse their fitted values. Do not call fit again on X_test and do not manually impute the test matrix before passing it to this pipeline.
Check the input contract before investigating scores
Confirm that each column is numeric, in the same order, and expressed in the same units in training and prediction. This example uses np.nan for missing values. Empty strings and a sentinel such as -999 do not automatically mean the same thing. Document any conversion rules and apply them consistently.
Look for columns that are entirely missing in the training subset. A median cannot be learned from an empty column. Investigate whether the feature should be removed or whether a deliberately chosen constant strategy is appropriate; check the installed imputer’s documented empty-feature behaviour rather than assuming the resulting feature count.
Do not silently fill missing targets with a median and call that supervised ground truth. Missing labels require a separate data-quality decision. Also ask whether a feature was genuinely available at prediction time: a final incident resolution field leaks future information regardless of how carefully its missing values are filled.
Keep preprocessing inside cross-validation too
When you later compare settings using cross-validation, pass the whole pipeline to the cross-validation or search function. Do not pre-impute the complete training matrix and then cross-validate just the final estimator. Each validation fold needs its own preprocessing learned from that fold’s training partition.
The random split above is only a demonstration. For chronological monitoring records, consider a time-aware split. For repeated records from the same host or customer, consider an appropriate grouped split. A pipeline prevents one class of leakage; it does not make a poorly chosen split representative of deployment.
Quick troubleshooting and safe rollback
- Conversion errors: inspect column types and missing-value markers before changing model parameters.
- Different feature counts: compare input schema and investigate all-missing training columns.
- Suspiciously strong validation: audit when every feature becomes available and how related records cross the split.
- Production drift: monitor missing-input rates and prediction quality; do not automatically refit on every incoming batch.
Keep the existing production model unchanged while testing this approach offline. If an experiment fails, return to the previous approved training script and dependency environment rather than overwriting the deployed model. For wider operational preparation, see our AI security and backup basics.
Educational-use notice: Test carefully before using these steps in production. The site owner is not responsible for misuse, damage, data loss or production issues. Do not place confidential operational records into an unapproved training environment.
