If cross-validation produces a missing score and a fit-failure warning, rerun the small diagnostic case with error_score='raise'. This exposes the fitting exception instead of assigning a numeric fallback score. Then repair the data, estimator or split responsible for the failure before reporting an average. A fold that could not fit is not simply a poorly performing model.
Understand what the fallback score means
The official cross_validate reference documents a default error_score of NaN. A numeric fallback can let evaluation continue after an estimator-fitting error, with a FitFailedWarning. Setting the option to 'raise' instead raises the error. This is especially useful while developing a new evaluation pipeline.
Do not replace a failed fit with zero just to obtain a convenient table. Zero has metric-specific meaning: it might be a legitimate result for one scorer and quite different for another. Likewise, averaging only the successful folds changes the evaluation population and can make two candidates incomparable. If every fit fails, evaluation can raise rather than return a useful score array; the fallback is not a guarantee that all errors become ordinary results.
Reproduce a one-class training fold
This deliberately small binary dataset places its two positive examples at the end. Ordinary non-shuffled KFold creates one training partition containing only class zero. LogisticRegression needs at least two training classes. Run this in a disposable Python environment with NumPy and scikit-learn installed; it does not load or change production data.
import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import KFold, cross_validate
X = np.arange(6, dtype=float).reshape(-1, 1)
y = np.array([0, 0, 0, 0, 1, 1])
cv = KFold(n_splits=3, shuffle=False)
for number, (train, test) in enumerate(cv.split(X), start=1):
labels, counts = np.unique(y[train], return_counts=True)
print(number, dict(zip(labels.tolist(), counts.tolist())))
try:
cross_validate(
LogisticRegression(), X, y,
cv=cv, scoring='accuracy', error_score='raise',
n_jobs=1,
)
except ValueError as exc:
print(type(exc).__name__, str(exc))
else:
raise AssertionError('Expected a one-class fitting failure')
The training-count summaries are {0: 2, 1: 2}, {0: 2, 1: 2} and {0: 4}. The last split explains the error before any hyperparameter tuning is relevant. The example catches the expected exception only so you can read its message. In an unattended evaluation job, normally let unexpected exceptions terminate the job rather than treating the printed message as success.
Repair the evaluation design, not the warning
For independent rows in this toy classification example, a two-fold stratified split can put both classes in each training partition. Two splits are chosen because there are only two minority-class examples. This is an illustration, not evidence that a six-row dataset supports a reliable real-world model.
from sklearn.model_selection import StratifiedKFold
fixed_cv = StratifiedKFold(n_splits=2, shuffle=True, random_state=7)
result = cross_validate(
LogisticRegression(), X, y,
cv=fixed_cv, scoring='accuracy', error_score='raise',
n_jobs=1,
)
assert result['test_score'].shape == (2,)
assert np.isfinite(result['test_score']).all()
print('Completed folds:', len(result['test_score']))
Do not substitute this split blindly for grouped customer records or chronological observations. Those constraints may require a different design even when stratification would eliminate the immediate exception. For forecasting data, first inspect TimeSeriesSplit boundaries; fixing a fitting error must not introduce future-data leakage.
Use a short diagnostic checklist
- Record the estimator, scorer, library version and split configuration.
- Inspect training class counts, missing values and input shapes per fold.
- Use one worker while tracing the first exception so the traceback is easier to follow.
- Correct the actual incompatibility and rerun the entire intended evaluation.
- Confirm the expected number of finite scores before calculating a summary.
Does a completed run prove the model is good?
No. Finite scores establish that this particular evaluation completed with numeric results; they do not establish useful generalisation, correct labels or freedom from leakage. A successful rerun still needs an appropriate baseline, a defensible split and metric interpretation. Keep the original failed-run evidence rather than quietly deleting inconvenient folds from your report.
Safety: This tutorial is for educational purposes. Test examples carefully before production use. The site owner is not responsible for misuse, damage, data loss or production issues.
