Diagnose Constant-Target R-Squared Scores with force_finite in scikit-learn

When every evaluation target has the same value, scikit-learn’s default R-squared result can hide an important limitation. Perfect predictions produce a reported score of one, while imperfect predictions produce zero. Those convenient finite replacements do not mean that a constant-target fold provides a normal, well-defined R-squared comparison. Inspect the targets and compare the default result with force_finite=False before interpreting a suspicious score.

This tutorial is about diagnosing an evaluation subset, not changing predictions to make a report look better. A constant target can be legitimate, such as a short observation period with no variation, or it can reveal a filtering, splitting or data-loading mistake.

Why a constant target changes the calculation

For an unweighted, single-output evaluation, R-squared compares squared prediction errors with the squared deviations of the observed targets from their mean. If all observed targets are identical, that second quantity is zero. Perfect predictions therefore lead to an undefined zero-over-zero calculation; imperfect predictions lead to negative infinity.

The official r2_score reference explicitly documents these cases. Its default force_finite=True replaces the resulting NaN with one and negative infinity with zero, helping higher-level searches avoid non-finite scores. Setting the parameter to false exposes the original result. It does not repair the evaluation data.

Run a small comparison with known targets

Run the following example in a disposable Python environment with scikit-learn installed. It uses invented observations rather than a trained model, allowing you to isolate metric behaviour from model training. Expected division warnings from the diagnostic calls are suppressed locally so that the printed comparison remains readable.

import math
import warnings
from sklearn.metrics import mean_absolute_error, r2_score

cases = [
    ('constant perfect', [5, 5, 5], [5, 5, 5]),
    ('constant imperfect', [5, 5, 5], [5, 5, 6]),
    ('variable perfect', [4, 5, 6], [4, 5, 6]),
]

for name, actual, predicted in cases:
    default = r2_score(actual, predicted)
    with warnings.catch_warnings():
        warnings.simplefilter('ignore', RuntimeWarning)
        diagnostic = r2_score(actual, predicted, force_finite=False)
    mae = mean_absolute_error(actual, predicted)
    print(name, default, diagnostic, round(mae, 3))
    if name == 'constant perfect':
        assert default == 1.0 and math.isnan(diagnostic)
    elif name == 'constant imperfect':
        assert default == 0.0 and diagnostic == -math.inf
        assert math.isclose(mae, 1 / 3)
    else:
        assert default == diagnostic == 1.0

The constant-perfect case should display a diagnostic NaN. The constant-imperfect case should display negative infinity and an absolute error rounded to 0.333. The variable-perfect case should retain a score of one in both modes. The assertions check these distinctions rather than treating successful program completion as the only evidence.

Inspect the evaluation fold before blaming the estimator

1. Confirm that actual and predicted arrays refer to the same rows in the same order. An accidental merge or stale export can invalidate otherwise plausible numbers.
2. Count observations and distinct target values for each evaluation subset. R-squared is also not well-defined for fewer than two observations; the finite-replacement setting does not make a one-row evaluation meaningful.
3. Check filters, rounding and target conversion. A conversion that collapses different values can create a constant target unexpectedly.
4. If variation exists in the complete dataset but not in one time window, investigate the split while preserving the deployment question. Do not shuffle time-dependent observations simply to obtain a more attractive metric.

For forecasting evaluations, the related tutorial on inspecting TimeSeriesSplit boundaries helps you examine which observations belong in each fold. A sound boundary and a usable metric are separate checks.

Report the limitation without hiding useful errors

Include the sample count, target variation, metric settings and absolute prediction errors in the evaluation record. Mean absolute error remains interpretable in the target’s units for this example, but an acceptable error still depends on the application. A zero default R-squared here is not proof of zero predictive usefulness, and a default one on a constant-perfect fold is not evidence of general performance on variable future data.

Keep per-fold results available instead of reporting only an average. If diagnostic non-finite scores appear, describe how they were handled; silently dropping difficult folds can bias a comparison. Do not pick a new split or hyperparameter after repeatedly inspecting the final held-out test set.

Quick questions

Should I always turn force_finite off? No. Use it deliberately for diagnosis and understand the requirements of your search or reporting pipeline. Record the setting either way.

Does negative R-squared always mean constant targets? No. Nonconstant targets can produce ordinary finite negative scores when predictions are worse than the evaluation-mean reference. Inspect the actual targets rather than diagnosing from the sign alone.

This tutorial is for educational use. Test carefully before applying code in production. The site owner is not responsible for misuse, damage, data loss or production issues.