ConformalRegressor¶

Reference¶

class conformity.ConformalRegressor(
    estimator,
    *,
    auto_calibrate=True,
    tts_kwargs=None,
    interval_method='symmetric',
)

Split-conformal prediction intervals for single-output regression.

Parameters¶

estimator BaseEstimator: Cloneable sklearn regressor or preprocessing-and-regression pipeline.
auto_calibrate bool, default=True: Reserve held-out calibration data during fit; defaults to True.
tts_kwargs TrainTestSplitKwargs or None, default=None: train_test_split options for automatic calibration. By default, test_size=0.2, shuffle=True and stratify=None.
interval_method {"symmetric", "asymmetric"}, default='symmetric': Symmetric absolute-residual calibration (default), or separate signed residual quantiles with alpha / 2 allocated to each tail.

Attributes¶

lower_q_level_ float: Most recent lower correction for asymmetric intervals.
upper_q_level_ float: Most recent upper correction for asymmetric intervals.

Attributes¶

estimator_ BaseEstimator: Fitted clone; created by fit and removed by parameter changes.
n_features_in_ int: Number of validated fitting features.
feature_names_in_ ndarray: Input names, present when fitting named string columns.
classes_ ndarray: Classifier labels in probability and membership-column order.
calibration_scores_ ndarray: Immutable held-out scores; created by successful calibration.
sorted_calibration_scores_ ndarray: Immutable sorted scores cached for rank queries.
n_calibration_ int: Number of held-out calibration observations.
alpha_used_ float: Miscoverage probability from the most recent threshold query.
quantile_level_ float: Finite-sample rank divided by calibration count; may exceed one.
q_level_ float: Most recent threshold, including infinity when the rank exceeds n.
is_calibrated_ bool: Whether held-out scores exist. Refitting clears calibration state.

Examples¶

>>> from sklearn.datasets import make_regression
>>> from sklearn.linear_model import Ridge
>>> from sklearn.model_selection import train_test_split
>>> from sklearn.pipeline import Pipeline
>>> from sklearn.preprocessing import StandardScaler
>>> from conformity import ConformalRegressor
>>> X, y = make_regression(
...     n_samples=120, n_features=3, noise=4, random_state=0
... )
>>> X_dev, X_test, y_dev, y_test = train_test_split(
...     X, y, test_size=0.25, random_state=1
... )
>>> X_train, X_calib, y_train, y_calib = train_test_split(
...     X_dev, y_dev, test_size=1 / 3, random_state=2
... )
>>> model = ConformalRegressor(
...     Pipeline([("scaler", StandardScaler()), ("ridge", Ridge())]),
...     auto_calibrate=False,
... )
>>> _ = model.fit(X_train, y_train)
>>> _ = model.calibrate(X_calib, y_calib)
>>> points, intervals = model.predict(X_test, alpha=0.1, return_interval=True)
>>> points.shape, intervals.shape
((30,), (30, 2))
>>> model.n_calibration_
30

Methods¶

Method	Purpose
calibrate	Replace held-out scores and return this predictor for method chaining.
evaluate	Return point error, coverage, tail misses, width and interval score.
fit	Fit a fresh clone; optionally reserve an independent calibration split.
get_metadata_routing	Get metadata routing of this object.
get_params	Get parameters for this estimator.
predict	Return points or (points, intervals).
predict_interval	Return [lower, upper] bounds with shape (n_samples, 2).
predict_point	Return point predictions without requiring calibration.
score	Return point-prediction R², using point predictions.
set_fit_request	Request metadata passed to the `fit` method.
set_params	Set sklearn parameters and invalidate fitted and calibrated state.
set_predict_request	Request metadata passed to the `predict` method.
set_score_request	Request metadata passed to the `score` method.
### calibrate

[source]

calibrate(X, y)

Replace held-out scores and return this predictor for method chaining.

Parameters¶

X {array-like, sparse matrix, DataFrame} of shape (n_samples, n_features): Feature matrix of shape (n_samples, n_features); dense, sparse, or DataFrame.
y array-like of shape (n_samples,): One target per sample; classification labels follow the fitted classes.

Returns¶

self object: Predictor with replacement calibration scores.

Raises¶

ValueError: If input shapes, values, weights or configuration are invalid.
NotFittedError: If fit has not established a fitted estimator.

evaluate¶

[source]

evaluate(
    X,
    y,
    alpha=0.05,
    *,
    sample_weight=None,
)

Return point error, coverage, tail misses, width and interval score.

Parameters¶

X {array-like, sparse matrix, DataFrame} of shape (n_samples, n_features): Feature matrix of shape (n_samples, n_features); dense, sparse, or DataFrame.
y array-like of shape (n_samples,): One target per sample; classification labels follow the fitted classes.
alpha float, default=0.05: Miscoverage probability strictly between zero and one.
sample_weight array-like of shape (n_samples,) or None, default=None: Optional finite, nonnegative weights; fitting or evaluation only, never calibration.

Returns¶

report dict of str to float: Keys are r2, mae, mse, coverage, lower_miss_rate, upper_miss_rate, mean_width, normalized_mean_width and interval_score. Normalized width divides mean width by the unweighted test-target IQR and is infinite when that IQR is zero. Tail miss rates are empirical diagnostics; for asymmetric intervals compare each with alpha / 2, allowing for finite test-sample variation.

Raises¶

ValueError: If input shapes, values, weights or configuration are invalid.
NotFittedError: If fit has not established a fitted estimator.
RuntimeError: If conformal output is requested before calibration.

fit¶

[source]

fit(
    X,
    y,
    auto_calibrate=None,
    tts_kwargs=None,
    *,
    sample_weight=None,
    **fit_params,
)

Fit a fresh clone; optionally reserve an independent calibration split.

Parameters¶

X {array-like, sparse matrix, DataFrame} of shape (n_samples, n_features): Feature matrix of shape (n_samples, n_features); dense, sparse, or DataFrame.
y array-like of shape (n_samples,): One target per sample; classification labels follow the fitted classes.
auto_calibrate bool or None, default=None: Reserve an independent calibration split when fitting.
tts_kwargs TrainTestSplitKwargs or None, default=None: train_test_split options for this fit; requires automatic calibration. If provided, this mapping replaces the constructor mapping. Missing options use defaults: test_size=0.2, train_size=None, random_state=None, shuffle=True and stratify=None. Set random_state here for reproducible automatic splits.
sample_weight array-like of shape (n_samples,) or None, default=None: Optional finite, nonnegative weights; fitting or evaluation only, never calibration.
**fit_params dict: Estimator-specific fit metadata forwarded to the cloned estimator.

Returns¶

self object: Fitted predictor; calibrated when automatic calibration is enabled.

Raises¶

TypeError: If the estimator cannot be cloned or lacks the required prediction methods.
ValueError: If input shapes, values, weights or configuration are invalid.

get_metadata_routing¶

[inherited API]

get_metadata_routing()

Get metadata routing of this object.

Returns¶

routing MetadataRequest: A :class:`~sklearn.utils.metadata_routing.MetadataRequest` encapsulating routing information.

get_params¶

[inherited API]

get_params(deep=True)

Get parameters for this estimator.

Parameters¶

deep bool, default=True, default=True: If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns¶

params dict: Parameter names mapped to their values.

predict¶

[source]

predict(
    X,
    alpha=0.05,
    *,
    return_interval=False,
)

Return points or (points, intervals).

Parameters¶

X {array-like, sparse matrix, DataFrame} of shape (n_samples, n_features): Feature matrix of shape (n_samples, n_features); dense, sparse, or DataFrame.
alpha float, default=0.05: Miscoverage probability strictly between zero and one.
return_interval bool, default=False: Request conformal output alongside ordinary predictions.

Returns¶

predictions ndarray or tuple of ndarray: Point predictions, or (points, intervals) when return_interval=True.

Raises¶

ValueError: If input shapes, values, weights or configuration are invalid.
NotFittedError: If fit has not established a fitted estimator.
RuntimeError: If conformal output is requested before calibration.

predict_interval¶

[source]

predict_interval(X, alpha=0.05)

Return [lower, upper] bounds with shape (n_samples, 2).

Parameters¶

X {array-like, sparse matrix, DataFrame} of shape (n_samples, n_features): Feature matrix of shape (n_samples, n_features); dense, sparse, or DataFrame.
alpha float, default=0.05: Miscoverage probability strictly between zero and one.

Returns¶

intervals ndarray of shape (n_samples, 2): Closed lower and upper bounds; may be outward infinite.

Raises¶

ValueError: If input shapes, values, weights or configuration are invalid.
NotFittedError: If fit has not established a fitted estimator.
RuntimeError: If conformal output is requested before calibration.

predict_point¶

[source]

predict_point(X)

Return point predictions without requiring calibration.

Parameters¶

X {array-like, sparse matrix, DataFrame} of shape (n_samples, n_features): Feature matrix of shape (n_samples, n_features); dense, sparse, or DataFrame.

Returns¶

predictions ndarray of shape (n_samples,): Point predictions from the fitted estimator.

Raises¶

ValueError: If input shapes, values, weights or configuration are invalid.
NotFittedError: If fit has not established a fitted estimator.

score¶

[source]

score(
    X,
    y,
    sample_weight=None,
)

Return point-prediction R², using point predictions.

Parameters¶

X {array-like, sparse matrix, DataFrame} of shape (n_samples, n_features): Feature matrix of shape (n_samples, n_features); dense, sparse, or DataFrame.
y array-like of shape (n_samples,): One target per sample; classification labels follow the fitted classes.
sample_weight array-like of shape (n_samples,) or None, default=None: Optional finite, nonnegative weights; fitting or evaluation only, never calibration.

Returns¶

score float: Point-prediction R² for regression or accuracy for classification.

Raises¶

ValueError: If input shapes, values, weights or configuration are invalid.
NotFittedError: If fit has not established a fitted estimator.

set_fit_request¶

[inherited API]

set_fit_request(
    *,
    auto_calibrate='$UNCHANGED$',
    sample_weight='$UNCHANGED$',
    tts_kwargs='$UNCHANGED$',
)

Request metadata passed to the fit method.

Parameters¶

auto_calibrate str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED, default='$UNCHANGED$': Metadata routing for auto_calibrate parameter in fit.
sample_weight str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED, default='$UNCHANGED$': Metadata routing for sample_weight parameter in fit.
tts_kwargs str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED, default='$UNCHANGED$': Metadata routing for tts_kwargs parameter in fit.

Returns¶

self object: The updated object.

set_params¶

[source]

set_params(**params)

Set sklearn parameters and invalidate fitted and calibrated state.

Parameters¶

**params dict: Sklearn parameters, including nested estimator parameters.

Returns¶

self object: Predictor with updated parameters and invalidated fitted state.

Raises¶

ValueError: If input shapes, values, weights or configuration are invalid.

set_predict_request¶

[inherited API]

set_predict_request(
    *,
    alpha='$UNCHANGED$',
    return_interval='$UNCHANGED$',
)

Request metadata passed to the predict method.

Parameters¶

alpha str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED, default='$UNCHANGED$': Metadata routing for alpha parameter in predict.
return_interval str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED, default='$UNCHANGED$': Metadata routing for return_interval parameter in predict.

Returns¶

self object: The updated object.

set_score_request¶

[inherited API]

set_score_request(*, sample_weight='$UNCHANGED$')

Request metadata passed to the score method.

Parameters¶

sample_weight str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED, default='$UNCHANGED$': Metadata routing for sample_weight parameter in score.

Returns¶

self object: The updated object.

is_calibrated_¶

Report whether calibration scores are available.

Keys	Action
`?`	Open this help
`n`	Next page
`p`	Previous page
`s`	Search