API reference¶

Public imports are available from conformity. Signatures, parameter defaults, fitted attributes and method contracts below are generated from the implementation.

For interpretation and evaluation choices, read diagnostics. Set metrics use boolean membership in explicit fitted class order; interval metrics use closed lower/upper bounds. Calibration is unweighted even when evaluation uses weights. See calibration assumptions and the examples gallery for complete workflows.

Classes¶

Class	Reference
`BaseConformalPredictor`	BaseConformalPredictor
`ConformalRegressor`	ConformalRegressor
`ConformalClassifier`	ConformalClassifier

Functions¶

prediction_set_coverage¶

[source]

conformity.prediction_set_coverage(
    y_true,
    prediction_set,
    *,
    classes=None,
    sample_weight=None,
)

Fraction of sets containing the true class; supports arbitrary labels.

Parameters¶

y_true: One observed target or class label per sample.
prediction_set: Boolean membership matrix in classes order.
classes default=None: Unique class labels in boolean membership-column order.
sample_weight default=None: Optional finite, nonnegative weights; fitting or evaluation only, never calibration.

Returns¶

result float: Fraction of samples whose set contains the true label; weighted when requested.

Examples¶

>>> from conformity import prediction_set_coverage
>>> sets = [[True, False], [False, True], [True, True]]
>>> prediction_set_coverage(["billing", "account", "billing"], sets,
...                         classes=["billing", "account"])
1.0

Raises¶

ValueError: If input shapes, values, weights or configuration are invalid.

prediction_set_coverage_by_class¶

[source]

conformity.prediction_set_coverage_by_class(
    y_true,
    prediction_set,
    *,
    classes,
    sample_weight=None,
)

Return empirical set coverage for each represented class.

Parameters¶

y_true: One observed class label per sample.
prediction_set: Boolean membership matrix in classes order.
classes: Unique class labels in boolean membership-column order.
sample_weight default=None: Optional finite, nonnegative weights for evaluation only.

Returns¶

result dict: Empirical coverage keyed by class label.

Examples¶

>>> from conformity import prediction_set_coverage_by_class
>>> sets = [[True, False], [False, True], [True, False]]
>>> prediction_set_coverage_by_class(
...     ["a", "b", "b"], sets, classes=["a", "b"]
... )
{'a': 1.0, 'b': 0.5}

Raises¶

ValueError: If shapes, labels, weights or class representation are invalid.

prediction_interval_coverage¶

[source]

conformity.prediction_interval_coverage(
    y_true,
    prediction_intervals,
    *,
    sample_weight=None,
)

Fraction of targets contained in closed prediction intervals.

Parameters¶

y_true: One observed target or class label per sample.
prediction_intervals: Ordered lower/upper bounds of shape (n_samples, 2).
sample_weight default=None: Optional finite, nonnegative weights; fitting or evaluation only, never calibration.

Returns¶

result float: Fraction of targets inside their closed interval; weighted when requested.

Examples¶

>>> from conformity import prediction_interval_coverage
>>> prediction_interval_coverage([1.0, 4.0], [[0.0, 2.0], [1.0, 3.0]])
0.5

Raises¶

ValueError: If input shapes, values, weights or configuration are invalid.

prediction_interval_lower_miss_rate¶

[source]

conformity.prediction_interval_lower_miss_rate(
    y_true,
    prediction_intervals,
    *,
    sample_weight=None,
)

Fraction of targets below their closed prediction intervals.

Parameters¶

y_true: One finite numeric target per sample.
prediction_intervals: Ordered lower/upper bounds of shape (n_samples, 2).
sample_weight default=None: Optional finite, nonnegative weights for evaluation only.

Returns¶

result float: Fraction of targets strictly below the lower bound.

Examples¶

>>> from conformity import prediction_interval_lower_miss_rate
>>> prediction_interval_lower_miss_rate([0, 2], [[1, 3], [1, 3]])
0.5

Raises¶

ValueError: If input shapes, values or weights are invalid.

prediction_interval_upper_miss_rate¶

[source]

conformity.prediction_interval_upper_miss_rate(
    y_true,
    prediction_intervals,
    *,
    sample_weight=None,
)

Fraction of targets above their closed prediction intervals.

Parameters¶

y_true: One finite numeric target per sample.
prediction_intervals: Ordered lower/upper bounds of shape (n_samples, 2).
sample_weight default=None: Optional finite, nonnegative weights for evaluation only.

Returns¶

result float: Fraction of targets strictly above the upper bound.

Examples¶

>>> from conformity import prediction_interval_upper_miss_rate
>>> prediction_interval_upper_miss_rate([0, 4], [[1, 3], [1, 3]])
0.5

Raises¶

ValueError: If input shapes, values or weights are invalid.

prediction_interval_width¶

[source]

conformity.prediction_interval_width(
    prediction_intervals,
    *,
    sample_weight=None,
)

Mean interval width; unbounded intervals have infinite width.

Parameters¶

prediction_intervals: Ordered lower/upper bounds of shape (n_samples, 2).
sample_weight default=None: Optional finite, nonnegative weights; fitting or evaluation only, never calibration.

Returns¶

result float: Mean upper-minus-lower bound width; may be infinite.

Examples¶

>>> from conformity import prediction_interval_width
>>> prediction_interval_width([[0.0, 2.0], [1.0, 3.0]])
2.0

Raises¶

ValueError: If input shapes, values, weights or configuration are invalid.

normalized_prediction_interval_width¶

[source]

conformity.normalized_prediction_interval_width(
    y_true,
    prediction_intervals,
    *,
    sample_weight=None,
)

Mean interval width divided by the target's interquartile range.

Parameters¶

y_true: One finite numeric target per sample; its unweighted IQR is the scale.
prediction_intervals: Ordered lower/upper bounds of shape (n_samples, 2).
sample_weight default=None: Optional finite, nonnegative weights for averaging interval widths.

Returns¶

result float: Mean width divided by the unweighted target IQR; may be infinite.

Examples¶

>>> from conformity import normalized_prediction_interval_width
>>> normalized_prediction_interval_width(
...     [0, 0, 2, 2], [[0, 2], [0, 2], [0, 2], [0, 2]]
... )
1.0

Raises¶

ValueError: If input shapes, values or weights are invalid.

interval_score¶

[source]

conformity.interval_score(
    y_true,
    prediction_intervals,
    alpha=0.05,
    *,
    sample_weight=None,
)

Mean Winkler score: width plus 2/alpha times each missed-bound distance.

Parameters¶

y_true: One observed target or class label per sample.
prediction_intervals: Ordered lower/upper bounds of shape (n_samples, 2).
alpha default=0.05: Miscoverage probability strictly between zero and one.
sample_weight default=None: Optional finite, nonnegative weights; fitting or evaluation only, never calibration.

Returns¶

result float: Mean interval score; may be infinite. Smaller values indicate a better score.

Examples¶

>>> from conformity import interval_score
>>> interval_score([1.0, 4.0], [[0.0, 2.0], [1.0, 3.0]], alpha=0.1)
12.0

Raises¶

ValueError: If input shapes, values, weights or configuration are invalid.

prediction_set_size¶

[source]

conformity.prediction_set_size(
    prediction_set,
    *,
    sample_weight=None,
)

Mean cardinality (smaller is more efficient at comparable coverage).

Parameters¶

prediction_set: Boolean membership matrix in classes order.
sample_weight default=None: Optional finite, nonnegative weights; fitting or evaluation only, never calibration.

Returns¶

result float: Mean number of included labels per sample; weighted when requested.

Examples¶

>>> from conformity import prediction_set_size
>>> prediction_set_size([[True, False], [False, False], [True, True]])
1.0

Raises¶

ValueError: If input shapes, values, weights or configuration are invalid.

prediction_set_empty_rate¶

[source]

conformity.prediction_set_empty_rate(
    prediction_set,
    *,
    sample_weight=None,
)

Fraction of empty sets.

Parameters¶

prediction_set: Boolean membership matrix in classes order.
sample_weight default=None: Optional finite, nonnegative weights; fitting or evaluation only, never calibration.

Returns¶

result float: Fraction of samples with no included labels; weighted when requested.

Examples¶

>>> from conformity import prediction_set_empty_rate
>>> prediction_set_empty_rate([[True, False], [False, False]])
0.5

Raises¶

ValueError: If input shapes, values, weights or configuration are invalid.

prediction_set_singleton_rate¶

[source]

conformity.prediction_set_singleton_rate(
    prediction_set,
    *,
    sample_weight=None,
)

Fraction of sets containing exactly one label.

Parameters¶

prediction_set: Boolean membership matrix in classes order.
sample_weight default=None: Optional finite, nonnegative weights; fitting or evaluation only, never calibration.

Returns¶

result float: Fraction of samples with exactly one included label; weighted when requested.

Examples¶

>>> from conformity import prediction_set_singleton_rate
>>> prediction_set_singleton_rate([[True, False], [True, True]])
0.5

Raises¶

ValueError: If input shapes, values, weights or configuration are invalid.

Keys	Action
`?`	Open this help
`n`	Next page
`p`	Previous page
`s`	Search