polars-learn
Skip to content
polars-learnSchemaSpec

SchemaSpec

Reference

[source]
class polars_learn.SchemaSpec(
    schema,
    *,
    allow_extra_columns=False,
    enforce_column_order=True,
    extra_columns='preserve',
    convert_pandas=False,
    warn_on_conversion=True,
    verbosity=0,
)

Describe a fixed contract for Polars DataFrames and LazyFrames.

Use this transformer when your application already defines the required column names and dtypes. Both fit and transform check that explicit contract; required dtypes must match exactly and are never cast. Output places required columns in schema-key order and handles additional columns according to the extra-column options. fit records the output layout, including preserved extras, which later transformations must match. Choose SchemaValidator instead to learn the contract from training data or permit configured dtype conversions. Neither class checks business rules such as allowed value ranges or required non-null values.

Polars is the execution engine. Native DataFrames and LazyFrames are accepted; pandas input requires convert_pandas=True and is converted behind the scenes. Output is native Polars by default, including an eager Polars frame for converted pandas input. Use set_output(transform="pandas") for compatibility output; install polars-learn[pandas] for pandas and PyArrow. Lazy output defers row execution, while eager Polars or pandas output collects during transform. Prefer native Polars input/output for speed and to avoid conversion overhead.

Parameters

schema dict of str to polars.DataType

Required input column names and dtypes in order.

allow_extra_columns bool, default=False

Whether additional, non-required columns are accepted.

enforce_column_order bool, default=True

Require the relative order of required columns to match schema key order.

extra_columns {"preserve", "drop"}, default="preserve"

If extra columns are permitted, whether to keep or drop them in output.

convert_pandas bool, default=False

Allow pandas DataFrames and convert them to Polars. Requires pandas; some dtypes also require PyArrow.

verbosity {0, 1, 2}, default=0

Print progress at 1 and schema/operation details at 2; 0 is silent.

warn_on_conversion bool, default=True

Emit PandasConversionWarning when converting pandas input.

Attributes

schema_ dict of str to polars.DataType

Copy of the required explicit schema recorded by fit.

feature_names_in_ ndarray of str, dtype=object

All training input column names in input order, including extras.

n_features_in_ int

Number of all training input columns.

output_features_ ndarray of str, dtype=object

Fitted output names, including preserved training extras.

Raises

TypeError

If schema keys/values or common options have invalid types.

ValueError

If schema is empty or not a dict, or an option has an invalid value.

See also

SchemaValidator: Learn required dtypes and allow inference-only extra names.

Notes

DataFrames return DataFrames and LazyFrames return unevaluated LazyFrames by default. Use set_output to force eager, lazy, or pandas output. Required dtypes must match exactly; there is no casting. Required output columns follow schema key order, then preserved extras follow input order. Transform requires fitting and exact fitted output names/order, including preserved extras. Extra dtypes are not validated.

Examples

>>> import polars as pl
>>> from polars_learn import SchemaSpec
>>> X = pl.DataFrame({"extra": [9], "x": [1]})
>>> contract = SchemaSpec({"x": pl.Int64}, allow_extra_columns=True, extra_columns="drop")
>>> contract.fit_transform(X.lazy()).collect().columns
['x']

Methods

Method Purpose
fit(X, y=None) Validate the explicit contract and record fitted output names.
transform(X) Check exact required dtypes and return the fitted output columns.
fit_transform(X, y=None, **fit_params) Fit and transform the same input using the configured return kind.
get_feature_names_out(input_features=None) Return the fitted output column names.
get_params(deep=True) Get estimator constructor parameters.
set_params(**params) Set constructor parameters using scikit-learn conventions.
set_output(*, transform=None) Configure native Polars output or pandas compatibility output.

fit

[source]
fit(X, y=None)

Validate the explicit contract and record fitted output names.

Call before transforming to verify the declared contract and record the output layout. Checks required names, exact dtypes, ordering, and allowed extras, then stores the explicit schema and output names. The schema comes from the constructor, not from inferred training dtypes. Preserved training extras become part of the output-name contract. No values are cast.

Parameters

X polars.DataFrame, polars.LazyFrame or pandas.DataFrame

Input with all required columns and exact required dtypes. Extra columns and ordering follow the constructor's policies. Pandas requires conversion enabled.

y object, default=None

Ignored; accepted for scikit-learn compatibility.

Returns

self SchemaSpec

Fitted contract. The provided schema is checked, not inferred anew.

Raises

TypeError

If input, option types, or schema keys/values are unsupported.

ValueError

If the schema or input contract is invalid.

Notes

Lazy input is checked through schema metadata without collecting rows. Fitted output names include preserved training extras. No casts occur.

transform

[source]
transform(X)

Check exact required dtypes and return the fitted output columns.

Call after fitting to enforce the application's fixed contract on new data. Checks required names, exact dtypes, configured order, and extra-column policies, then selects required columns in schema-key order. The output names and order must match fitting, including preserved extras. No casting occurs and the original input is never modified.

Output follows set_output: native Polars by default, eager Polars for "polars", a LazyFrame for "polars-lazy", or a pandas DataFrame for "pandas". Eager Polars and pandas output collect lazy rows. Pandas output uses Polars conversion defaults and may change dtype/null representation; missing pandas or PyArrow raises ImportError at conversion. Pandas input requires convert_pandas=True independently of the output choice.

Parameters

X polars.DataFrame, polars.LazyFrame or pandas.DataFrame

Input satisfying the explicit required schema and configured extra-column/order policies. Pandas requires conversion enabled.

Returns

X_validated polars.DataFrame, polars.LazyFrame or pandas.DataFrame

Required columns in schema order, then preserved extras in input order. Default preserves input kind; set_output can force eager, lazy, or pandas output.

Raises

sklearn.exceptions.NotFittedError

If fit has not completed successfully.

TypeError

If input or parameter types are unsupported.

ValueError

If the contract fails, configuration is invalid, or resulting output names/order differ from fitting, including preserved extra names.

Notes

No casting occurs; extra dtypes are not checked. Lazy validation uses metadata. Explicit eager output collects the result; source/collection errors are not wrapped.

fit_transform

[source]
fit_transform(
    X,
    y=None,
    **fit_params,
)

Fit and transform the same input using the configured return kind.

Convenience method for preparing training data: calls fit(X, y) and then transform(X) on the same input. This updates fitted state and returns the prepared frame. The target is ignored and extra fit parameters are unsupported. See fit for the class-specific learned contract and transform for validation and output behavior.

Output follows set_output: native Polars by default, eager Polars for "polars", a LazyFrame for "polars-lazy", or a pandas DataFrame for "pandas". Eager Polars and pandas output collect lazy rows. Pandas output uses Polars conversion defaults and may change dtype/null representation; missing pandas or PyArrow raises ImportError at conversion. Pandas input requires convert_pandas=True independently of the output choice.

Parameters

X polars.DataFrame, polars.LazyFrame or pandas.DataFrame

Input accepted by fit and transform. Pandas requires conversion enabled.

y object, default=None

Ignored; accepted for scikit-learn compatibility.

**fit_params dict

Extra fit parameters are unsupported and raise TypeError.

Returns

X_transformed polars.DataFrame, polars.LazyFrame or pandas.DataFrame

Transformed input. Default preserves input kind; set_output can force an eager DataFrame or LazyFrame. Converted pandas is eager.

See also

fit: Validate input and record fitted state. transform: Apply the fitted transformation and output configuration.

get_feature_names_out

[source]
get_feature_names_out(input_features=None)

Return the fitted output column names.

Use these names to label model inputs or inspect a fitted pipeline. Returns a copy of the names recorded during fitting; it does not transform data, collect lazy rows, or refit the transformer. Supplied input names are checked against all fitted input names in exact order. For SchemaValidator, inference-only extra columns are not included; this is the fitted contract rather than a live query's column list.

Parameters

input_features array-like of str or None, default=None

If supplied, names must exactly match feature_names_in_.

Returns

feature_names_out ndarray of shape (n_features_out_,), dtype=object

Copy of fitted output names. SchemaValidator excludes inference-only extras. SchemaSpec includes preserved training extras. A selector lists selected names, including any renames; a passthrough lists unchanged selected names. An engineer lists all output names.

Raises

sklearn.exceptions.NotFittedError

If the transformer has not been fitted.

ValueError

If supplied input names differ from fitted names or order.

get_params

[inherited API]
get_params(deep=True)

Get estimator constructor parameters.

Parameters

deep bool, default=True

Include nested estimator parameters when present.

Returns

params dict

Constructor parameters and, when requested, nested parameters. Output configuration is managed separately by set_output.

set_params

[inherited API]
set_params(**params)

Set constructor parameters using scikit-learn conventions.

Parameters

**params dict

Constructor parameter names and values. Nested estimator parameters use the step__parameter convention where supported.

Returns

self estimator instance

This estimator with updated configuration.

Raises

ValueError

If a parameter name is unknown.

Notes

Changing parameters does not immediately validate configuration or clear fitted attributes. Refit after contract changes. Invalid parameter values fail at the next relevant fit or transform operation.

set_output

[source]
set_output(*, transform=None)

Configure native Polars output or pandas compatibility output.

Call this when a downstream consumer needs a particular frame container. It only stores a choice for subsequent transform/fit_transform calls; it does not fit, transform, or collect data at configuration time. Use "default" to follow each call's input kind, "polars" for eager Polars, "polars-lazy" for a lazy plan, or "pandas" for an eager pandas DataFrame. Converted pandas input is treated as eager Polars by default. Pandas input permission remains separate: enable convert_pandas on any transformer receiving pandas, even when pandas output is requested.

Parameters

transform {"default", "polars", "polars-lazy", "pandas"} or None, default=None

Default preserves the input kind. "polars" collects an eager DataFrame; "polars-lazy" returns a LazyFrame. None leaves the configuration unchanged. "pandas" collects if necessary and converts the result to a pandas DataFrame. Requires the pandas extra.

Returns

self estimator instance

This transformer.

Raises

ValueError

If an unsupported output container is requested.

Notes

Prefer native Polars output for speed and to avoid conversion overhead. Pandas output is provided for compatibility and uses Polars to_pandas defaults, including a fresh RangeIndex and dtype/null conversion. Conversion raises ImportError if pandas or PyArrow is unavailable. Input conversion remains separately controlled by convert_pandas. Automatic scikit-learn output wrapping and global output settings are bypassed to preserve native names and permitted extra columns. Output choices survive cloning. None leaves the current choice intact; "default" resets input-kind preservation. Fit always returns self and does not collect lazy rows. Forcing lazy output from eager input may wrap an already evaluated result; use lazy input to defer evaluation.

See also

Shared estimator conventions · Production guidance · Persistence

Concepts and worked workflows.