polars-learn
Skip to content
polars-learnFeatureEngineer

FeatureEngineer

Reference

[source]
class polars_learn.FeatureEngineer(
    features,
    *,
    convert_pandas=False,
    warn_on_conversion=True,
    verbosity=0,
)

Apply ordered feature specifications to Polars DataFrames or LazyFrames.

Use this transformer to keep Polars feature logic inside a model pipeline, so training and prediction apply the same preparation to raw observations. It applies each FeatureSpec in entry order using Polars with_columns, then applies that specification's drops before starting the next. Later specifications can therefore use retained outputs from earlier stages. fit records input names/order and the resulting output schema; transform replays the definitions and checks that schema. It does not learn expression-specific training statistics: add a fitted statistical transformer for quantities that must remain fixed after training.

Polars is the execution engine. Native DataFrames and LazyFrames are accepted; pandas input requires convert_pandas=True and is converted behind the scenes. Output is native Polars by default, including an eager Polars frame for converted pandas input. Use set_output(transform="pandas") for compatibility output; install polars-learn[pandas] for pandas and PyArrow. Lazy output defers row execution, while eager Polars or pandas output collects during transform. Prefer native Polars input/output for speed and to avoid conversion overhead.

Parameters

features FeatureSpec, dict, or sequence of FeatureSpec/dict

A specification or a dict of FeatureSpec keyword arguments, optionally in an ordered sequence mixing both forms. Dicts are unpacked into FeatureSpec and validated before processing input. Single definitions are treated as one-element sequences. Specifications are applied in the supplied sequence order. Each specification adds its expressions, then drops its configured columns, before the next specification is applied. Later specifications may reference earlier outputs that remain present. An empty sequence produces an identity transformation; an empty dict is invalid. The original parameter is retained unchanged for estimator cloning.

convert_pandas bool, default=False

Allow pandas DataFrames and convert them to Polars. Requires pandas; some dtypes also require PyArrow. No conversion occurs for Polars input.

verbosity {0, 1, 2}, default=0

Print progress at 1 and schema/operation details at 2; 0 is silent.

warn_on_conversion bool, default=True

Emit PandasConversionWarning when converting pandas input.

Attributes

n_features_in_ int

Number of columns observed during fitting.

feature_names_in_ ndarray of shape (n_features_in_,), dtype=object

Input column names in training order.

input_schema_ dict of str to polars.DataType

Input dtypes observed during fitting, independent of other estimators.

output_schema_ dict of str to polars.DataType

Expected transformed names, order, and dtypes.

output_features_ ndarray of shape (n_features_out_,), dtype=object

Output column names, including retained inputs and engineered features.

Raises

TypeError

If feature definitions or common options have invalid types, a config dict has unknown keys, or a config has invalid FeatureSpec value types.

ValueError

If verbosity is outside 0, 1, and 2 or a config dict fails FeatureSpec validation, such as an empty expression sequence or an ambiguous alias.

See also

FeatureSpec: Describe one feature operation. SchemaValidator: Validate and normalise raw input before engineering.

Notes

Specifications preserve the supplied sequence order in fit and transform. Expressions within one specification are siblings: they read the columns available before that specification, not new sibling outputs. Use later specifications to reference newly created columns, or compose expressions directly from existing input columns in one specification. The public API accepts DataFrames and LazyFrames. Lazy inputs are checked using schema metadata and return unevaluated plans by default. Eager input fit resolves the complete output schema from metadata without evaluating rows. Value-dependent errors in lazy plans surface when collected. Transform requires fitted input names and order and the fitted output schema. Default output preserves input kind. set_output(transform="polars") collects a DataFrame; "polars-lazy" returns a LazyFrame, which may wrap an evaluated result for eager input. "pandas" converts an eager result for compatibility. Prefer native Polars output to avoid conversion overhead. Fit always returns self. fit_transform calls fit then transform, so feature expressions execute once during transform. Pickling retains input definitions, fitted schemas, and output settings. Expressions use Polars' native persistence, with cloudpickle for embedded Python callbacks. Version mismatches warn before state restoration, then compatible loading is attempted. Incompatible state raises SerializationError. Load trusted artifacts only and validate features after upgrading.

Examples

>>> import polars as pl
>>> from polars_learn import FeatureEngineer, FeatureSpec
>>> X = pl.DataFrame({"price": [10.0], "quantity": [2]})
>>> engineer = FeatureEngineer(
...     FeatureSpec(pl.col("price") * pl.col("quantity"), alias="notional")
... )
>>> engineer.fit_transform(X)["notional"].to_list()
[20.0]
>>> engineer.get_feature_names_out().tolist()
['price', 'quantity', 'notional']
>>> configured = FeatureEngineer(
...     {
...         "expression": pl.col("price") * pl.col("quantity"),
...         "alias": "notional",
...     }
... )
>>> configured.fit_transform(X)["notional"].to_list()
[20.0]

Methods

Method Purpose
fit(X, y=None) Validate training expressions and learn input and output schemas.
transform(X) Generate features and verify the fitted output schema.
fit_transform(X, y=None, **fit_params) Fit to training data and return its transformed representation.
get_feature_names_out(input_features=None) Return the fitted output column names.
get_params(deep=True) Get estimator constructor parameters.
set_params(**params) Set constructor parameters using scikit-learn conventions.
set_output(*, transform=None) Configure native Polars output or pandas compatibility output.

fit

[source]
fit(X, y=None)

Validate training expressions and learn input and output schemas.

Call on raw training data to establish the expected input and feature layout. Resolves the ordered feature plan from schema metadata and records input names/order and resulting output names/dtypes. It does not evaluate rows or mutate the input. Expressions are not converted into learned statistics. Returns the fitted transformer, not the engineered frame.

Parameters

X polars.DataFrame, polars.LazyFrame or pandas.DataFrame

Training data. Pandas input requires convert_pandas=True. Lazy training validates plan schemas without evaluating row values.

y object, default=None

Ignored. Present for scikit-learn estimator compatibility.

Returns

self FeatureEngineer

Fitted transformer.

Raises

TypeError

If input, feature definitions, or options have invalid types, or a config dict contains unknown keys.

ValueError

If verbosity or FeatureSpec config values are invalid.

FeatureEngineeringError

If expressions or drop operations fail. The underlying exception is preserved as the cause.

transform

[source]
transform(X)

Generate features and verify the fitted output schema.

Call after fitting to prepare new raw observations with the same feature logic. Requires the fitted input names in their original order, applies specifications in entry order, and checks output names, order, and dtypes against output_schema_. Each stage adds its expressions and then drops its configured columns; subsequent stages can use retained earlier outputs. The original input is never modified.

Output follows set_output: native Polars by default, eager Polars for "polars", a LazyFrame for "polars-lazy", or a pandas DataFrame for "pandas". Eager Polars and pandas output collect lazy rows. Pandas output uses Polars conversion defaults and may change dtype/null representation; missing pandas or PyArrow raises ImportError at conversion. Pandas input requires convert_pandas=True independently of the output choice.

Parameters

X polars.DataFrame, polars.LazyFrame or pandas.DataFrame

Input with the fitted column names and order. Pandas input requires convert_pandas=True. Input dtypes may differ only if the final output schema still matches the fitted expectations.

Returns

X_transformed polars.DataFrame, polars.LazyFrame or pandas.DataFrame

Retained and engineered columns in native Polars order. Default preserves input kind; set_output can force eager, lazy, or pandas output.

Raises

sklearn.exceptions.NotFittedError

If fit has not completed successfully.

TypeError

If input, features, or options are unsupported, config keys are unknown, or pandas conversion is disabled.

ValueError

If verbosity or FeatureSpec config values are invalid.

FeatureEngineeringError

If input columns differ, feature execution fails, or the generated output schema differs from the fitted schema.

Notes

Schema/dependency errors fail before a lazy plan is returned. Value errors in that plan occur when collected and can be native Polars errors. Explicit eager output wraps execution errors in FeatureEngineeringError, preserving their causes.

fit_transform

[source]
fit_transform(
    X,
    y=None,
    **fit_params,
)

Fit to training data and return its transformed representation.

Convenience method for preparing training data: calls fit(X, y) and then transform(X) on the same input. This updates fitted state and returns the prepared frame. The target is ignored and extra fit parameters are unsupported. See fit for the class-specific learned contract and transform for validation and output behavior. Feature expressions execute once during transform. Lazy input remains a plan unless eager output is requested.

Output follows set_output: native Polars by default, eager Polars for "polars", a LazyFrame for "polars-lazy", or a pandas DataFrame for "pandas". Eager Polars and pandas output collect lazy rows. Pandas output uses Polars conversion defaults and may change dtype/null representation; missing pandas or PyArrow raises ImportError at conversion. Pandas input requires convert_pandas=True independently of the output choice.

Parameters

X polars.DataFrame, polars.LazyFrame or pandas.DataFrame

Training data. Pandas input requires convert_pandas=True. Lazy training validates plan schemas without evaluating row values.

y object, default=None

Ignored. Present for scikit-learn estimator compatibility.

**fit_params dict

Accepted for the scikit-learn method signature. Additional fit parameters are unsupported and raise TypeError when supplied.

Returns

X_transformed polars.DataFrame, polars.LazyFrame or pandas.DataFrame

Result of fitting and transforming the same input. Default preserves input kind; set_output can force eager, lazy, or pandas output.

See also

fit: Learn the fitted schema and feature names. transform: Apply the fitted transformer and its validation policies.

get_feature_names_out

[source]
get_feature_names_out(input_features=None)

Return the fitted output column names.

Use these names to label model inputs or inspect a fitted pipeline. Returns a copy of the names recorded during fitting; it does not transform data, collect lazy rows, or refit the transformer. Supplied input names are checked against all fitted input names in exact order. For SchemaValidator, inference-only extra columns are not included; this is the fitted contract rather than a live query's column list.

Parameters

input_features array-like of str or None, default=None

If supplied, names must exactly match feature_names_in_.

Returns

feature_names_out ndarray of shape (n_features_out_,), dtype=object

Copy of fitted output names. SchemaValidator excludes inference-only extras. SchemaSpec includes preserved training extras. A selector lists selected names, including any renames; a passthrough lists unchanged selected names. An engineer lists all output names.

Raises

sklearn.exceptions.NotFittedError

If the transformer has not been fitted.

ValueError

If supplied input names differ from fitted names or order.

get_params

[inherited API]
get_params(deep=True)

Get estimator constructor parameters.

Parameters

deep bool, default=True

Include nested estimator parameters when present.

Returns

params dict

Constructor parameters and, when requested, nested parameters. Output configuration is managed separately by set_output.

set_params

[inherited API]
set_params(**params)

Set constructor parameters using scikit-learn conventions.

Parameters

**params dict

Constructor parameter names and values. Nested estimator parameters use the step__parameter convention where supported.

Returns

self estimator instance

This estimator with updated configuration.

Raises

ValueError

If a parameter name is unknown.

Notes

Changing parameters does not immediately validate configuration or clear fitted attributes. Refit after contract changes. Invalid parameter values fail at the next relevant fit or transform operation.

set_output

[source]
set_output(*, transform=None)

Configure native Polars output or pandas compatibility output.

Call this when a downstream consumer needs a particular frame container. It only stores a choice for subsequent transform/fit_transform calls; it does not fit, transform, or collect data at configuration time. Use "default" to follow each call's input kind, "polars" for eager Polars, "polars-lazy" for a lazy plan, or "pandas" for an eager pandas DataFrame. Converted pandas input is treated as eager Polars by default. Pandas input permission remains separate: enable convert_pandas on any transformer receiving pandas, even when pandas output is requested.

Parameters

transform {"default", "polars", "polars-lazy", "pandas"} or None, default=None

Default preserves the input kind. "polars" collects an eager DataFrame; "polars-lazy" returns a LazyFrame. None leaves the configuration unchanged. "pandas" collects if necessary and converts the result to a pandas DataFrame. Requires the pandas extra.

Returns

self estimator instance

This transformer.

Raises

ValueError

If an unsupported output container is requested.

Notes

Prefer native Polars output for speed and to avoid conversion overhead. Pandas output is provided for compatibility and uses Polars to_pandas defaults, including a fresh RangeIndex and dtype/null conversion. Conversion raises ImportError if pandas or PyArrow is unavailable. Input conversion remains separately controlled by convert_pandas. Automatic scikit-learn output wrapping and global output settings are bypassed to preserve native names and permitted extra columns. Output choices survive cloning. None leaves the current choice intact; "default" resets input-kind preservation. Fit always returns self and does not collect lazy rows. Forcing lazy output from eager input may wrap an already evaluated result; use lazy input to defer evaluation.

See also

Shared estimator conventions · Production guidance · Persistence

Concepts and worked workflows.