1.4 How AutoML Facilitates Model and Feature Selection

1.4 How AutoML Facilitates Model and Feature Selection

Databricks AutoML answers two questions that otherwise consume the first week of any tabular project: which algorithm family fits this data, and which columns actually carry signal. It answers both empirically — by training and ranking real candidate models — rather than by heuristic.

---

Supported Problem Types and Algorithms

Databricks AutoML supports three fundamental supervised learning paradigms:

1. Classification (Binary and Multi-Class):

   • Objective: Predict discrete categorical labels (e.g., customer churn: 0 or 1, fraud classification, support ticket categorization).
   • Candidate Model Families: LightGBM, XGBoost, Random Forest, Decision Tree, Logistic Regression.
   • Primary Metric: f1 is the default; log_loss, precision, accuracy, and roc_auc can be selected with primary_metric.

2. Regression (Continuous Numeric Prediction):

   • Objective: Predict real-valued continuous quantities (e.g., housing prices, lifetime customer value, transaction volume).
   • Candidate Model Families: LightGBM Regressor, XGBoost Regressor, Random Forest Regressor, Decision Tree Regressor, Linear Regression (Ridge / Lasso / ElasticNet).
   • Primary Metric: r2 is the default; rmse, mae, and mse can be selected with primary_metric.

3. Forecasting (Time-Series Extrapolation):

   • Objective: Predict future sequential values across single or multiple time-series entities (e.g., store-item retail sales forecasting).
   • Candidate Model Families: Facebook Prophet, AutoARIMA.
   • Primary Metric: smape is the default; mse, rmse, mae, and mdape can be selected with primary_metric.

---

Automated Data Preprocessing & Validation Pipeline

Before initiating model training, Databricks AutoML performs automated sanity checks and feature transformations on the input dataset:

Exam Trap Alert: Databricks AutoML does not perform heavy feature engineering such as domain-specific ratios, polynomial interaction features, or window aggregations. It focuses strictly on standard cleaning, imputation, and encoding. Custom domain features should be engineered in Unity Catalog Feature Tables prior to feeding data into AutoML.

---

How AutoML Narrows the Feature Set

AutoML does not run a formal feature-selection algorithm such as recursive feature elimination. It reduces the feature space through a sequence of concrete, inspectable decisions:

1. Semantic type detection. Each column is classified as numeric, categorical, datetime, or text. Columns AutoML cannot represent — images, nested arrays, binary blobs — are dropped and reported in the data exploration notebook.
2. Uninformative column removal. Constant columns and columns that are null in effectively every row carry no signal and are excluded before training.
3. High-cardinality truncation. A categorical column with thousands of levels would explode the one-hot matrix, so AutoML retains the most frequent levels and consolidates the tail rather than encoding every value.
4. Explicit exclusion. exclude_cols is how you remove leakage: primary keys, post-outcome timestamps, or any column computed after the label was known. This is the parameter exam scenarios reach for when a model scores suspiciously well.
5. Post-hoc importance, on request. Each generated trial notebook contains SHAP code, but it is gated behind shap_enabled = False, because the computation is memory-intensive. Setting shap_enabled = True in the notebook and re-running the feature-importance cell produces the summary plot that tells you which features the model actually relied on — the practical feature-selection output of an AutoML run, and one you have to opt into.

Bringing curated features in

feature_store_lookups lets AutoML join Unity Catalog feature tables into the training set by lookup key, with an optional timestamp key for point-in-time correctness. That keeps engineered features governed and reusable instead of recomputed inside a one-off notebook.

Exam trap: AutoML performs cleaning and encoding, not feature creation. It will impute, encode, and split; it will not build a 30-day rolling spend ratio or a customer-tenure interaction term. Domain features belong in a feature table before AutoML runs — see Section 1.7.

Reading the leaderboard as a feature-selection tool

AutoML's output is more than a winning model, and two artefacts drive feature decisions.

The data exploration notebook is generated automatically and imported into the workspace alongside the best trial notebook. It profiles the input data and raises warnings for high cardinality, strong correlations, and null-heavy columns, which is usually the fastest read on which columns are worth keeping.

Feature importance comes from the trial notebooks, and it is opt-in rather than automatic: the SHAP cell ships with shap_enabled = False and has to be switched on and re-run. Doing that for two or three notebooks near the top of the leaderboard is worth the compute, because agreement across different algorithms is stronger evidence than any single model's ranking — when a linear model and a tree ensemble independently put the same three columns first, that is the justification for pruning a wide feature set before a hand-built pipeline is written. Note that only the data exploration notebook and the best trial notebook land in the workspace automatically; the remaining trial notebooks are stored as MLflow artifacts and are imported on demand from the experiment UI or with databricks.automl.import_notebook.