2.1 Choosing, Training & Predicting with BigQuery ML Models
Key Takeaways
- BigQuery ML trains and runs models with SQL inside BigQuery, so data does not move to a separate training environment.
- For forecasting, BigQuery ML offers ARIMA_PLUS for one variable, ARIMA_PLUS_XREG with covariates, and AI.FORECAST with the pre-trained TimesFM model.
- ML.PREDICT uses a default binary classification threshold of 0.5, which you can change with the THRESHOLD argument.
- ML.DETECT_ANOMALIES works with ARIMA_PLUS and ARIMA_PLUS_XREG for time series and with k-means, PCA, and autoencoder models for independent data.
- Setting MODEL_REGISTRY = 'VERTEX_AI' registers a BigQuery ML model in Agent Platform Model Registry for deployment to an online endpoint without exporting it.
BigQuery ML (BQML) lets you create, evaluate, and use ML models with GoogleSQL. The data stays in BigQuery, SQL-skilled analysts can build models, and BigQuery's engine handles the scaling. On the exam, BQML is often the right low-code answer when the data is already in BigQuery and the team knows SQL better than Python.
Match the Business Problem to a Model Type
| Business problem | BQML MODEL_TYPE options | Typical output function |
|---|---|---|
| Yes/no or category outcome (churn, fraud flag, product tier) | LOGISTIC_REG, BOOSTED_TREE_CLASSIFIER, RANDOM_FOREST_CLASSIFIER, DNN_CLASSIFIER, DNN_LINEAR_COMBINED_CLASSIFIER, AUTOML_CLASSIFIER | ML.PREDICT |
| Numeric value (spend, delivery time, price) | LINEAR_REG, BOOSTED_TREE_REGRESSOR, RANDOM_FOREST_REGRESSOR, DNN_REGRESSOR, DNN_LINEAR_COMBINED_REGRESSOR, AUTOML_REGRESSOR | ML.PREDICT |
| Future values of a time series | ARIMA_PLUS, ARIMA_PLUS_XREG, or no model with AI.FORECAST (TimesFM) | ML.FORECAST, AI.FORECAST |
| Customer or store segments without labels | KMEANS | ML.PREDICT (nearest centroid) |
| Product or content recommendations | MATRIX_FACTORIZATION (FEEDBACK_TYPE = EXPLICIT or IMPLICIT) | ML.RECOMMEND |
| Dimensionality reduction | PCA, AUTOENCODER | ML.PREDICT, ML.GENERATE_EMBEDDING |
| Unusual rows or time points | ARIMA_PLUS/ARIMA_PLUS_XREG (time series); KMEANS, PCA, AUTOENCODER (independent rows) | ML.DETECT_ANOMALIES |
BQML can also import trained models (TENSORFLOW, TENSORFLOW_LITE, ONNX, XGBOOST) to score them in SQL. It can also create remote models over Gemini and other Agent Platform models for generative tasks (Section 2.3).
Choosing among the supervised options
- Linear/logistic regression: fastest to train, easy to interpret through
ML.WEIGHTS, and a good baseline. - Boosted trees and random forests (XGBoost-based): usually the strongest choice for tabular data with non-linear interactions. They handle categorical features through label encoding.
- DNN and Wide & Deep: for large datasets with complex interactions. They are slower and harder to explain.
- AutoML classifier/regressor: BQML sends training to Agent Platform AutoML, which handles feature engineering and tuning. You control the
BUDGET_HOURSvalue (1.0-72.0, default 1.0) and anOPTIMIZATION_OBJECTIVEsuch asMAXIMIZE_AU_ROC.
Anatomy of CREATE MODEL
CREATE OR REPLACE MODEL `retail.churn_bt`
OPTIONS (
model_type = 'BOOSTED_TREE_CLASSIFIER',
input_label_cols = ['churned'],
auto_class_weights = TRUE,
data_split_method = 'AUTO_SPLIT',
enable_global_explain = TRUE,
model_registry = 'VERTEX_AI'
) AS
SELECT tenure_months, plan, monthly_spend, support_tickets, churned
FROM `retail.customer_features`;
input_label_colsnames the label.auto_class_weights = TRUEreweights classes by inverse frequency, which helps with imbalanced labels.DATA_SPLIT_METHODdefaults toAUTO_SPLIT. Below 500 rows, all rows go to training. From 500 to 50,000 rows, 20% goes to evaluation. Above 50,000 rows, 10,000 rows are held out. With hyperparameter tuning, the split is 80/10/10 for training, evaluation, and test.RANDOM,CUSTOM(a BOOL column),SEQ(ordered column, for time-aware splits), andNO_SPLITare also available.- Hyperparameter tuning uses
NUM_TRIALS,HPARAM_RANGE/HPARAM_CANDIDATES, andHPARAM_TUNING_ALGORITHM(VIZIER_DEFAULT,RANDOM_SEARCH, orGRID_SEARCH). You can check trials withML.TRIAL_INFO.
Forecasting Choices
| Need | Best BQML choice |
|---|---|
| Forecast one metric per series, explain trend, seasonality, and holiday effects | ARIMA_PLUS with TIME_SERIES_TIMESTAMP_COL, TIME_SERIES_DATA_COL, optional TIME_SERIES_ID_COL, HORIZON, HOLIDAY_REGION |
| Forecast using covariates (price, promotions, weather) | ARIMA_PLUS_XREG |
| Forecast immediately with no model to manage | AI.FORECAST with the built-in TimesFM foundation model |
ARIMA_PLUS trains one model per time series. Setting TIME_SERIES_ID_COL to a store or product ID fits thousands of series in a single statement. ML.EXPLAIN_FORECAST breaks forecasts into components, and ML.ARIMA_EVALUATE compares candidate models. TimesFM needs no training and offers little customization or explainability, so it fits quick baselines rather than cases that require explanations.
Evaluation and Prediction Functions
| Function | Purpose |
|---|---|
ML.EVALUATE | Metrics on held-out or supplied data (precision, recall, F1, log loss, ROC AUC for classifiers; MAE, MSE, R² for regressors) |
ML.CONFUSION_MATRIX, ML.ROC_CURVE | Threshold analysis for classifiers |
ML.PREDICT | Batch scoring. The THRESHOLD argument (default 0.5) sets the binary cutoff, and TRIAL_ID picks a tuning trial |
ML.FORECAST / AI.FORECAST | Future values with prediction intervals |
ML.DETECT_ANOMALIES | Flags anomalies with ANOMALY_PROB_THRESHOLD (time series) or contamination settings |
ML.RECOMMEND | Top items per user from matrix factorization |
ML.EXPLAIN_PREDICT | Predictions plus per-row feature attributions (TOP_K_FEATURES defaults to 5) |
SELECT customer_id, predicted_churned, predicted_churned_probs
FROM ML.PREDICT(
MODEL `retail.churn_bt`,
(SELECT * FROM `retail.active_customers`),
STRUCT(0.35 AS threshold));
Lowering the threshold to 0.35 catches more likely churners (higher recall) and flags more false positives (lower precision). Choose it from ML.ROC_CURVE output and the cost of each error type.
Getting Predictions to Consumers
- Batch in BigQuery: schedule
ML.PREDICTas a scheduled query or pipeline step and write results to a table for dashboards or activation. - Online through Agent Platform: register the model (
MODEL_REGISTRY = 'VERTEX_AI', optionally withVERTEX_AI_MODEL_IDand version aliases). Then deploy it from Model Registry to an endpoint. You don't need to export it or build a serving container. - Export:
EXPORT MODELwrites to Cloud Storage. Most model types export as TensorFlow SavedModel, and boosted tree and random forest models export as an XGBoost Booster. Automatic preprocessing is saved with the model, so clients send raw features that match whatML.PREDICTexpects.
Worked Scenario: Picking the Right Low-Code Path
A subscription company keeps 40 million customer-month rows in BigQuery. Marketing wants weekly churn scores for campaign targeting, and the analytics team writes SQL but not Python. Walk through the decision:
- Problem type: a yes/no label (churned next month), so this is binary classification.
- Where the data lives: BigQuery, which strongly favors BQML because nothing has to be exported.
- Model choice: start with
LOGISTIC_REGas a baseline, then trainBOOSTED_TREE_CLASSIFIERand compareML.EVALUATEresults on the same holdout. If there's more time than feature-engineering skill, tryAUTOML_CLASSIFIERwith a budget of a few hours. - Consumption: weekly scores go into a table, so a scheduled
ML.PREDICTquery is enough. There's no need for an online endpoint. - Imbalance: churners are rare, so use
AUTO_CLASS_WEIGHTS = TRUEand pick the threshold fromML.ROC_CURVE.
If the same company later needs a churn score during a live support chat, the need changes to online inference. Register the model in Model Registry and deploy it to an endpoint.
Exam Traps
- Choosing Python custom training when the data is already in BigQuery and a standard classifier or forecast would do. BQML avoids moving data and adds almost no operational overhead.
- Using
LINEAR_REGfor a yes/no label. Use a classifier. - Using a random split for time series. Use
SEQor a time-basedCUSTOMsplit so future rows don't leak into training. - Expecting BQML to host a low-latency REST endpoint by itself. Register the model or export it for online serving.
A retailer stores three years of daily sales for 4,000 stores in BigQuery. Analysts need per-store forecasts with explainable trend, seasonality, and holiday components, and they want to stay in SQL. Which approach fits best?
A fraud model built with LOGISTIC_REG in BigQuery ML misses too many fraudulent transactions at the default settings. The business accepts more false alarms. What is the most direct SQL change at prediction time?
A team trained a boosted tree classifier in BigQuery ML and now needs low-latency online predictions for a web app, with no container to build. What should they do?