2.3 Fine-Tuning & Evaluating Gemini Models from BigQuery

The June 2026 exam guide lists "fine-tuning Gemini models using BigQuery" under low-code AI. The scenario: an organization's examples (support replies, product descriptions, labeled tickets) already sit in BigQuery, and prompting alone doesn't consistently produce the right style or labels. BigQuery ML runs a supervised tuning job on Agent Platform without the team writing Python or moving data to Cloud Storage.

Step 1: Connect BigQuery to Agent Platform

BigQuery reaches Agent Platform models through a Cloud resource connection. You can reference a specific connection (project.region.connection_id) or use DEFAULT. The connection must be in the same location as the dataset that holds the model.

Step 2: Create a Baseline Remote Model

CREATE OR REPLACE MODEL `support.gemini_baseline`
REMOTE WITH CONNECTION DEFAULT
OPTIONS (ENDPOINT = 'gemini-2.5-pro');

You can name the model (BigQuery picks the regional endpoint for the dataset's location) or give a full endpoint URL. Supported Gemini models can also use the global endpoint. It improves availability and cuts 429 resource-exhausted errors, but you can't control which region processes the request, so skip it when data-processing location matters.

Run the baseline over held-out rows with AI.GENERATE_TEXT. It is table-valued and supports Gemini, partner, and open models. AI.GENERATE_TABLE returns output that follows a schema you define.

Step 3: Create the Tuned Model

Tuning uses the same CREATE MODEL ... REMOTE statement plus an AS SELECT that supplies training pairs:

CREATE OR REPLACE MODEL `support.gemini_tuned`
REMOTE WITH CONNECTION DEFAULT
OPTIONS (
  endpoint = 'gemini-2.5-pro',
  prompt_col = 'prompt',
  input_label_cols = ['label'],
  max_iterations = 1500,        -- 3 epochs for 500 rows
  learning_rate_multiplier = 1.0,
  data_split_method = 'RANDOM',
  data_split_eval_fraction = 0.1,
  evaluation_task = 'CLASSIFICATION')
AS
SELECT CONCAT('Classify this ticket as BILLING, TECH, or ACCOUNT: ', body) AS prompt,
       category AS label
FROM `support.labeled_tickets`;

With AUTO_SPLIT, fewer than 500 rows all go to training. From 500 to 50,000 rows, 20% is held out for evaluation. Above 50,000 rows, 10,000 rows are held out. After splitting, the training set needs at least 10 rows.

Cost: you pay BigQuery for bytes processed from the training table, and Agent Platform for tokens processed during tuning.

Step 4: Evaluate Baseline vs. Tuned

SELECT * FROM ML.EVALUATE(
  MODEL `support.gemini_tuned`,
  (SELECT prompt AS input_text, label AS output_text FROM `support.holdout_tickets`),
  STRUCT('classification' AS task_type));

Run the same ML.EVALUATE query against the baseline model and the tuned model on identical holdout rows. Only keep the tuned model if it's clearly better. Not every use case improves with tuning.

Preparing Tuning Data That Actually Helps

Tuning copies patterns from your examples, including mistakes. Before running CREATE MODEL:

• Use the production prompt template. If inference will send "Classify this ticket as BILLING, TECH, or ACCOUNT: ...", the training prompts need that same prefix. A template that differs between tuning and inference is a quiet form of training-serving skew.
• Make labels exact and consistent. Normalize casing and spelling (BILLING, not Billing or billing issue) so the model learns one output form.
• Cover the real distribution. Include rare categories, long and short inputs, and edge cases you see in production, not only easy examples.
• Remove duplicates and leakage. Duplicate rows inflate effective epochs, and rows copied into both training and holdout inflate evaluation scores.
• Hold out a separate evaluation table from the same time period and distribution, so baseline and tuned models are judged on identical inputs.
• Redact sensitive data before tuning if the use case doesn't need it (see Chapter 5 on Sensitive Data Protection).

Running the tuned model in production

A tuned remote model is used like any other BigQuery ML model. Scheduled queries or pipeline steps call AI.GENERATE_TEXT or AI.GENERATE_TABLE over new rows and write results back to BigQuery. Because the tuned model is tied to a specific base model version, plan to re-tune and re-evaluate when that base version reaches its retirement date. Treat the evaluation query as a reusable regression test.

Tune, Prompt, or Ground?

Exam Traps

• Recommending an export to Cloud Storage and a Python tuning script when the data is in BigQuery and the team wants low-code SQL.
• Forgetting the IAM grant on the connection's service account, a common cause of failed tuning statements.
• Judging tuning by a few eyeballed outputs instead of comparing ML.EVALUATE metrics for baseline and tuned models on the same holdout set.
• Using the global endpoint when regulations require processing in a specific region.