4.1 Evaluating & Selecting Models in Model Garden
Key Takeaways
- Model Garden groups models into foundation models, fine-tunable models, and task-specific solutions from Google, partners, and open-model providers.
- Model as a Service (MaaS) serves partner and open models through serverless APIs, while self-deployment runs model weights on dedicated endpoints in your project.
- Google recommends MaaS for rapid prototyping and minimal operations, and self-deployment for custom weights, predictable high volume, and strict infrastructure control.
- An organization policy for Model Garden can allow only vetted models and deny access to all others at the organization, folder, or project level.
- Gemini model versions follow a published lifecycle with retirement dates, such as October 20, 2026 for gemini-2.5-pro, so model selection includes planning migrations.
The exam guide asks you to evaluate and select the appropriate model for a given task from Model Garden. Model Garden is a single catalog of models from Google, Google partners, and open-model providers. It offers a consistent deployment pattern and built-in links to tuning, evaluation, and serving. Picking well means matching the task, the data, and the operating constraints, not reaching for the biggest model.
What Model Garden Contains
| Category | Description | Examples of use |
|---|---|---|
| Foundation models | Pretrained multitask large models you can prompt, ground, and tune through Agent Studio, the API, or the SDK | Gemini for text, code, and multimodal reasoning. Gemini image models and Veo for media generation. Embedding models for retrieval |
| Fine-tunable models | Models you tune with a notebook or pipeline | Open models such as Gemma or Llama tuned with LoRA |
| Task-specific solutions | Prebuilt models, mostly ready to use, often customizable | Specialized vision or language task models |
Filters include task, model collection (managed by Google, partners, or you), provider, and features. Each model has a model card that describes capabilities, supported regions, limits, and license.
How a Model Is Served
| Serving option | What it is | Choose it when |
|---|---|---|
| Google models via API (Gemini, Veo) | Fully managed, pay per use or Provisioned Throughput | Default for Google's first-party models |
| Model as a Service (MaaS) | Serverless managed APIs for selected partner and open models. Enable the model's "API Service" card first | Rapid prototyping, minimal operations, bursty or variable traffic, standard non-customized model |
| Self-deployed from Model Garden | One-click deployment of model weights (including custom or fine-tuned weights) to a dedicated endpoint in your project | Custom weights, predictable high volume (lower cost per token at scale), exact hardware choice, strict residency, private VPC data path |
| Prebuilt serving containers | Optimized vLLM, Hex-LLM, SGLang, TGI, or TensorRT-LLM images | You want a tuned serving framework without maintaining images |
| Custom vLLM container | Your own image | Serving logic or dependencies the prebuilt containers don't support |
For self-deployed open models, you pay for the compute the endpoint uses (at prediction pricing) and for tuning compute (at custom training rates), not per token.
A Repeatable Selection Process
- Define the task and modality. Classification, extraction, summarization, code, chat with tools, image generation, video generation, or embeddings? Text-only or multimodal input?
- List the hard constraints. Latency (time to first token), throughput, cost per request, context length, data residency, whether customization is required, and license terms.
- Shortlist by tier. Within the Gemini family, Pro tiers give the most capability for complex reasoning, Flash balances quality and speed, and Flash-Lite is the fast, budget option for high-volume simple tasks. Add a partner or open model when a requirement calls for it, such as self-hosting in your VPC or a specific open license.
- Build an evaluation set from your own data. Public benchmarks don't reflect your prompts, documents, or edge cases. Use 50-200 representative prompts with reference answers or rubrics.
- Evaluate side by side. Run Gen AI evals with computation metrics, rubric-based metrics, or pairwise comparison with a judge model (Chapter 7). Compare quality, latency, and token cost together.
- Pick the smallest model that meets the quality bar. Then check that it supports the features you need: tuning, context caching, Provisioned Throughput, batch inference, and the global endpoint.
- Plan the lifecycle. Note the model version's retirement date and schedule re-evaluation against newer versions.
Worked example
A bank needs to extract 20 fields from loan documents, running 300,000 documents a night. Latency doesn't matter, but cost and accuracy do. Evaluation on 150 labeled documents shows Flash-Lite at 94% field accuracy, Flash at 97%, and Pro at 97.5%. The accuracy target is 96%. Flash meets the target at a fraction of Pro's cost. Because the job runs overnight, it should use batch inference for the discount (Section 4.3).
Governance and Safety in Model Selection
- Restrict models with organization policy. A Model Garden organization policy can allow only vetted models and deny all others. It applies at the organization, folder, or project level.
- Check provenance. Google tests and scans the serving and tuning containers it provides. Featured partner checkpoints are scanned for authenticity. Hugging Face models are scanned by Hugging Face. Unsafe models are blocked, and suspicious ones (for example, able to run remote code) are flagged but can still be deployed, so review them first.
- Licenses and terms differ between Google, partner, and open models. Include legal review in the shortlist.
- Data path: if regulations forbid multi-tenant managed services, self-deploy inside your project and VPC.
Model Lifecycle Awareness
Model versions follow a published lifecycle. For example, gemini-2.5-pro and gemini-2.5-flash list an October 20, 2026 retirement date, and several open-model MaaS endpoints were deprecated in July 2026 with retirement on October 21, 2026 (self-deployment from Model Garden remains an option). Good designs:
- Put the model ID in configuration, not scattered through code.
- Keep the evaluation set as a regression suite for testing replacement models.
- Re-run evaluations and re-tune tuned models before the base version retires.
Signals That You Picked the Wrong Model
| Symptom in production | Likely selection issue | Next step |
|---|---|---|
| Good quality but cost far over budget | Tier too large for the task | Evaluate Flash or Flash-Lite, or tune a smaller model |
| Fast and cheap but frequent reasoning errors | Tier too small | Evaluate a larger tier or add grounding and examples |
| Frequent 429s during business hours | Capacity plan, not model quality | Review consumption options (Section 4.3) |
| Can't meet residency or no-multi-tenancy rules | Wrong serving option | Self-deploy an open model in your project |
Embedding Models Are Selections Too
RAG and semantic search depend on the embedding model. Gemini embedding models (gemini-embedding-001, gemini-embedding-2) are current. Older text embedding models such as text-embedding-005 and multimodalembedding@001 list an April 1, 2027 retirement date. Changing embedding models means re-embedding the corpus, because vectors from different models aren't comparable.
A startup wants to test three open LLMs for a chatbot over the next two weeks. Traffic will be low and spiky, and no one on the team manages GPUs. Which Model Garden serving option fits best?
A healthcare company must run a fine-tuned open model with custom weights inside its own project and VPC, and regulations prohibit multi-tenant managed inference. Which option meets these requirements?
An enterprise wants developers to use only models that its AI governance board has approved in Model Garden. What is the most direct control?