1.6 Model Garden: Evaluating and Selecting the Right Model
Key Takeaways
- Model Garden catalogues three families with different operational contracts: Google first-party models, third-party partner models served as a managed API, and open-weight models you deploy onto your own endpoint.
- Managed API models are billed per token and require no infrastructure; self-deployed open models are billed for the accelerator hours the endpoint holds, whether or not traffic arrives.
- Model selection is a four-way trade-off among task capability, latency, cost per unit of work, and deployment constraints such as data residency and VPC isolation.
- A model card records the intended use, training data characteristics, evaluation results, and known limitations, and is the artifact a governance review reads before approval.
- Selection should be settled with a measured evaluation on representative data — an offline eval set scored with task metrics or an LLM-as-a-judge rubric — rather than a leaderboard.
1.6 Model Garden: Evaluating and Selecting the Right Model
Blueprint reference: Section 1.2, "Evaluating and selecting the appropriate model for a given task from Gemini Enterprise Agent Platform Model Garden."
Model Garden is the single catalogue of models available on the Agent Platform, and the exam treats "which model would you choose" as an architecture question rather than a benchmark question. The differentiator between a good and a bad answer is almost never raw quality — it is the operational contract attached to each family.
Three Families, Three Contracts
| Family | Examples | How it is served | How it is billed | Who patches it |
|---|---|---|---|---|
| Google first-party | Gemini, Imagen, Veo, embedding models | Managed API endpoint, no infrastructure | Per input/output token or per generated asset | |
| Partner models as a service | Third-party frontier models offered through Model Garden | Managed API endpoint | Per token, billed through Google Cloud | The partner, via Google |
| Open-weight models | Gemma, Llama-family, and other open checkpoints | You deploy to an Agent Platform Inference endpoint, GKE, or Cloud Run | Accelerator + node hours for as long as the endpoint exists | You |
That last column is the exam's favourite. A managed API model costs nothing when idle. A self-deployed open model on a GPU-backed endpoint bills for every hour that endpoint is up, whether it serves ten requests or ten million. For spiky, low-volume workloads a managed API is dramatically cheaper; for sustained high-throughput workloads with a fixed, predictable request rate, a reserved self-deployed endpoint can win on unit cost.
The Four Selection Axes
1. Task capability. Does the model do the job at all? Multimodal input, long context, tool calling, structured output, and non-English language coverage are hard capability gates. A cheaper model that cannot accept a PDF is not cheaper; it is disqualified.
2. Latency. Model families ship in tiers — a fast, small variant and a slower, stronger variant. Interactive chat and autocomplete need the fast tier. Nightly document summarization does not, and paying frontier prices for a batch job is pure waste.
3. Cost per unit of work. Compare cost per completed task, not per token. A stronger model that answers correctly on the first attempt can beat a cheaper model that needs three retries plus a validation pass.
4. Deployment constraints. This is where architecture decisions get made:
- Data residency. A regulated workload confined to a specific region can only use models available in that region.
- Network isolation. If inference traffic may not traverse a public endpoint, a self-deployed open model behind Private Service Connect may be the only viable option.
- Weight control. Some governance regimes require the organization to hold the weights, which forces the open-weight path.
- Customization depth. Managed APIs support supervised tuning and adapters; full-parameter modification of a proprietary model is not on offer.
Reading a Model Card
Every Model Garden entry carries a model card. Governance reviews read four parts of it:
- Intended use and out-of-scope use. Deploying outside the stated intent is the fastest way to fail an internal review.
- Training data characteristics. Drives fairness questions and jurisdictional risk.
- Evaluation results. Benchmarks reported by the publisher, on their chosen data.
- Known limitations. Documented failure modes, language gaps, and safety caveats.
- Licence. Open weights do not mean unrestricted commercial use; licences vary and some carry usage restrictions.
The exam-relevant instinct: a model card is evidence for a decision, not proof of fitness for your task. Publisher benchmarks are run on public datasets that rarely resemble enterprise data.
Selecting by Measurement, Not Leaderboard
The defensible selection process is short:
- Build a representative eval set. 100–300 real inputs with expected outputs or grading criteria, drawn from actual traffic and deliberately including the hard cases.
- Shortlist by hard constraints. Region, modality, context length, isolation. This usually eliminates most of the catalogue immediately.
- Score the shortlist. Use task metrics where a ground truth exists (exact match, F1, ROUGE for summarization) and an LLM-as-a-judge rubric where it does not.
- Measure latency and cost on the same runs. Record p50 and p95 latency and actual token spend per task, not list prices.
- Record the decision. Which models were compared, on what data, with what result — this is what an audit asks for.
Candidate Task score p95 latency Cost / 1k tasks Region OK
------------- ----------- ------------ ---------------- ---------
Fast tier 0.81 640 ms low yes
Strong tier 0.93 2.4 s ~8x fast tier yes
Open-weight 0.86 410 ms* endpoint hours yes
* self-deployed on a dedicated accelerator; cost is fixed, not per call
A frequent exam pattern presents exactly this table in prose and asks for the choice given a stated constraint. Anchor on the constraint: an interactive assistant with a strict latency budget takes the fast tier or the self-deployed model; a nightly batch job with a quality bar takes the strong tier; a workload that must not leave a VPC takes the self-deployed model regardless of score.
Exam Traps
- Choosing the highest-benchmark model by default. Nearly always wrong when latency or cost is stated.
- Ignoring idle cost of self-deployed endpoints. Low-volume plus dedicated accelerator equals waste.
- Treating "open model" as "free." The weights are free; the accelerators are not.
- Assuming every Model Garden model is in every region. Availability varies and residency requirements bind.
- Selecting on a public benchmark instead of your own eval set.
A compliance team must run document classification entirely inside their VPC with no inference traffic traversing a public endpoint, and the workload is a steady 400 requests per second during business hours. Which Model Garden option fits?
A team must summarize roughly 900 support transcripts once per night. Quality matters more than speed, and there is no interactive user waiting. Two candidates score 0.79 and 0.94 on their evaluation rubric, with the stronger model costing about eight times more per task and taking four times longer. Which choice is best supported?
During a governance review, a reviewer asks for evidence that a selected Model Garden model is appropriate for screening job applications. Which artifact and which caveat should the ML engineer present?
A product team deploys an open-weight model from Model Garden to a dedicated GPU-backed Agent Platform Inference endpoint. Traffic averages 30 requests per day with occasional bursts. Finance flags the cost as unexpectedly high. What is the underlying cause and the correct fix?