13.1 AI/ML Platform Architecture: Vertex AI, Gemini Enterprise Agent Platform & AI Hypercomputer
Key Takeaways
- Choose prebuilt Google AI APIs when the task is commodity (vision labeling, translation, speech); choose custom Vertex AI model development when the model is a differentiator trained on proprietary data.
- The Gemini Enterprise Agent Platform orchestrates end-to-end ML workflows with pipelines that automate and schedule data integration, training, evaluation, and deployment steps.
- AI Hypercomputer is Google Cloud's integrated supercomputing stack — co-designed TPUs and GPUs with optimized networking and software (e.g., GKE, JAX/PyTorch support) — for large-scale training and serving.
- TPUs deliver the best price-performance for large dense training; GPUs provide framework and library flexibility; consumption models include on-demand, reserved/committed capacity, and preemptible or Dynamic Workload Scheduler options.
- Agent Platform data integration grounds agents and models in enterprise data sources so workflows act on governed, current data rather than stale exports.
AI/ML Platform Architecture: Vertex AI, Gemini Enterprise Agent Platform & AI Hypercomputer
Architectural Objective: The current exam guide treats AI capability as core architect knowledge: Domain 1.3 (Designing network, storage, and compute resources) explicitly includes Google Cloud AI and machine learning solutions — Gemini LLMs, Agent Builder, Model Garden, and AI Hypercomputer — and Domain 2.4 (Leveraging Gemini Enterprise Agent Platform for end-to-end ML workflows) requires provisioning decisions for ML pipelines, data integration, and compute at AI scale.
Decision Layer 1: Prebuilt APIs vs. Custom Models vs. Agents
Every AI exam scenario starts with a build-versus-buy disposition:
+-----------------------------------------------------------------------------------+
| AI SOLUTION DISPOSITION FRAMEWORK |
+-----------------------------------------------------------------------------------+
| 1. PREBUILT API | Commodity task, no proprietary training data: Vision AI, |
| | Video AI, Natural Language AI, Speech-to-Text, Translation. |
| 2. GENERATIVE & | Grounded generation, search, or conversation over enterprise|
| AGENTS | data: Gemini models, Model Garden models, Agent Builder / |
| | Gemini Enterprise Agent Platform agents. |
| 3. CUSTOM MODEL | Proprietary data or novel task: full Vertex AI lifecycle - |
| (Vertex AI) | data prep, training, evaluation, deployment, monitoring. |
+-----------------------------------------------------------------------------------+
The exam punishes jumping straight to training a custom model when a prebuilt API or a grounded Gemini workflow solves the requirement in days with no MLOps burden.
Decision Layer 2: Vertex AI as the Unified ML Platform
Vertex AI is the single platform for the custom-model lifecycle:
- Data & features: managed datasets and Feature Store for shared, governed features.
- Training: custom training jobs and Pipelines (Kubeflow Pipelines/Vertex AI Pipelines) that codify reproducible training workflows; hyperparameter tuning jobs.
- Registry & evaluation: the Model Registry versions models; evaluation and Vertex Explainable AI support auditable deployment decisions (critical for the Altostrat-style explainable AI requirement).
- Serving: online endpoints with traffic splitting (canary between model versions) and batch prediction jobs.
- Monitoring: production drift and skew detection feeding retraining triggers.
Decision Layer 3: Gemini Enterprise Agent Platform for End-to-End Workflows
Blueprint 2.4 names the Gemini Enterprise Agent Platform as the vehicle for end-to-end ML workflow automation:
- Agent Platform Pipelines automate and orchestrate the ML lifecycle: data ingestion, preprocessing, training/fine-tuning, evaluation, and packaging agents or models for deployment as repeatable, scheduled, event-triggered flows rather than notebooks-driven manual steps.
- Data integration preparation: connectors and grounding patterns bring enterprise data (BigQuery, Cloud Storage, operational systems) into the agent/ML workflow so inferences act on governed current data with lineage.
- Agent delivery: agentic behaviors — multi-step tool use, retrieval over enterprise corpora, task automation — are composed, secured, and deployed through the platform rather than bespoke orchestration code.
Decision Layer 4: AI Hypercomputer & Compute Provisioning
Large-scale training and inference is a compute architecture decision. AI Hypercomputer is Google Cloud's integrated supercomputing environment: co-designed TPU pods and GPU (A/H-series, e.g., H100/B200-class) machines wired with high-bandwidth Jupiter-class datacenter networking, orchestrated by GKE or managed schedulers, and consumable through several capacity models.
| Provisioning Question | Architectural Answer |
|---|---|
| Dense large-model training at best price-performance | Cloud TPUs co-designed with the JAX/XLA software stack |
| Framework flexibility, CUDA-dependent libraries, inference diversity | GPU-backed Compute Engine or GKE node pools |
| Guaranteed capacity for a fixed training calendar | Reservations / committed capacity |
| Elastic batch training tolerant of interruption | Spot/preemptible capacity or Dynamic Workload Scheduler (flex-start) |
| Serving models behind low-latency APIs | Vertex AI endpoints (or GKE) with autoscaling and GPU/TPU node pools |
Cost governance still applies: training capacity choices feed the CUD/Spot arithmetic from Chapter 2, and idle GPU capacity is one of the fastest ways to blow a cloud budget — pair every AI design with scheduling and shutdown automation.
Concrete Scenario: Retail Demand Forecasting + Shopping Agent
A retailer needs weekly demand forecasts on proprietary sales history and a customer-facing shopping agent:
- Forecasting (custom, differentiating): BigQuery historical data -> Vertex AI Pipelines scheduled retraining -> Model Registry -> endpoint; or BigQuery ML (ARIMA_PLUS-class) when the team wants SQL-native modeling.
- Shopping agent (gen-AI platform): grounded in the product catalog via Agent Platform data integration; deployed through the Gemini Enterprise Agent Platform with policies and observability.
- Compute: seasonal retraining bursts run on Spot/flex-scheduled GPU capacity; the agent serves on autoscaling Vertex endpoints.
[!IMPORTANT] Exam Watch: Map requirements to the disposition ladder first: prebuilt API -> grounded generative/agent -> custom Vertex AI. For end-to-end workflow automation answers favor Agent Platform pipelines over ad-hoc cron/notebook orchestration; for large-scale training compute answers name AI Hypercomputer (TPUs/GPUs) with an explicit consumption model (reserved vs. Spot vs. Dynamic Workload Scheduler).
A startup needs image labeling, text sentiment scoring, and speech transcription in its app next month, has no proprietary training data, and has no ML staff. What should the architect recommend first?
An enterprise data science team must automate its ML lifecycle — data ingestion, training, evaluation, and deployment — as repeatable, schedulable workflows grounded in current enterprise data. Which blueprint-aligned approach is correct?
A research team must train a large dense transformer with maximum throughput per dollar over a fixed 3-week window, while a separate batch of nightly fine-tuning jobs can tolerate interruption. Which compute provisioning choices fit both?
After deploying a custom Vertex AI model to production, the team observes prediction accuracy decaying as input data drifts. What is the architecturally correct response?