2.3 The Five Layers of the Gen AI Landscape

The Five Layers of the Gen AI Landscape

Executive Summary: Generative AI is not a single technology product; it is a complex, multi-tiered ecosystem. To formulate effective enterprise strategies, leaders must understand the five foundational layers of the Gen AI landscape: Hardware & Infrastructure, Foundation Models, Platforms & Developer Tooling, AI Agents & Workflows, and Enterprise End-User Applications. By mapping workloads across these five layers, organizations can determine where to invest capital, where to leverage managed cloud services, and how to execute the optimal "Build vs. Buy vs. Assemble" strategy.

Without a coherent architectural mental model, enterprises frequently misallocate engineering resources—such as attempting to build bespoke model training platforms when managed cloud platforms already exist, or buying rigid end-user SaaS tools that fail to integrate with proprietary enterprise data. Deconstructing the Gen AI stack into five distinct layers provides clarity for architectural governance and executive investment.

---

The Five Architectural Layers of Generative AI

┌─────────────────────────────────────────────────────────────┐
│ Layer 5: Enterprise End-User Applications                   │
│ (Gemini for Workspace, Salesforce Einstein, Custom Portals)  │
├─────────────────────────────────────────────────────────────┤
│ Layer 4: AI Agents & Workflows                              │
│ (Agent Platform, Multi-Agent Orchestration, Tools) │
├─────────────────────────────────────────────────────────────┤
│ Layer 3: Platforms & Developer Tooling                      │
│ (Agent Platform Model Garden, Vector Search, MLOps, Studio)       │
├─────────────────────────────────────────────────────────────┤
│ Layer 2: Foundation Models                                  │
│ (Gemini 3.1 Pro/Flash, Gemma, Imagen, Med-LM, Claude, Llama)│
├─────────────────────────────────────────────────────────────┤
│ Layer 1: Hardware & Cloud Infrastructure                    │
│ (Cloud TPUs, NVIDIA GPUs, Jupiter Fabric, Cloud Storage)    │
└─────────────────────────────────────────────────────────────┘

Layer 1: Hardware & Cloud Infrastructure

The physical and virtual foundation that powers all generative AI computation. Training massive models and serving real-time inference across billions of parameters requires specialized hardware architectures:

• AI Accelerators (TPUs & GPUs): General-purpose CPUs cannot efficiently perform the massive matrix multiplications required by transformer architectures. Google Cloud designs proprietary Tensor Processing Units (TPUs)—including TPU v5p (optimized for massive foundation model training), TPU v5e (cost-optimized for high-efficiency inference and medium training), and the sixth-generation Trillium (TPU v6e). Concurrently, Google Cloud offers massive clusters of NVIDIA GPUs (including H100, H200, and Blackwell B200 accelerators).
• High-Throughput Network Fabric (Jupiter & OCS): Accelerators do not operate in isolation; thousands of chips must synchronize model weights with microsecond latency. Google Cloud utilizes its custom Jupiter datacenter network fabric and innovative Optical Circuit Switching (OCS) to dynamically route petabits of inter-accelerator traffic, eliminating network bandwidth as a training bottleneck.
• High-Performance Storage: Training and inference pipelines require rapid data delivery. Services like Cloud Storage FUSE and Filestore stream multi-terabyte datasets to accelerator clusters without stalling compute cycles.

Layer 2: Foundation Models

The cognitive core of the generative stack. This layer includes raw base models and instruction-tuned models across text, code, audio, image, and video modalities:

• Proprietary Frontier Models: State-of-the-art models developed by hyperscalers and frontier labs, such as Google's Gemini 3.1 Pro and Gemini 3.5 Flash. These models deliver industry-leading reasoning, multimodal perception, and multi-million token context windows, accessed via secure cloud APIs.
• Open-Weights Models: Freely downloadable model weights, such as Google's Gemma 4 family (2B, 9B, 27B) or Meta's Llama series. These allow complete architectural inspection, private hosting, and local customization, but place operational and security maintenance onto the enterprise.
• Domain-Specialized Models: Foundation models tuned specifically for regulated industry taxonomies, such as Google's Med-LM (trained on clinical medical knowledge) and Sec-LM (integrated into Google Cloud Security Operations for threat intelligence).
• Specialized Media Models: Models engineered for non-text generation, including Google's Imagen 4 (photorealistic image synthesis) and Veo (high-definition video generation).

Layer 3: Platforms & Developer Tooling

The operational nervous system that allows developers to discover, customize, deploy, and evaluate foundation models securely. Google Cloud's primary manifestation of this layer is Agent Platform:

• Model Garden: An enterprise registry providing single-click access to over 150 first-party (Gemini, Imagen), open-source (Gemma, Llama, Mistral), and third-party partner (Anthropic Claude) models with enterprise SLAs.
• Agent Studio: A collaborative web-based prototyping environment where developers and business analysts test prompt designs, experiment with system instructions, and compare model outputs.
• Vector Infrastructure (Agent Platform Vector Search): Extremely fast, horizontally scalable vector database capability that powers enterprise RAG by matching user query embeddings against millions of document vectors with millisecond latency.
• MLOps & Pipeline Orchestration (Agent Platform Pipelines): Automated CI/CD pipelines for data preprocessing, continuous model evaluation, scheduled batch inference, and automated retraining.
• Evaluation & Guardrail Harnesses: Tooling that quantifies model performance (faithfulness, hallucination, safety violation rates) before production rollout.

Layer 4: AI Agents & Workflows

The dynamic layer that transforms static foundation models into active, goal-oriented autonomous systems. While a foundation model simply responds to text prompts, an AI Agent can perceive, plan, execute tools, and retain stateful memory across multi-step business processes:

• Agent Frameworks (Agent Platform): Low-code and code-first platforms that simplify building enterprise-grade search engines and conversational agents grounded in company data.
• Reasoning and Action (ReAct) Loops: Architectural patterns where the model iteratively reasons about a problem, decides what external action to take, inspects the result, and refines its next step.
• Tool Execution & Function Calling: Enabling the model to securely invoke external APIs, execute SQL queries in BigQuery, update tickets in ServiceNow, or fetch live inventory from SAP.
• Enterprise Grounding & Memory Systems: Maintaining session state, user preferences, and enterprise security context so agents can carry out complex, multi-day customer interactions.

Layer 5: Enterprise End-User Applications

The user-facing software applications that deliver tangible business value directly to end users, knowledge workers, and customers:

• Turnkey Workplace Productivity: Tools like Gemini for Google Workspace, which embeds generative capabilities directly into Docs, Gmail, Sheets, Slides, and Meet, enabling automated draft writing, data visualization, and meeting summarization.
• Enterprise Search Portals: Internal Google-quality search engines (built via Agent Search) allowing employees to query millions of internal HR documents, engineering wikis, and legal policies using natural language.
• Customer Service Engagement Platforms: Autonomous contact center agents (e.g., Google Cloud Customer Engagement Suite) resolving voice and chat support inquiries without human intervention.
• Developer Productivity Copilots: Integrated development environment (IDE) assistants (e.g., Gemini Code Assist) that generate, refactor, explain, and review enterprise code in real time.

---

The Strategic Enterprise Spectrum: Build vs. Buy vs. Assemble

A central responsibility of a Generative AI Leader is deciding where an organization should direct its capital and engineering talent across the five layers. Enterprises approach this through the Build vs. Buy vs. Assemble framework:

Strategic Value Capture Across the Stack

High  ▲
      │                                  [ASSEMBLE]
      │                             Domain Agents & RAG
Value │                              (Layers 3 & 4)
Diff- │                           High ROI / Core IP
eren- │
tiation│ [BUY]
      │ Turnkey Apps (Layer 5)
      │ Fast ROI / Non-Core
      │                                           [BUILD]
      │                                    Base Foundation Models
      │                                         (Layers 1 & 2)
Low   │                                     Extreme Cost / Low ROI
      └─────────────────────────────────────────────────────────────►
       Low                    Engineering Complexity             High

1. The "Buy" Strategy (Layer 5)

• Best suited for: Universal, non-differentiating business productivity tasks (e.g., email drafting, general document proofreading, routine code completion, generic sales outreach).
• Mechanism: Procuring commercial SaaS licenses (such as Gemini for Google Workspace or Gemini Code Assist).
• Pros: Instant time-to-value, zero machine learning engineering required, vendor maintains infrastructure and safety.
• Cons: Offers no proprietary competitive moat; competitors have access to the exact same capabilities.

2. The "Assemble" Strategy (Layers 3 & 4) — The Enterprise Sweet Spot

• Best suited for: Differentiating business workflows that rely on proprietary enterprise data, company-specific standard operating procedures (SOPs), and internal transaction systems (e.g., automated insurance claims triage, proprietary algorithmic equity research, personalized patient care plans).
• Mechanism: Assembling managed platform services (Agent Platform, Vector Search, Agent Builder) with frontier foundation models (Gemini) and connecting them to enterprise databases.
• Pros: Creates a sustainable competitive advantage using proprietary company knowledge while avoiding infrastructure maintenance. Maximizes ROI.
• Cons: Requires cross-functional engineering talent (software engineers, data engineers, prompt architects, domain experts).

3. The "Build" Strategy (Layers 1 & 2)

• Best suited for: The handful of global tech giants, well-funded national defense entities, or specialized AI research labs.
• Mechanism: Procuring thousands of AI accelerators to collect, clean, and pre-train proprietary base models from scratch.
• Pros: Total architectural control over model weights and training distribution.
• Cons: Financially catastrophic for standard commercial enterprises. Extreme capital expenditure ($10M–$100M+), massive execution risk, and ongoing obsolescence as cloud providers release newer, cheaper, more capable models every few months.

---

Architectural Layer Matrix

---

Concrete Business Scenario: Omni-Channel Retailer Transformation

A national department store chain with 400 retail locations modernizes its business using the five-layer stack:

• Layer 1 (Infrastructure): The retailer relies on Google Cloud's managed TPU and GPU infrastructure underlying Agent Platform, avoiding any direct management of physical data center hardware.
• Layer 2 (Foundation Models): They utilize Gemini 3.5 Flash for high-frequency customer chatbot queries and Gemini 3.1 Pro for back-office supplier contract review and seasonal assortment planning.
• Layer 3 (Platform): Developers leverage Agent Platform Model Garden to test models and Agent Platform Vector Search to index 200,000 product catalog specifications, real-time inventory counts, and return policies.
• Layer 4 (Agents): Engineers use Agent Platform to construct an autonomous "Personal Shopping & Returns Agent." The agent uses function calling to check live stock in the retailer's ERP database, initiate return shipping labels via FedEx APIs, and apply loyalty discounts to the customer's shopping cart.
• Layer 5 (Applications): Store associates use Gemini for Workspace to summarize regional inventory meetings, while customers interact with the shopping agent embedded directly in the mobile app.

By focusing their engineering investments on Layers 3 and 4 (Assemble), the retailer delivers a proprietary, revenue-generating customer experience in three months without attempting to train a base model.

---

Exam Tips & Common Pitfalls

[!TIP] Exam Tip: Understand the distinction between Layer 2 (Foundation Models) and Layer 4 (AI Agents & Workflows). A foundation model in isolation can only generate text or predict tokens based on its prompt. An AI Agent in Layer 4 incorporates a reasoning framework (e.g., ReAct) and tool/function calling capabilities that allow it to take real-world actions, query external databases, and execute external APIs.

[!WARNING] Common Pitfall: Believing that an enterprise must choose between exclusively "Buying" or exclusively "Building." Highly effective organizations adopt a hybrid approach: they Buy Layer 5 turnkey productivity tools for generic office tasks, and Assemble Layer 3 and 4 platform tools with enterprise data to build proprietary, high-value workflows.