18.2 Generative AI Integration with the Google Gen AI SDK

Generative AI Integration with the Google Gen AI SDK

Cloud developers can add summarization, classification, semantic search and assisted workflows to an ordinary application. The exam's current role description includes generative AI integration, while its detailed objectives emphasize development, security, deployment and observability. Apply those same engineering controls to the model call: explicit identity, bounded execution, tested contracts and measured results. A model endpoint is a dependency with quotas and failure modes, not an infallible source of business truth.

Choose a Supported API, SDK and Model

Use the Google Gen AI SDK for new Gemini code. The old vertexai.generative_models and vertexai.language_models modules were deprecated with a June 24, 2026 removal date. Other non-generative Vertex AI SDK modules have separate lifecycles. Do not copy an old tutorial's import or retired Gemini 1.5 model name into a new service without checking its status.

Choose a supported model ID and location from the current catalog, and pin the selection in configuration. Flash-class and Pro-class labels describe product families, not permanent guarantees about context length, output quality, price or latency. Benchmark representative requests, including long inputs and concurrent load. A global endpoint and a regional endpoint can have different availability or residency implications; selecting a region in unrelated application configuration does not automatically constrain every model request.

For a Google Cloud backend, attached service identity and Application Default Credentials (ADC) avoid distributing service-account keys. Enable the required service, grant only the needed model permissions, and configure project and location explicitly. The Gemini Developer API and Google Cloud endpoint have different account, authentication and governance choices. Neither an API key nor ADC is inherently an authorization design for end users. Keep credentials out of browser bundles and inspect the chosen endpoint's actual data-handling terms and supported controls.

Structured Output Is a Contract to Validate

A response schema constrains supported output structure. It does not prove that an order ID exists, a refund is authorized, a summary is accurate, or a classification is unbiased. The application must handle blocked, empty, truncated and invalid responses as well as successful ones. Lower temperature changes sampling behavior; zero temperature is not a guarantee of reproducibility or factual correctness. Sampling parameter names, ranges and defaults depend on the model, so do not apply a universal topK limit or assume every model permits every setting.

## Requires google-genai. Configure a currently supported model ID externally.
import json
import os
from google import genai
from google.genai import types

client = genai.Client(
    vertexai=True,
    project=os.environ["GOOGLE_CLOUD_PROJECT"],
    location=os.environ["GOOGLE_CLOUD_LOCATION"],
    http_options=types.HttpOptions(api_version="v1"),
)
response = client.models.generate_content(
    model=os.environ["GEMINI_MODEL_ID"],
    contents="Classify this synthetic feedback: the package arrived damaged.",
    config=types.GenerateContentConfig(
        response_mime_type="application/json",
        response_schema={
            "type": "OBJECT",
            "properties": {
                "category": {"type": "STRING", "enum": ["DAMAGE", "OTHER"]},
                "needs_review": {"type": "BOOLEAN"},
            },
            "required": ["category", "needs_review"],
        },
        max_output_tokens=256,
    ),
)
if not response.text:
    raise ValueError("No usable classification; route to review")
result = json.loads(response.text)
if result.get("category") not in {"DAMAGE", "OTHER"}:
    raise ValueError("Unexpected classification")
## A classifier result must not directly approve a refund.

This is a small integration example, not a complete request handler. Add an overall deadline, supported error handling, output validation, usage accounting and a fallback path. Preserve enough non-sensitive telemetry to distinguish an authentication error from quota exhaustion, safety blocking and an application parser failure. Do not log complete customer prompts or sensitive responses by default. Test the exact SDK/model combination before shipping, because schema support and response metadata evolve.

Function Calling and the Execution Boundary

Function calling lets the model request a typed application operation, such as get_order_status(order_id). A declaration describes the name, arguments and purpose. The model can return a function-call payload instead of a final answer. The application then checks the request, executes an allowed operation, and returns the result as a function response before asking for the final answer. Some SDK features automate part of this loop, but the application's trust boundary remains its responsibility.

A useful execution sequence is:

1. Authenticate the user outside the model and obtain the user's tenant/account scope.
2. Supply narrow tool declarations and only the context necessary for the task.
3. Validate the returned function name and arguments against an allowlist and schema.
4. Apply ordinary server-side authorization to the requested resource. A model-supplied customer ID is not proof of ownership.
5. Execute the operation with bounded time, least privilege and an idempotency key where effects can be retried.
6. Return a minimized result to the model, then validate the final response for the intended workflow.

For example, a balance lookup tool should derive the account scope from the authenticated session and use parameterized queries. An unrestricted execute_sql(query) tool gives the model far too much authority. A payment or delete operation needs its normal business authorization and audit trail even when the model correctly formats the arguments. Tool choice modes control whether a call is requested; they do not grant permission to carry it out.

Separate tool requests from successful execution. If the database call times out, do not invent a balance or report that a refund completed. Return an appropriate tool failure and preserve the transaction's actual status. Cap the number of tool iterations to prevent loops, and remember that retries of the model request can produce repeated requests for the same business action. Use durable operation IDs and explicit receipts rather than relying on the conversation to remember what happened.

Retrieval-Augmented Generation and Vector Search

Retrieval-augmented generation (RAG) adds selected external evidence to a model request. It is useful for internal manuals or policies that are absent from the model's training data or change frequently. It can improve grounding, but does not eliminate hallucination, stale evidence, prompt injection or unauthorized disclosure.

During ingestion, split documents into meaningful chunks, preserve document IDs and source locations, attach access-control metadata, and generate embeddings with a supported model. The embedding version and dimensions used for stored documents must be compatible with the query embeddings. Re-embedding with another model is a data migration, not just changing an endpoint string.

A vector index stores vectors and identifiers; a nearest-neighbor result does not necessarily contain the full original document. Retrieve the matching text from its source store, recheck authorization, and provide only authorized chunks to the generation call. Enforce access control during candidate selection and before returning citations. A restricted document should not appear in a user's answer simply because its embedding is close to the question.

Vertex AI Vector Search offers approximate and brute-force search configurations. Tree-based approximate search trades some recall for efficiency; brute force is useful for a quality baseline or smaller datasets. Select distance metrics and normalization consistently with the embedding model. Tune recall, candidate count and capacity using evaluation data. A service-level product description does not promise that a particular 50-million-document system meets a twenty-millisecond end-to-end requirement.

Evaluate the Complete Application

Build a held-out evaluation set from realistic tasks. Include questions whose answer is absent, conflicting source documents, inaccessible records and malicious instructions embedded in retrieved text. The expected result can be a refusal to infer missing data or an escalation for human review. A system instruction is useful context, but cannot make untrusted documents safe by itself.

Measure retrieval precision/recall separately from answer quality. If the correct source was never retrieved, changing the generation prompt may not solve the problem. If retrieval is correct but the answer ignores it, inspect prompt structure, source ordering and response evaluation. Keep model and embedding version changes observable so a regression can be associated with a release.

For a document classifier, record the class confusion matrix, invalid-output rate and latency distribution. For a support assistant, check citation accuracy and whether answers respect document permissions. For a tool-using assistant, test duplicate execution, partial failure and unauthorized IDs. Cost depends on actual usage, input/output size, retrieval and serving resources; token limits help bound a call but do not cap an entire application's monthly spending.

A practical rollout uses a small authorized cohort, clear fallback behavior and monitoring by model/configuration version. Preserve the previous supported configuration for rollback, while avoiding an assumed ability to call a model after its retirement. Treat each model upgrade as a dependency change requiring evaluation and operational checks.

Official references checked October 18, 2026: Google Cloud 1, Google Cloud 2.