4.4 Tool Calling, FM API Resilience & Model Routing

4.4 Tool Calling, FM API Resilience & Model Routing

Tool / Function Calling with the Converse API

One of the most powerful features of the Converse API is its native, unified tool-calling interface. Tool calling allows a foundation model to recognize when external data or computation is required (e.g., checking database records, invoking a REST API, or running mathematical scripts) and emit structured parameters for external execution.

[!IMPORTANT] Client-Side Tool Execution vs. Bedrock Agents: A vital exam distinction: In the Converse API, Amazon Bedrock does NOT execute the tool code. Bedrock merely determines which tool to call and outputs the validated JSON input parameters. The client application must intercept this request, execute the local code, database query, or Lambda function, and return the execution output back to Bedrock as a toolResult message.

┌─────────────────────────────────────────────────────────────────────────────┐
│                     CONVERSE API TOOL USE CYCLE                             │
├─────────────────────────────────────────────────────────────────────────────┤
│  1. Client sends User Prompt + Tool Specifications (toolConfig)             │
│         │                                                                   │
│         ▼                                                                   │
│  2. Bedrock Model evaluates prompt and decides to invoke a tool             │
│         │                                                                   │
│         ▼                                                                   │
│  3. Bedrock returns Response: stopReason = 'tool_use'                       │
│     Payload: contentBlock containing toolUse (toolUseId, name, input)       │
│         │                                                                   │
│         ▼                                                                   │
│  4. Client application executes local tool logic (e.g. SQL Query)           │
│         │                                                                   │
│         ▼                                                                   │
│  5. Client calls Converse API again with 3 messages in history:             │
│     - Original user prompt                                                  │
│     - Assistant response containing toolUse block                           │
│     - New user message containing toolResult block matching toolUseId       │
│         │                                                                   │
│         ▼                                                                   │
│  6. Bedrock Model synthesizes final grounded answer (stopReason = 'end_turn')│
└─────────────────────────────────────────────────────────────────────────────┘

Step 1: Defining the Tool Schema in toolConfig

Tools are declared using JSON Schema definitions inside the toolConfig structure:

{
  "toolConfig": {
    "tools": [
      {
        "toolSpec": {
          "name": "get_customer_account_balance",
          "description": "Retrieves real-time checking and savings balance for a verified customer ID.",
          "inputSchema": {
            "json": {
              "type": "object",
              "properties": {
                "customer_id": {
                  "type": "string",
                  "description": "The 8-digit alphanumeric customer identifier."
                },
                "account_type": {
                  "type": "string",
                  "enum": ["CHECKING", "SAVINGS", "INVESTMENT"],
                  "description": "The specific account category."
                }
              },
              "required": ["customer_id", "account_type"]
            }
          }
        }
      }
    ],
    "toolChoice": {
      "auto": {}
    }
  }
}

toolChoice Configuration Modes

• auto: The model autonomously determines whether to call a tool or respond with regular conversational text.
• any: Forces the model to call at least one of the provided tools, preventing direct conversational responses.
• tool: Forces the model to invoke one specific, designated tool (e.g., "tool": {"name": "get_customer_account_balance"}).

Step 2: Handling the Model's toolUse Response

When the model decides to invoke a tool, the HTTP response contains:

• stopReason: "tool_use"
• output.message.content: Contains a toolUse block with an auto-generated toolUseId:

{
  "stopReason": "tool_use",
  "output": {
    "message": {
      "role": "assistant",
      "content": [
        {
          "text": "Checking balance records for customer CUST-8492."
        },
        {
          "toolUse": {
            "toolUseId": "tooluse_01AB987XYZ",
            "name": "get_customer_account_balance",
            "input": {
              "customer_id": "CUST-8492",
              "account_type": "CHECKING"
            }
          }
        }
      ]
    }
  }
}

Step 3: Returning the toolResult to Converse

The client application executes the backend function and submits the conversation history back to the Converse API. Crucially, the tool result must be encapsulated inside a user role turn:

import boto3

bedrock_runtime = boto3.client("bedrock-runtime", region_name="us-east-1")

messages = [
    {"role": "user", "content": [{"text": "What is the checking balance for customer CUST-8492?"}]},
    {"role": "assistant", "content": [
        {"text": "Checking balance records for customer CUST-8492."},
        {"toolUse": {
            "toolUseId": "tooluse_01AB987XYZ",
            "name": "get_customer_account_balance",
            "input": {"customer_id": "CUST-8492", "account_type": "CHECKING"}
        }}
    ]},
    {"role": "user", "content": [
        {"toolResult": {
            "toolUseId": "tooluse_01AB987XYZ",
            "content": [{"json": {"balance": 14850.75, "currency": "USD", "status": "ACTIVE"}}],
            "status": "success"
        }}
    ]}
]

response = bedrock_runtime.converse(
    modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
    messages=messages
)
# Final response: "The checking account balance for customer CUST-8492 is $14,850.75 USD."

---

InvokeModel vs. Converse API Comparison

---

Exam Scenarios & Common Traps

Real-World Exam Scenario

A multinational insurance firm maintains a customer service orchestration engine on AWS Lambda. The application currently routes claims inquiries to Anthropic Claude 3.5 Sonnet using InvokeModel. The engineering team decides to implement dynamic model routing to send simple questions to Meta Llama 3 8B and complex policy queries to Claude 3.5 Sonnet. When changing the modelId to Llama 3, the Lambda function throws execution exceptions because Llama rejects the anthropic_version and messages payload syntax.

Architecture Solution: Refactor the Lambda integration to utilize the Amazon Bedrock Converse API (bedrock_runtime.converse). The standardized payload structure allows the Lambda function to dynamically set modelId to either meta.llama3-8b-instruct-v1:0 or anthropic.claude-3-5-sonnet-20241022-v2:0 with zero changes to request serialization, system prompts, inference parameters, or response parsing logic.

Common Architectural Traps

• Trap 1: Placing System Prompts in the messages Array: In the Converse API, attempting to pass {"role": "system", "content": ...} inside the messages array results in an immediate ValidationException. System prompts must always be defined in the dedicated top-level system parameter.
• Trap 2: Consecutive Same-Role Messages: Sending two user messages back-to-back will cause Bedrock to fail the request. All inputs for a single turn must be aggregated as elements of the content list in a single user turn.
• Trap 3: Expecting Bedrock to Invoke External Code: Assuming that defining a tool in toolConfig causes Bedrock to execute an AWS Lambda function automatically. The Converse API only emits the structured toolUse JSON. Autonomous serverless execution is the responsibility of Amazon Bedrock Agents, not the raw Converse API.

FM API resilience and model routing

Treat the foundation-model call as a distributed dependency. Validate request size and schema, set connect and read deadlines, classify retryable errors, use exponential backoff with jitter, and bound total attempts. Streaming clients must handle an error after partial output without presenting an incomplete structure as final.

Static routing maps known request classes to eligible targets. Dynamic routing can use Step Functions, application metrics, or a Bedrock prompt router for supported model pairs. The eligibility layer must run first: modality, tenant entitlement, residency, tool support, and safety policy are not optional quality preferences. AWS X-Ray can trace the surrounding API Gateway, Lambda, queue, and tool calls, while Bedrock request IDs and CloudWatch metrics support correlation at the managed service boundary.