3.2 Files API, Document Blocks & Citations

Multimodal Document Capabilities in Claude

Enterprise AI architectures frequently require reasoning over information-dense business documents—including corporate annual reports, regulatory filings, legal agreements, technical manuals, and financial audits. Anthropic's Claude provides native multimodal document intelligence, allowing developers to submit raw documents directly to the Messages API without requiring third-party optical character recognition (OCR) or lossy document conversion utilities.

Claude processes documents through a Hybrid Multimodal Ingestion Engine:

• Native Text Extraction: For digital documents (such as plaintext, Markdown, CSV, and PDFs with accessible text streams), Claude extracts the underlying character streams and tokenizes them using standard Byte-Pair Encoding (BPE).
• Visual Page Rendering: For scanned documents, complex multi-column layouts, financial spreadsheets, embedded diagrams, flowcharts, and handwritten annotations, Claude renders pages into high-resolution visual representations.
• Token Consumption Mechanics for PDFs: Understanding the token cost of rendered pages is a crucial architectural concept tested on the CCDV-F exam. When Claude renders a PDF page visually, it does not tokenize individual characters. Instead, each page is converted into visual image tokens—typically consuming approximately 1,600 to 2,000+ tokens per rendered page, depending on the document's aspect ratio and resolution. Consequently, a scanned 40-page contract will consume between 64,000 and 80,000+ input tokens solely for visual ingestion, regardless of how few words appear on each page. Engineering teams must account for this visual token overhead when sizing context windows and estimating production operational costs.

---

Document Content Blocks vs. The Files API

Developers can ingest documents into Claude using two primary architectural paradigms: inline base64 document blocks or the asynchronous Files API.

Pattern A: Inline Base64 Document Blocks

For transient, single-turn workflows where a document is processed once and immediately discarded, developers transmit documents inline within the content array of a user message turn:

{
  "role": "user",
  "content": [
    {
      "type": "document",
      "source": {
        "type": "base64",
        "media_type": "application/pdf",
        "data": "JVBERi0xLjQKJ..."
      },
      "title": "Q3_Financial_Filing.pdf",
      "context": "Quarterly filing submitted to regulatory authorities."
    },
    {
      "type": "text",
      "text": "What was the operating margin for the cloud infrastructure division?"
    }
  ]
}

Supported media types for document content blocks include:

• application/pdf: Portable Document Format. The page ceiling is 600 pages per request when the request's context window is 1M tokens, and 100 pages when it is under 1M.
• text/plain: Unstructured plaintext files.
• text/markdown: Structured markdown documentation.
• text/csv: Comma-separated tabular datasets.
• text/html: Raw web document markup.

Payload Size Limitations: Base64 encoding introduces an approximate 33% bandwidth expansion compared to raw binary data. Because the Anthropic Messages API enforces a strict 32 MB request body ceiling, raw unencoded files submitted via inline base64 must remain below ~24 MB.

Pattern B: The Files API (/v1/files)

When documents are large, referenced repeatedly across multi-turn user conversations, or shared across parallel worker agents, transmitting multi-megabyte base64 strings in every API call wastes client bandwidth, increases network latency, and bloats payload sizes. The Files API decouples file upload from inference:

1. Upload Stage: The client uploads the raw document once via POST /v1/files using standard multipart/form-data. Anthropic processes and stores the file securely, returning a unique identifier (e.g., "file_011CNnbx8xyz").
2. Inference Stage: In subsequent calls to POST /v1/messages, the application references the file by ID within the document content block source:

{
  "type": "document",
  "source": {
    "type": "file",
    "file_id": "file_011CNnbx8xyz"
  }
}

1. Lifecycle Management: Applications manage their file storage footprint using standard REST endpoints: listing files (GET /v1/files), retrieving file metadata (GET /v1/files/{file_id}), and deleting stale assets (DELETE /v1/files/{file_id}).

---

Pre-Flight Payload Sizing with count_tokens

Ingesting large documents presents serious operational risks: an unexpectedly large document can breach the model's context window (1M tokens on Claude Sonnet 5 and Claude Opus 5, 200K on Claude Haiku 4.5), hit rate limits, or generate massive unintended API billing. To mitigate these risks, Anthropic provides the Token Counting Endpoint (POST /v1/messages/count_tokens).

Mechanics and Behavioral Contract

The count_tokens endpoint accepts the exact same payload schema as POST /v1/messages (including model, system, messages, tools, and document blocks).

curl https://api.anthropic.com/v1/messages/count_tokens \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-sonnet-5",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "document",
            "source": {
              "type": "base64",
              "media_type": "application/pdf",
              "data": "JVBERi0xLjQKJ..."
            }
          },
          {
            "type": "text",
            "text": "Extract all risk factors outlined in Section 3."
          }
        ]
      }
    ]
  }'

Response:

{
  "input_tokens": 38420
}

Architectural Value for Production Systems

• Zero-Inference Cost: Token counting executes server-side tokenization and visual page resolution without running model inference or generating output tokens. It incurs zero generation cost and minimal API latency.
• Pre-Flight Context Verification: Verifies whether a complex payload—combining system prompts, tools, conversation history, and a 40-page PDF—fits comfortably inside the context window before committing expensive compute.
• Dynamic Cost Governance: Computes exact input cost before triggering large batch jobs (input_tokens * unit_price). If token count is small, the orchestrator can dynamically route the query to Claude Haiku 4.5; if extensive context reasoning is needed, it routes to Claude Sonnet 5.
• Dynamic Chunking Trigger: If input_tokens exceeds a predefined budget (e.g., 150,000 tokens), the ingestion engine automatically branches to a semantic chunking pipeline.

---

The Citations API: Verifiable Enterprise Provenance

In enterprise environments—such as legal discovery, clinical healthcare, financial compliance, and contract management—unsubstantiated claims and model hallucinations carry legal and financial liability. Stakeholders cannot act on AI-generated insights unless each assertion can be traced directly to an auditable source passage.

The Citations API provides native, cryptographically verifiable source attribution. By enabling citations directly on document blocks, Claude anchors its generation to verifiable text spans within the source documents.

Enabling Citations in the Request

Citations are activated on individual document blocks by configuring "citations": {"enabled": true}:

{
  "role": "user",
  "content": [
    {
      "type": "document",
      "source": {
        "type": "base64",
        "media_type": "application/pdf",
        "data": "JVBERi0xLjQKJ..."
      },
      "title": "Master_Services_Agreement_2026.pdf",
      "citations": {
        "enabled": true
      }
    },
    {
      "type": "text",
      "text": "What are the termination provisions and notice requirements?"
    }
  ]
}

Structure of Citation Response Blocks

When citations are enabled, Claude annotates generated text blocks with structured citation objects:

{
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "Either party may terminate this Agreement without cause upon sixty (60) days prior written notice to the non-terminating party.",
      "citations": [
        {
          "type": "char_location",
          "cited_text": "Either party may terminate this Agreement without cause upon giving sixty (60) days prior written notice to the other party.",
          "document_index": 0,
          "document_title": "Master_Services_Agreement_2026.pdf",
          "start_char_index": 18450,
          "end_char_index": 18575
        }
      ]
    }
  ]
}

Anatomy of a Citation Object

• type: Indicates the citation anchoring mechanism ("char_location" for character offset boundaries).
• cited_text: The verbatim quote extracted directly from the source document that proves the assertion.
• document_index: Zero-based integer indicating which document block in the request array supplied the citation.
• document_title: The title attribute supplied on the original document block.
• start_char_index & end_char_index: Exact zero-based character offsets within the extracted document text string.

Enterprise Applications of Citations

1. Automated Compliance Auditing: Background verification services programmatically verify that 100% of claims in a generated report contain matching citations with zero string edit distance from the source repository.
2. Interactive UI Highlighting: Document viewing applications (e.g., PDF.js) use start_char_index and end_char_index to draw interactive bounding boxes and citation tooltips directly over original contract pages.
3. Hallucination Suppression: Activating citations structurally discourages speculative generation, as Claude will only generate factual claims that can be anchored to verbatim source passages.

---

Document Handling Best Practices & Cost Optimization

Document Hard Boundaries

• Page Ceiling per Request: 600 pages when the request's context window is 1M tokens, dropping to 100 pages when it is under 1M. The ceiling applies to the entire request payload, not to one document block, so several PDFs in one request share the budget. Dense pages can also exhaust the context window before the page ceiling is reached.
• 32 MB Request Size Limit: The 32 MB ceiling also covers the whole request payload. Larger documents must be uploaded through the Files API and referenced by file_id, which keeps the request itself small.
• Multi-Document Concurrency: Applications can supply multiple document blocks in a single request (e.g., comparing three competing vendor contracts), provided cumulative tokens remain within the model context window.

Chunking Strategies for Massive Documents

When working with 500-page SEC 10-K filings or technical manuals that exceed the 100-page threshold:

1. Semantic Partitioning: Split documents along logical chapter or section headings rather than arbitrary byte offsets.
2. Overlapping Sliding Windows: Partition documents into 30-page windows with a 3-page overlap to prevent factual fragmentation across page boundaries.
3. Hierarchical Indexing: Extract high-level chapter summaries using Claude Haiku 4.5, and route targeted chapters to Claude Sonnet 5 for detailed analysis.

Combining Documents with Prompt Caching

Re-evaluating a 60-page document across multi-turn conversations can rapidly become cost-prohibitive if the document is re-tokenized on every turn. Combining document blocks with Prompt Caching (cache_control: {"type": "ephemeral"}) is the most impactful optimization available:

• Cache Breakpoint Placement: Attach "cache_control": {"type": "ephemeral"} directly to the document content block or the preceding system message.
• Cost Reductions: On the initial call, the document is tokenized and written to the cache (regular input token price + 25% cache write fee). On subsequent calls within the 5-minute cache TTL, cached document tokens are read at a 90% discount (0.1x regular input token price).
• Latency Reductions: Bypassing document tokenization and visual prefill reduces Time-To-First-Token (TTFT) by up to 80% on large documents.

---

Comparison of Document Ingestion Strategies

---

Common CCDV-F Exam Traps & Pitfalls

• Underestimating Scanned Document Tokens: Assuming PDF token counts correlate strictly with word count. Scanned pages or pages with complex visual diagrams are rendered as images, consuming ~1,600 to 2,000+ tokens per page regardless of word count.
• Passing Incomplete Payloads to count_tokens: Treating count_tokens as a simple string utility. The endpoint requires the full Messages API request structure (including model and messages). Omitting required fields results in a 400 Bad Request.
• Confusing Character Offsets with PDF Byte Offsets: Believing start_char_index and end_char_index correspond to binary byte positions in the PDF file. They represent zero-based character offsets in the extracted text representation generated by Claude.
• Assuming a Fixed 100-Page Ceiling: The limit is 600 pages per request when the request's context window is 1M tokens and 100 pages below that, and it applies to the whole request payload rather than to a single document block.