2.3 Data Ingestion, Parsing & Chunking Strategies

2.3 Data Ingestion, Parsing & Chunking Strategies

The success of a Retrieval-Augmented Generation (RAG) architecture is fundamentally constrained by the quality of its data ingestion pipeline. Even the most capable foundation model cannot recover from poorly chunked data that severs critical context, truncates tabular structures, or dilutes semantic density. Amazon Bedrock Knowledge Bases provides parsing choices plus default, fixed-size, no-chunking, hierarchical, and semantic text chunking approaches. A custom transformation Lambda can alter content, metadata, or chunk output for supported ingestion configurations.

Document Parsing: Standard vs. Advanced Foundation Model Parsing

Before documents can be chunked and vectorized, raw file formats must be parsed into clean, structured text.

Standard Parsing

Standard parsing extracts plain text from supported enterprise formats:

• Supported Formats: Plain Text (.txt), Markdown (.md), HTML (.html), PDF (.pdf), Microsoft Word (.docx), and Comma-Separated Values (.csv).
• Processing Characteristics: Standard parsing extracts sequential text streams using deterministic text extraction libraries. It is computationally lightweight, fast, and incurs no additional model inference charges.
• Limitations: Standard parsing struggles with multi-column layouts, reading order ambiguities, embedded scanned images, charts, and complex financial tables. In standard PDF extraction, a two-column research paper is often read straight across horizontal lines, interweaving unrelated sentences from separate columns.

Advanced Parsing with Foundation Models

To handle complex documents such as scanned PDFs, quarterly financial statements, and technical schematics, Bedrock Knowledge Bases offers Advanced Parsing powered by Foundation Models.

• Underlying Engine: Employs vision-capable foundation models (such as Anthropic Claude 3 Sonnet or Haiku) to visually analyze document pages.
• Capabilities:
  • Converts complex tables into cleanly formatted Markdown or HTML tables, preserving column-row relationships and headers.
  • Corrects multi-column reading orders by identifying visual layout blocks and reading left column to completion before moving to right column.
  • Extracts text embedded inside images, flowcharts, graphs, and corporate diagrams.
• Operational Tradeoffs: Advanced parsing increases ingestion latency and incurs additional Bedrock model inference costs per parsed page. Consequently, teams often route simple text documents through standard parsing while targeting dense PDF reports with advanced parsing.

Detailed Analysis of Chunking Strategies

Bedrock Knowledge Bases supports five chunking strategies, each tailored to specific document structures and retrieval objectives:

[Document Ingestion]
       │
       ├─► 1. Fixed-Size Chunking (Default tokens + overlap)
       ├─► 2. Hierarchical Chunking (Small Child vectors ──► Large Parent context)
       ├─► 3. Semantic Chunking (Embed sequential sentences ──► Split on topic shift)
       ├─► 4. Custom Chunking (AWS Lambda preprocessing)
       └─► 5. No Chunking (1 Document = 1 Vector chunk)

---

1. Fixed-Size Chunking (Default)

Fixed-size chunking divides text into uniform segments defined by token count:

• Key Parameters:
  • maxTokens: Maximum number of tokens per chunk (default: 300; range: 20 to model limit).
  • overlapPercentage: Percentage of tokens repeated between adjacent chunks (default: 20%; range: 1% to 99%).
• Mechanics: A sliding window moves across the tokenized text. When the token count hits maxTokens, a boundary is created, and the next chunk starts by rewinding overlapPercentage tokens back into the preceding chunk.
• Overlap Role: Overlap prevents hard semantic cliffs where a critical sentence or entity name is cut in half across chunk boundaries.
• Best For: Homogeneous, narrative text documents (blogs, general prose, news articles) where structure is uniform and high ingestion throughput is required.
• Limitations: Does not understand semantic boundaries; can slice through tables, lists, or structured legal clauses.

---

2. Hierarchical Chunking (Parent-Child)

Hierarchical chunking addresses the classic RAG Retrieval-Context Dilemma:

• Small chunks produce precise vector embeddings that closely match specific search queries, but they lack sufficient context for the foundation model to generate a complete answer.
• Large chunks provide rich context for generation, but their embeddings are broad and diluted, causing vector search to miss specific details.

Hierarchical chunking resolves this by creating a two-tier structure:

• Parent Chunks: Larger blocks of text (e.g., 1,500 tokens) that provide broad, comprehensive context.
• Child Chunks: The parent chunk is subdivided into smaller sub-chunks (e.g., 300 tokens) with overlap.
• Runtime Mechanics:
  1. Vector embeddings are generated and indexed only for the child chunks.
  2. During runtime retrieval, the user's query vector matches the most relevant child chunk via vector similarity.
  3. Instead of returning just the child chunk, Bedrock replaces the child chunk with its corresponding parent chunk before passing context to the foundation model.
• Best For: Complex technical manuals, research papers, legal agreements, and corporate policies where granular facts must be understood within their broader section context.

---

3. Semantic Chunking

Semantic chunking uses natural language understanding to determine where one topic ends and another begins:

• Mechanics:
  1. The document is divided into individual sentences.
  2. An embedding model generates vector representations for each sentence or small sentence sliding window.
  3. Bedrock computes the cosine distance between consecutive sentence embeddings.
  4. Significant drops in semantic similarity indicate a shift in topic or theme. A chunk boundary is dynamically placed at that transition point.
• Key Parameters:
  • maxTokens: Upper boundary cap to prevent chunks from growing excessively long.
  • bufferSize: Number of surrounding sentences evaluated together to smooth out transient sentence-level variance.
  • breakpointPercentileThreshold: Sensitivity threshold for detecting topic shifts (typically 80th–95th percentile).
• Best For: Unstructured long-form documents with frequent topical transitions (meeting transcripts, multi-topic whitepapers, executive briefings).
• Limitations: Incurs higher ingestion latency and computational cost because every sentence must be embedded during the ingestion phase.

---

4. Custom Chunking via AWS Lambda

When standard algorithms cannot capture proprietary document conventions, Bedrock allows developers to inject an AWS Lambda function directly into the ingestion pipeline:

• Workflow:
  1. Bedrock retrieves documents from the data source and applies initial parsing.
  2. Bedrock invokes the designated Lambda function, delivering the parsed document content and metadata in an event payload.
  3. The Lambda function executes custom parsing, regex boundary splitting, metadata tagging, or table serialization.
  4. Lambda returns an array of discrete chunks and associated custom metadata back to Bedrock.
  5. Bedrock generates embeddings for the Lambda-provided chunks and writes them to the vector store.
• Best For:
  • Splitting proprietary codebases along class or function boundaries.
  • Parsing regulatory documents by statutory section numbers (e.g., Section 4.1(a)).
  • Injecting dynamic document-level metadata (such as author, security classification, or publication year) into individual chunk records.

---

5. No Chunking (Document-as-a-Chunk)

In this configuration, Bedrock treats each ingested file as a single, undivided chunk:

• Constraint: The total token count of each document must not exceed the maximum context window of the configured embedding model (e.g., 8,192 tokens for Amazon Titan Text Embeddings V2).
• Best For:
  • Pre-chunked datasets prepared by external ETL pipelines.
  • Standalone Frequently Asked Question (FAQ) documents where each file contains one discrete question and answer pair.
  • Short customer support knowledge articles or product catalog item descriptions.

---

Strategy Comparison & Decision Matrix

Exam Scenarios & Architectural Gotchas

Exam Scenario 1: Tabular Financial Discrepancies

An accounting firm notices that queries asking for specific EBITDA figures from 10-K financial reports return incorrect numbers or hallucinated estimates. Standard parsing and 300-token fixed chunking are currently enabled.

• Analysis: Standard parsing destroyed the tabular formatting of the PDF balance sheet, and fixed-size chunking split rows across two different chunks, separating column headers from dollar amounts.
• Remediation:
  1. Enable Advanced Parsing with Foundation Models (Claude 3) to convert financial tables into Markdown tables.
  2. Implement Hierarchical Chunking with 1,500-token parent chunks and 300-token child chunks. The child chunk matches the specific financial metric query, while the 1,500-token parent provides the entire Markdown table context to the foundation model.

Exam Scenario 2: Maintaining Document Structure in Standardized Contracts

An enterprise legal department has 50,000 commercial agreements formatted strictly with standard headers: ARTICLE I: DEFINITIONS, ARTICLE II: OBLIGATIONS, etc. Fixed-size chunking frequently splits definitions across chunks, causing contract review agents to misinterpret terms.

• Remediation: Implement Custom Chunking with AWS Lambda. Write a Python script in Lambda that utilizes regular expressions to split text on ARTICLE [I|V|X]+ boundaries, ensuring each contractual clause remains intact as a single semantic entity.

Gotcha: Excessive Overlap Percentage

Setting fixed-size chunking overlap to 40%–50% does not improve accuracy; it significantly inflates storage costs in the vector database and floods top-k retrieval results with redundant duplicate passages, pushing other potentially relevant context out of the foundation model's prompt window. Treat overlap as a measured parameter: evaluate retrieval quality, duplicate-result rate, prompt-token use, and storage cost on the actual corpus rather than declaring one percentage universally optimal.