2.4 Embedding Models & Vector Representations
Key Takeaways
- Benchmark dimensions and embedding models on labeled domain queries before migrating an index.
- Unit-normalized vectors make dot-product and cosine rankings equivalent, but normalization must be verified.
- Apply authorization metadata before retrieval results reach the model and keep metadata synchronized with source changes.
2.4 Embedding Models & Vector Representations
An embedding model maps content into a numeric vector so semantically related items can be found by distance. Retrieval quality depends on the complete contract among the embedding model, input preparation, vector dimensions, distance metric, index, metadata, and evaluation set. A dimension count by itself does not prove quality.
Select an embedding model from workload evidence
Check the exact model documentation for supported languages, modalities, maximum input size, output dimensions, normalization options, and Region availability. Text-only models cannot directly represent image content. Multilingual support should be tested with the organization's actual language pairs, not inferred from an English benchmark. If a provider uses asymmetric embeddings, encode corpus passages with the document input type and queries with the query input type.
Amazon Titan Text Embeddings V2 supports configurable output dimensions. Lower dimensions reduce raw vector storage and arithmetic, but there is no universal percentage of quality retained for every corpus. AWS service-card benchmark values are aggregate results, not a promise that a particular 512-dimension index will preserve a fixed percentage of a 1024-dimension index's recall. Benchmark the candidate dimensions against labeled queries and acceptance thresholds.
Distance and normalization
Cosine similarity compares direction, dot product compares the inner product, and Euclidean distance compares geometric separation. When both vectors are normalized to unit length, ranking by dot product is equivalent to ranking by cosine similarity. This mathematical fact does not mean every service normalizes automatically. Confirm the model output and index configuration.
| Choice | Benefit | Risk to test |
|---|---|---|
| Fewer dimensions | Lower vector storage and faster comparisons | Lost distinctions and lower recall on domain terms |
| Normalized vectors | Stable cosine or dot-product behavior | Incorrect assumptions if only one side is normalized |
| Approximate nearest neighbor index | Low-latency search at scale | Recall varies with graph or probe parameters |
| Exact search | Useful ground truth for smaller evaluation sets | Cost and latency grow with corpus size |
Raw storage is approximately record count multiplied by dimensions multiplied by bytes per value, but a production index also stores graph edges, document text, metadata, replicas, and engine overhead. Capacity planning should measure the actual index rather than multiplying vectors alone.
Metadata is part of retrieval design
Metadata improves precision and enforces query constraints that semantic distance cannot express. Useful fields include tenant ID, source type, jurisdiction, effective date, product, language, and access group. Define types consistently: a year stored sometimes as text and sometimes as a number produces unreliable filters. Avoid high-cardinality or oversized fields that the selected store handles poorly.
Apply authorization filters before results reach the model. Post-filtering a globally retrieved top-k list can leak metadata and can return no useful passages even when authorized matches exist lower in the ranking. For especially sensitive tenants, physical or logical index separation may be required in addition to filters.
Metadata has a lifecycle. When a policy changes jurisdiction or a user's access group changes, stale attributes can make an otherwise current embedding unsafe. Synchronization must update vectors, text, and metadata as one governed record.
Worked dimension comparison
For ten million float32 vectors, raw vector values alone require about 40.96 GB at 1,024 dimensions and 20.48 GB at 512 dimensions. That arithmetic excludes graph edges, text, metadata, replicas, and engine overhead, so it is a lower bound rather than a cluster-size recommendation.
Create a labeled set of real queries, relevant documents, and hard negatives. Build parallel indices at both dimensions with the same corpus and measure recall at k, mean reciprocal rank, p95 query latency, build time, and total storage. Slice results by language and document type. If 512 dimensions meets every threshold, the savings are supported by evidence; if a rare but critical class loses recall, the aggregate average is insufficient.
Document the chosen embedding model ID and preprocessing. Even dimensions with the same number are not compatible across different models. Mixing their vectors in one field destroys geometric meaning.
Input preparation controls
Chunk content before embedding according to its semantic structure and the embedding model's input limit. Preserve headings and source identifiers, normalize encoding, and avoid silently truncating the decisive sentence. Generate document and query vectors with the same compatible model version and preprocessing contract, except for documented asymmetric input roles.
A final selection report should name the corpus snapshot, query set, relevance judgments, index settings, hardware or managed capacity, and confidence limits. Without these details, a retrieval score cannot be reproduced after a model or engine update.
A team wants to reduce Titan Text Embeddings V2 dimensions to lower index cost. How should it decide whether 512 dimensions are acceptable?