The Geometric Limitations of Vector Embeddings in Retrieval Systems

A recent paper from Google DeepMind reveals fundamental mathematical constraints that affect how well vector embeddings can perform retrieval tasks. These findings have significant implications for anyone building or relying on semantic search systems.

The Core Problem with Fixed-Dimension Embeddings

Dense embedding models convert queries and documents into fixed-dimension vectors. The retrieval process depends on a simple mechanism: calculate the cosine similarity (or dot product) between a query vector and document vectors, where values close to 1 indicate high relevance and values near 0 suggest low relevance.

This approach requires all vectors to share the same dimension, typically denoted as D. For practical reasons related to memory and storage, we want D to remain reasonably small. However, the Google DeepMind research demonstrates that when D is too small relative to the number of queries and documents, the system cannot adequately represent all possible relevance relationships.

Understanding the Geometric Constraint

Think of vector embeddings as arrows pointing in different directions within a geometric space. In two-dimensional space (like a flat plane), there are only so many distinct directions available. As you add more vectors, they inevitably start pointing in similar directions, limiting your ability to create the precise similarity relationships you need.

When you have:

• M queries

• N documents

• A fixed embedding dimension D

Each query needs to maintain specific similarity scores with every document. If D is too small, the geometric space simply doesn't have enough "room" for all vectors to point in the directions needed to achieve the desired similarity values.

How Retrieval Systems Should Work

In an ideal retrieval system, you would have a ground truth matrix where each entry indicates whether a document is relevant to a query:

Ground Truth Matrix Structure:

• Rows represent queries

• Columns represent documents

• Entry = 1 if document is relevant to query

• Entry = 0 if document is irrelevant

The embedding model creates this matrix through multiplication:

• Query matrix (M × D): Each row is a query vector

• Document matrix (D × N): Each column is a document vector

• Result: Score matrix B (M × N)

The Threshold Partition Goal

A well-functioning embedding system should produce a score matrix where, for each query, you can identify a threshold value (lambda) that cleanly separates relevant from irrelevant documents:

• All relevant documents have similarity scores > lambda

• All irrelevant documents have similarity scores < lambda

This threshold property would allow the system to partition similarity scores into two distinct groups for each query, making it straightforward to identify which documents matter.

The Impossibility Result

The Google DeepMind paper proves that when D is not large enough relative to M, N, and the specific pattern of relevance relationships, achieving this clean partition becomes mathematically impossible.

Key Finding: There is no embedding model that can represent every possible relevance pattern (every possible configuration of relevant and irrelevant document-query pairs) when the dimension D is too constrained.

This isn't a limitation of current technology or algorithms. It's a fundamental geometric constraint. The vector space simply lacks sufficient dimensionality to encode all the distinct similarity relationships required.

What This Means for Retrieval Systems

The implications are significant:

Theoretical Limitation: Even with perfect training and optimal embeddings, fixed-dimension vector models cannot universally represent all relevance patterns when working with large document collections and diverse queries.

Practical Trade-offs: System designers face an unavoidable tension between:

• Keeping embedding dimensions small (for efficiency)

• Maintaining retrieval quality across diverse queries

• Scaling to large document collections

Design Considerations: Understanding these geometric constraints helps explain why:

• Increasing embedding dimensions often improves retrieval quality

• Different embedding models perform better on different query types

• No single embedding approach works optimally for all use cases

Moving Forward

This research doesn't suggest abandoning vector embeddings. Rather, it provides a mathematical framework for understanding their inherent limitations. When building retrieval systems, consider:

• The relationship between your embedding dimension and collection size

• Whether your use case requires representing highly diverse relevance patterns

• Hybrid approaches that combine embeddings with other retrieval methods

• The specific trade-offs between dimension size and system performance

Vector embeddings remain powerful tools for semantic search, but recognizing their geometric limitations allows for more informed system design and realistic performance expectations. The key is understanding that these constraints exist not because of implementation choices, but because of the fundamental mathematics of high-dimensional spaces.