Vector search is a well-understood problem when every vector is eligible.
Now add a filter. In practice, most useful vector searches are scoped by factors the embedding does not encode: tenant, permissions, time, geography, document type, availability, or another business rule. Unfiltered nearest-neighbor search is important, but filtered retrieval is the typical pattern for many production queries.
Imagine an index containing one billion document embeddings. A user asks for the passages most relevant to a question, but only those within one customer’s contracts, in one jurisdiction, and created in the past year:
tenant_id = 1847
AND document_type = "contract"
AND jurisdiction = "EU"
AND created_at >= "2025-01-01"
Suppose the predicate eliminates 99.99% of the corpus. We are no longer looking for the nearest vectors in a billion-vector dataset. Instead, we are looking for the nearest vectors within a small, irregular subset scattered throughout that dataset.
The filter did not merely make the original problem smaller. It changed the problem.
That realization is why we built SABLE.
SABLE, the Selectivity-Aware Bi-Level Engine, is the retrieval engine underlying VectorAmp. We built it around two connected goals: to integrate metadata constraints into retrieval planning and to make billion-scale search practical without requiring a corpus-wide graph and raw vectors to remain resident in expensive RAM.
We have published headline figures for SABLE, including single-node comparisons against DiskANN at 100M vectors. What we have not yet published is a harness someone else can run. Benchmarking vector databases is unusually easy to do poorly, and a number nobody can reproduce is a number nobody should have to believe. This article is the architecture, not the evidence. The evidence package is next. In the meantime, this is where conventional approaches spend effort, why the right strategy varies with selectivity, and what we built in response.
Vector proximity and business eligibility are different geometries
Approximate nearest-neighbor indexes organize data by vector similarity. That is their purpose. Semantically similar contracts may be close together; unrelated documents may be far apart.
Metadata predicates describe a different structure. Tenant boundaries, dates, permissions, regions, product availability, case numbers, and document types do not necessarily align with neighborhoods in embedding space.
That mismatch matters. An ANN index can quickly lead us toward vectors close to the query while repeatedly landing on records that the filter deems ineligible. The more selective the predicate becomes, the less useful “nearby” is unless eligibility is part of the search plan.
Consider a request for 10 results when only 1 in 10,000 vectors is eligible. A system that retrieves globally relevant candidates and filters afterward may discard nearly all of them. Asking it to over-retrieve more candidates can recover some results, but the required expansion depends on the data distribution and the query. A fixed over-retrieval factor is not a solution; it is a gamble.
The difficult part is not evaluating the predicate. Modern systems can efficiently test metadata. The difficult part is avoiding irrelevant search work before it happens while still preserving good nearest-neighbor recall within the eligible population.
The three obvious strategies all have a place
There is no single foolish approach to filtered vector search. Each common strategy is reasonable in the context where it applies.
Search first, then filter
Post-filtering preserves the fast path of an existing ANN index: retrieve nearby candidates, apply the predicate, and return the survivors.
For broad filters, this can work well. If most candidates are eligible, little work is discarded. Under a highly selective predicate, however, the engine may inspect or retrieve a large candidate set to find a small number of usable results. It may also return fewer than the requested top-k if too few candidates survive.
Filter first, then search the survivors
At the other extreme, the engine can first identify all eligible records and compute distances only within that set. When the surviving population is small, an exact or compressed linear scan can be highly effective. It is simple, predictable, and avoids distance calculations for ineligible vectors.
But “small” is not a fixed threshold. A filter matching 500 vectors and one matching 50 million vectors should not trigger the same plan. The crossover depends on list sizes, hardware, compression, top-k, and the distribution of eligible records.
Carry a filter through graph traversal
Filter-aware graph methods carry eligibility into the search itself. Systems such as Qdrant can use payload indexes to add filter-aware connections, check predicates during HNSW traversal, estimate filter cardinality, and fall back to a direct scan when the eligible set is small. Newer traversal methods, such as ACORN, can explore beyond direct neighbors when strict or combined filters disrupt standard graph paths. These are substantial improvements over naive pre- and post-filtering and should not be dismissed.
They still face a structural tension: retrieval remains organized around graph navigation, while the predicate may carve out a sparse and irregular population. Preserving reachability can require additional edges, broader exploration, or an alternative fallback plan. These are valid engineering choices, but graph construction, connectivity, storage, and traversal still carry costs. At a very large scale, keeping graph structures hot can require substantial RAM. Moving graph state to disk lowers the fixed memory requirement, but traversal can become irregular I/O.
We did not conclude that graphs are bad. We concluded that graphs are one tool, and that a filtered query planner should not be forced to use the same tool across all selectivity regimes.
Selectivity is a query-planning signal
Database query planners have long relied on statistics to avoid unnecessary work. A predicate that matches half a table should not be executed the same way as one that matches only twelve rows.
Vector retrieval deserves the same treatment.
Selectivity is the fraction of records expected to satisfy a predicate. A filter that matches 50% of a corpus has a selectivity of 0.5. A filter that matches 0.01% has a selectivity of 0.0001.
Global selectivity is useful, but it is not enough. Eligible records are rarely distributed evenly. A jurisdiction filter may eliminate one partition entirely, retain nearly all of another, and leave a thin slice of a third. The useful question is therefore not only “How selective is this filter?” but also “How selective is it here?”
That led to SABLE’s central design decision: estimate selectivity at the inverted-list level, eliminate lists that cannot contribute, and select a search path for each surviving list.
How SABLE approaches the problem
SABLE is built on a compact, disk-oriented foundation: optimized product quantization, inverted-file partitioning, and product-quantized residual codes. In plain terms, it groups nearby vectors into lists and stores compressed representations to enable efficient comparison.
The differentiating behavior lies within and around those lists.
1. Put filter summaries beside the vectors they describe
Each IVF list includes its own metadata structures that describe its contents. These are not merely global dataset statistics. Categorical values are represented using compact, bitmap-based structures. Numeric fields include local distribution summaries and exact bounds suitable for range pre-gating.
When a query arrives, SABLE can ask inexpensive questions before computing vector distances:
- Can this list contain the requested category or tenant?
- Does its numeric range overlap the requested dates, prices, or scores?
- Roughly what fraction of this list is likely to survive the predicate?
If the answer to the first two questions is no, the list can be skipped. Hash-based categorical summaries are designed to be conservative: a collision may retain an unnecessary list, but it does not remove a list containing a valid match. Numeric min/max bounds provide the corresponding hard exclusion for ranges.
This is pre-gating. It moves the cheapest, safest elimination step ahead of distance computation.
2. Give each list more than one way to be searched
For dense or unfiltered work, SABLE can use a compact, navigable micrograph built within an individual inverted list. These NSG-like graphs accelerate list traversal without requiring a single global graph spanning the entire corpus.
For selective filtering or for lists where graph traversal is not advantageous, SABLE uses a linear asymmetric-distance-computation scan over the relevant compressed codes and applies the predicate at the vector level.
That may sound counterintuitive. “Linear scan” is often treated as synonymous with “slow.” But a linear scan over a small, pre-gated, contiguous, compressed candidate population can be exactly the right operation. It is cache-friendly and predictable, and it does not waste a traversal budget by wandering through ineligible graph nodes.
The point is not that linear search universally beats graph search. The point is that neither does.
3. Let the planner decide per list
The query planner makes that decision independently for each surviving list. It combines the list’s filter metadata, size, graph readiness, and estimated selectivity to choose the appropriate path. One query can prune some lists entirely, scan others, and use micrograph traversal when a local candidate population is dense enough to justify it.
Results from the surviving lists are merged into a global top-k candidate set. Compressed-distance candidates can then be reranked against exact vectors stored in a memory-mapped file.
Why the “bi-level” part matters
SABLE’s name refers to two related levels of organization.
At the coarse level, IVF partitions the vector space into lists. Filtering metadata at that level lets the engine avoid opening lists that cannot help answer the query.
At the fine level, each surviving list has its own retrieval decision and, when appropriate, its own proximity micrograph. This keeps the graph scope local and allows search behavior to follow the eligible population rather than imposing a single global traversal strategy.
This structure is designed for a disk-backed system, not merely compatible with one. Product-quantized codes, list data, partition-local graph adjacency, and exact reranking vectors can be memory-mapped from local NVMe storage. Compact centroids, list summaries, planner state, and the active working set remain hot. Large index regions are paged in only when a query needs them, so physical memory tracks the working set rather than the full corpus size.
That design avoids a false choice between keeping an enormous global graph hot in RAM and performing graph hops across cold pages. Pre-gating first narrows which lists can contribute. The planner then selects the local access pattern, and exact vectors are accessed only for the final reranking set.
SABLE adds a mutable L0 write buffer in front of the durable L1 index. New vectors are written through a write-ahead log, remain searchable in L0, and are asynchronously folded into the disk-backed structure. Queries combine results from both tiers. Micrographs can be rebuilt in the background; while a graph is unavailable, the corresponding list retains a linear-search fallback.
The result is not merely an ANN algorithm. It is a retrieval-and-storage architecture designed for live datasets: inserts, deletes, recovery, compaction, background construction, and exact reranking must coexist with queries.
What this design does and does not claim
SABLE is designed for workloads where vector similarity and structured scope are inseparable: multi-tenant retrieval, permissioned enterprise search, time- or region-bounded discovery, filtered recommendations, and retrieval-augmented generation over governed corpora.
If your workload consists of a few million unfiltered vectors that fit comfortably in memory, mature HNSW implementations are excellent. If a predicate leaves only a few hundred known candidates, a direct exact search may be all you need. If filters are broad and predictable, post-filtering may be perfectly adequate.
SABLE is designed for the less convenient middle and the large-scale end: restrictive or unpredictable predicates, uneven distributions, large corpora, and systems where keeping a global graph and raw vectors permanently resident becomes costly.
We believe the architecture is fast because it is designed to avoid unnecessary work: reject impossible lists early, use compact sequential scans where they fit, use navigation when it helps, and fetch exact vectors only for reranking. Our internal testing supports that, and the figures we have published reflect it. What we are not doing in this article is asking you to take those figures on faith.
Benchmarking vector databases is unusually easy to do poorly. Results vary with recall targets, dimensions, metadata distributions, filter selectivity, concurrency, cache state, index configuration, hardware, and whether all systems are allowed to be tuned comparably. A headline latency number without those details is mostly theater.
We are building a reproducible public benchmark package. When it is ready, we will publish the datasets, workloads, configurations, hardware, recall methodology, filter distributions, and raw results needed to reproduce and critique our conclusions.
Why we built it
We built VectorAmp to answer questions across large, messy, governed data collections. In those systems, metadata is not decorative, and filtered retrieval is not an edge case. Real applications almost always scope similarity by one of the following: identity, access, time, geography, content type, inventory, policy, or workflow state.
tenant_id is a security boundary. jurisdiction can be a legal boundary. created_at can determine a document’s relevance. inventory_status can determine a recommendation’s usefulness. A retrieval system that treats those fields as cleanup after semantic search is solving the wrong problem.
So, we stopped asking, “How do we add filters to a vector index?”
We asked a different question:
What would the index look like if the filter were part of retrieval from the beginning?
SABLE is our answer to both constraints: a selectivity-aware, bi-level engine that integrates list-level pre-gating, per-list path selection, compact micrographs, compressed vector search, NVMe-backed storage, bounded hot state, and exact reranking into a single system.
It is running beneath VectorAmp today. The next step is to make the evidence behind it as inspectable as the architecture itself.
Until then, the most important claim is also the simplest: when 99.99% of the data is filtered out, you should not search as if all of it still qualifies.
Explore SABLE or talk with our engineering team about your filtered retrieval workload.
