.nornir/vann-design.md

vann — design

Class: design · Function: vann (the exact vector index). Strict: chosen shape only — the exact flat oracle + a provably-lossless int8/VNNI prefilter → f32 rerank. The removed IVF layer and why exact-only won are in stree-reasoning.md and reasoning.md.

Source: src/vann/mod.rs. Extracted from nornir's src/vector.

What it is

A pure vector-ANN engine that is, by contract, exact: VectorIndex is a brute-force ("flat") f32 nearest-neighbour oracle with 100 % recall. Its speed comes not from skipping candidates (it never does) but from a runtime-detected SIMD cosine kernel and a quantized prefilter that narrows the f32 rerank set without ever changing the answer.

Public surface (`vann/mod.rs`)

Item	Role
`VectorIndex::new(dim)` (`:59`)	build an index over `dim`-wide L2-normalized vectors
`add(&mut, vectors, ids)` (`:94`) / `remove(id)` (`:127`)	mutate the set
`search(query, k)` (`:154`)	exact top-`k` over f32 (the oracle)
`search_i8(query, k)` (`:207`)	int8/VNNI cosine scan (fast, approximate score)
`search_i8_rerank(query, k)` (`:266`)	int8 prefilter → f32 rerank; byte-identical to `search`
`search_auto(query, k)` (`:323`)	pick the kernel by availability
`search_batch(queries, k)` (`:353`)	many queries, prefetch overlapped across them
`write(path)` / `load(path)` (`:465`,`:482`)	round-trip through a file (fully resident; the int8 row blob is READ, not mapped — the `vann-mmap` flag that claimed otherwise gated nothing and was deleted 2026-08-22)

Kernels: score_i8_batch(query, rows, dim, row_sums) (:741) scores a run of int8 rows with vpdpbusd (64 int8 MACs/instruction); quantize_i8 (:636), normalized (:973), top_k (:1000), active_simd() (:518), vnni_available() (:724), i8_score_error_bound(dim) (:628), and bench_kernels (:865) → BenchReport.

The lossless int8 prefilter

Vectors are L2-normalized, so components live in [-1, 1] and quantize to i8 at ~1e-2 cosine error (i8_score_error_bound). search_i8 scores every row with the int8 kernel; search_i8_rerank uses those scores only to choose the candidate set, then rescores the candidates in exact f32 — so the returned top-k is byte-identical to the flat search. int8 quarters the bytes the scan streams (the throughput win); the f32 rerank restores exactness (the correctness guarantee). The int8 kernel borrows VNNI's unsigned×signed form via a +128 query bias and a per-row Σr correction.

Shared DNA with `stree`

vann and stree win the same way: a branchless SIMD inner loop over contiguous memory, and a software-pipelined batch that prefetches the next rows while scoring the current one (search_batch mirrors stree's batch_stream::<P>, and is a tested-identical overlap of single-query search). They stay two structures because the domains differ — ordered keys vs unordered vectors — which is exactly the exact-vs-approximate line drawn in stree-reasoning.md.

Verify

tests/vann_matrix.rs pins the rerank invariant (int8 rerank == flat f32) and the batch invariant (search_batch == looped search).