nordisk/nornir

ragnars_emb — Ragnar-style IVF over the int8 vector path (reuse map + bench)

⛔ DECISION 2026-07-02 — REJECTED for live search: exactness is required.

nornir vector search must return exact top-k. Both approximate ANN options are out:

• Ragnar IVF (ragnars_emb / znippy-zoomies::vann) — recall < 1.0. Kept as a bench/experiment only; never wired into the live search path.
• CozoDB HNSW (the G6 spike, src/index/cozo.rs + the cozo-index feature) — removed 2026-07-02. At nornir's real corpus size (~n=4000) exact brute-force was ~20× faster and exact, so HNSW's approximation bought nothing; the only remaining justification was Cozo's unified Datalog query, which does not outweigh a second embedded engine + the airgap-binary cost.

The exact oracle is always kept: flat brute-force f32 (VectorIndex), with an optional int8 prefilter → exact f32 rerank (fast and exact top-k). The material below is retained as the historical IVF design/bench record.

Update (post-decision): a vendored, rayon-free HNSW ANN (znippy_zoomies::vann) has since landed as a scale-gated live dense path — built only when a snapshot is ≥ HNSW_MIN_ROWS (4096) and NORNIR_VECTOR_HNSW isn't disabled, with the flat oracle retained (and A/B-able) beneath it. That reintroduces HNSW at scale specifically — it does not revive the rejected CozoDB engine or the IVF path; see vector.md.

Status (REMOVED — see decision above): the IVF prototype was removed from the tree. There is no src/vector/ragnars_emb.rs, no RagnarsEmbIndex type, and no nornir vector ragnar-bench CLI — grep confirms all three are absent. The int8 SIMD kernels and the ANN/bench_kernels sweep it was built on now live in the external znippy_zoomies::vann crate (see src/vector/mod.rs), not in nornir. The live search path is exact flat brute-force (VectorIndex) with the optional int8 prefilter → exact f32 rerank. Everything below is retained only as the historical IVF design/bench record; the module/type/CLI names it uses no longer exist.

This is the sub-linear companion to G2 (.nornir/vector-cpu-simd.md): G2 made the per-vector dot product fast (SIMD + int8 VNNI); ragnars_emb makes the search stop touching every vector. It reuses G2's kernels verbatim — it is a layout + traversal change, not a new numeric core.

The progression (flat-today → batched → IVF → placement)

1. Flat brute-force, today (VectorIndex, the baseline + ORACLE)

src/vector/mod.rs scores EVERY stored vector per query — O(N·dim), exact, 100 % recall. Perfect for small corpora and it is the exact oracle every approximate path is graded against. It does not scale: at 1 M × 384 every query streams ~370 MB of int8. ragnars_emb keeps VectorIndex unchanged as baseline/oracle.

2. Batched multi-query search (the throughput lever)

RagnarsEmbIndex::search_batch(queries, k, nprobe) processes M queries together and software-prefetches the next posting-list rows (_mm_prefetch, the same hint stree32::batch_stream uses) so the memory-bound int8 scans overlap instead of each query serially stalling on cache misses. It mirrors the pipelined overlap of knowledge::query's Ragnar batch_stream, applied to the cosine scan rather than an exact-key tree walk.

Correctness contract: search_batch is byte-identical to looping search_np — prefetch changes only when bytes arrive, never which bytes are scored, and the scoring (score_i8_batch) + top_k merge are the same calls in the same order. Asserted by search_batch_is_byte_identical_to_single and in every bench cell (batch== column).

3. IVF (inverted-file) — the structural, sub-linear win

Build clusters the corpus into nlist centroids with k-means++ (√N default nlist), assigns each vector to its nearest centroid, and stores rows in cluster (CSR) order so each posting list is one contiguous i8 slice. Search:

1. score the query vs the nlist centroids (score_i8_batch, small),
2. take the nprobe nearest centroids,
3. scan ONLY those clusters' contiguous rows (score_i8_batch again),
4. top_k merge.

That turns O(N·dim) into ≈ O((nlist + nprobe·N/nlist)·dim). nprobe is the recall/speed knob: recall rises with nprobe, measured against the flat oracle.

Reuse, verbatim. No dot product is reimplemented. Rows and centroids are quantized with super::quantize_i8 (the exact G2 quantizer, made pub(crate)); every cosine — k-means seeding distances, Lloyd assignment, the centroid pre-scan, and the posting scan — runs through super::score_i8_batch (AVX-512 VNNI → scalar). The merge reuses super::top_k. ragnars_emb adds structure, not arithmetic.

Determinism. k-means is seeded (a dependency-free SplitMix64 fed a u64) and runs a FIXED Lloyd-iteration count — no wall-clock convergence test, no rand. Two builds with the same seed + inputs are byte-identical (write/to_bytes round-trip proves it: seeded_build_is_deterministic). Only the bench reads the wall clock, and never inside an assertion.

4. Memory placement (resident → mmap → NUMA seam)

• Resident + CSR layout. The int8 blob + CSR postings stay resident; rows are laid out in cluster order so a posting scan is a single sequential, cache-line aligned stream straight into score_i8_batch — same LRU-resident pattern as the flat store::reconstruct cache.
• mmap (feature vector-mmap). RagnarsEmbIndex::load_mmap memory-maps the dominant n × dim int8 row blob (written LAST in the file for a clean tail) so the OS page-cache backs large corpora and multiple processes share one physical copy. The Blob enum (Owned(Vec<i8>) | Mapped(Mmap)) derefs to &[i8], so the scan path is identical resident-or-mapped. memmap2 is already a hard dep (DWARF/artifact readers), so the feature adds no new dependency. Proven by mmap_load_matches_resident.
• NUMA — honest seam, not yet wired. First-touch / mbind placement of the mapped blob per socket is the next lever for a multi-socket bench host; the mmap split is the hook it would attach to. Called out here so it is not mistaken for done.

Where the Ragnar stree genuinely reappears (IVF? no — hybrid/term-dict yes)

The stree32 (Ragnar's sorted-key Eytzinger S+tree, already reused in knowledge::query for batched call-graph lookups) is an exact-key structure. Pure IVF does not want it: the centroid nearest-search is a tiny brute-force scan, and the CSR posting index is already O(1) — bolting an exact-key tree onto either would add nothing. Its real slot is a future hybrid / lexical layer: an exact term → posting dictionary (BM25-style keyword postings, or an exact id → row map) fused with the IVF ANN so one batched query mixes lexical + semantic recall. That is exactly the stree32::batch_stream shape (hash the term batch, route through the tree with prefetch, cross-check by string) — documented as the natural next slot, deliberately NOT forced onto pure IVF.

Tie to G6 (CozoDB / HNSW go/no-go)

G6 (.nornir/nordisk_plan.md) asks whether to adopt CozoDB's built-in HNSW as the ANN, or keep a hand-written Rust index. ragnars_emb is the "keep it in Rust" data point: IVF-over-our-own-int8-kernels, no new engine, no C/C++ backend, one airgapped binary. This bench (IVF crossover vs flat) is the apples-to-apples baseline the HNSW eval must beat to justify a second embedded engine + dep weight. IVF and HNSW are both approximate ANN with a recall/speed knob; if IVF hits the recall target at the corpus sizes nornir actually indexes, the "adopt Cozo HNSW" case rests on the unified Datalog semantic+structural query, not raw ANN speed.

API surface

• RagnarsEmbIndex::build(dim, vectors, ids, &RagnarsParams) -> Result<Self>
• search(query, k) / search_np(query, k, nprobe) — top-k (id, cosine)
• search_batch(queries, k, nprobe) — byte-identical to the single-query loop
• write / to_bytes / load / load_mmap (feature vector-mmap)
• RagnarsParams { nlist, nprobe, kmeans_iters, seed } — nlist=0⇒√N, nprobe=0⇒nlist/16
• Bench: ragnars_emb::bench(sizes, dim, k, nprobe_sweep, n_queries, seed) -> BenchReport (BenchReport::to_markdown, SizeReport::knee(0.95), crossover_n)

Selection (flat vs ragnars_emb) is by corpus size or an explicit flag; the flat VectorIndex stays the exact oracle + small-corpus default.

Tests (inject / assert — real recall + values, seeded)

Behind --features testmatrix the recall / batch-identity / crossover tests emit nornir_testmatrix::functional_status("nornir-ragnars-emb", "<check>", ok, detail) for ivf_recall, batch_matches_single, crossover_measured.

Bench results

nornir vector ragnar-bench on a deterministic clustered synthetic corpus (~k points per tight, well-separated cluster → a stable top-k SET, so recall is a real measurement, not an int8 near-tie coin-flip). Flat = exact f32 oracle/baseline; IVF = ragnars_emb at the ≥0.95-recall operating point.

Release build, dim 384, k 10, 200 queries/cell, AVX-512 VNNI box (nornir vector ragnar-bench --sizes 10000,100000,1000000 --dim 384):

At N=1 M the recall/speed knob is visible: nprobe 1→2→4 gives recall 0.988 → 0.995 → 1.000 (23.8 → 37.6 → 68.4 µs p50), so ≥0.99 costs one extra probe. Every cell's batched search_batch was byte-identical to the single-query loop.

Crossover. IVF (≥0.95 recall) beats flat p50 from N ≈ 150 upward (--sizes 20,50,100,150): N=100 → 0.91× (flat wins, IVF's fixed ~3.2 µs centroid-scan floor dominates and there are too few clusters to hit 0.95 recall); N=150 → 1.28×; N=200 → 1.5×; N=1 000 → 6.5×; and it widens to 3–4 orders of magnitude by 1 M. So the flat VectorIndex stays the right default only for tiny corpora (≲100 vectors); everywhere nornir actually indexes, IVF wins big.

k-means build cost is O(nlist·N·dim·iters) — the 97 s at 1 M (nlist 1000, 10 iters) is a one-time index build, amortized over every subsequent query, and is itself a candidate for the same fork-join parallelism the flat scan already uses (an honest follow-up seam).

Honest seams / caveats.

• k-means quality. Fixed Lloyd iteration count (determinism > convergence); empty clusters keep their prior centroid. Good enough for the bench; a real build could add k-means++ restarts or balanced assignment.
• int8 recall ceiling. On dense clusters (many near-identical points) int8 reordering caps recall below f32 — inherent to an int8 index. The bench uses tight, separated clusters where the top-k SET is stable; a future exact-f32 re-rank of the IVF candidate set would lift recall on dense data.
• mmap / NUMA. mmap is wired + tested; NUMA first-touch is a documented seam.
• 10 M. Skipped unless RAM allows (a 10 M × 384 int8 blob ≈ 3.6 GB resident); run nornir vector ragnar-bench --sizes 10000000 on the bench host (loki) where the mmap path + 4 TB scratch make it cheap.