nordisk/nornir

Mímir v2 — tables and columns

Every column below is taken from crates/mimir-schema/src/lib.rs (the four payload tables), crates/mimir-schema/src/code_mir.rs (the three lossless tables of schema 8) and crates/mimir-ingest/src/schema.rs (the lake's event prefix and table names). utf8 is an Arrow string; "null" means nullable.

1. The driver's payload tables

code_nodes (*.nodes.arrow)

Node kinds (mimir_schema::kind):

crate mod fn method struct enum union trait trait_alias impl const static type macro
ffi foreign_type field variant param ret local generic assoc_type assoc_const
mono_instance ffi_import feature

plus the MIR part mir_body (one per fn), at level part. Its payload (the block fingerprints, terminator kinds and successors) is derived from the fn's code_mir rows. Since schema 8 the driver writes no mir_block node: a reader derives the blocks from the same rows (section 1.1).

code_edges (*.edges.arrow)

Edge kinds (mimir_schema::EdgeKind):

CONTAINS DEPENDS_ON CALLS IMPLEMENTS FOR_TYPE USES_TYPE READS_FIELD EXPANDS GATED_BY
TESTS TOUCHES_TABLE COMPATIBLE INSTANCE_OF ENABLES ENABLES_WEAK BUILT_FROM SYSCALLS CFG_SUCC
WRITES_FIELD

The order is EdgeKind::ALL: a new kind is appended (mimir-room keys its kind masks on the position), so WRITES_FIELD (schema 6) is index 18. CFG_SUCC keeps its position, but since schema 8 the driver writes no CFG_SUCC edge: the successors of a block are derived from a code_mir row (section 1.1).

The two field kinds, for a body owner b and a field f (a struct, union or enum-variant field):

READS_FIELD(b, f)   ⇔ b has an expression `e.f` (struct / union): a MENTION, read or written
WRITES_FIELD(b, f)  ⇔ f ∈ written(p) for a place p of b with  p = v | p op= v | &mut p | &raw mut p
                      | p auto-borrowed &mut (a `&mut self` method on p)
                    ∨ b has a literal `S { f: v, .. }` / `E::V { f: v, .. }`  (not the fields `..base` fills)
written(p)          = { g : `.g` on the path of p, outside in, up to the first deref }
count               = |{ (file, line) of the field expression }|      (as for every kind)

A read-only use is READS_FIELD(b, f) ∧ ¬WRITES_FIELD(b, f); a literal names a field without a field expression, so it is a WRITES_FIELD only. a.b.c = v writes c and b, and a too unless a is reached through a pointer; r.f = v with r: &mut S writes only f.

code_spans (*.spans.arrow)

code_symbols (*.symbols.arrow)

1.1 The lossless tables (schema 8)

Three more files per unit hold the MIR facts in ONE lossless form (crates/mimir-schema/src/code_mir.rs; the plan is exec-graph-plan.md). A unit may lack any of them (MIMIR_MIR=off, a dump of an older driver). A program's --exec unit has only these files and <stem>.exec.json.

The files are Arrow IPC with zstd buffers, at most 2,048 rows per record batch. A row holds its lists as Arrow List columns (a struct of arrays). A string column inside a row holds indices into the row's own pool strs (0xFFFF_FFFF = none). Every table ends with content.

code_mir, columns in schema order:

code_instances, columns in schema order:

code_data, columns in schema order:

The schema-7 view is derived, never stored. For a code_mir row with legacy not null, mimir_schema::code_mir::legacy gives:

view(row)         the non-cleanup blocks: name, fingerprint, terminator word, successors
body_payload      the `mir_body` node's `ty` (the driver writes it as the derived column it is)
block_nodes(row)  per block: id `<fn>bb<name>:`, the schema-7 `mir_block` payload, the CFG_SUCC successor ids

Every batch carries schema metadata mimir.schema = "8" and mimir.table = <table name>.

Versions (mimir_schema::SCHEMA_VERSION): 1 the four tables; 2 the visibility / doc columns, negated, partial, the feature kinds; 3 ignored, doc_hidden; 4 access, link_name; 5 stable identities; 6 WRITES_FIELD and every MIR block of a body written (no 1,024 cap); 7 REFERS_FN; 8 the three lossless tables, no mir_block node and no CFG_SUCC edge. Migration: a version that only adds nullable columns is evolved in place; a lake stamped before the re-read epoch (8, mimir-ingest REREAD_SCHEMA) is wiped by EventLog::open_lake and the consumer re-reads every repo from its tip, because the facts of every unit changed and a dependency already in the ring is never re-dumped. The wipe keeps no backup: the lake is derived data.

2. The lake (skade namespace mimir_v2)

An event table is a payload table with eight prefix columns in front:

Tables:

Table properties: mimir.schema (the mimir-schema version the columns are at) and the comma-joined list of columns added in place by an evolution.

The lossless tables in the lake (crates/mimir-ingest/src/lossless.rs):

• Identity. The key columns are made stable by mimir_schema::code_mir::stable_key: a moniker's version is *, and a <package>@<version>:: inside a def, type or instance key loses its version. A version bump re-emits a row on the same identity. The stored columns keep the real versions.
• Delta. payload_hash is the row's content. A row whose content did not change writes no event. base_line is outside content: moved code writes no event, and the stored base_line is the one of the last content change. The owner's span in code_span_events is the truth for the line.
• Storage. skade's Iceberg bridge maps no nested type. A list<T> column is stored as one binary column: the row's items back to back, an integer item as its little-endian bytes, a utf8 or binary item behind its length (u32, little endian). u32 is stored as long, u64 as decimal(20, 0), as in every event table. mimir_ingest::dump_form gives a scanned batch its Arrow lists back.
• Reading. No room folds these tables and the READ-FAST zone has no interval table for them. A reader asks for one owner: EventLog::code_mir, code_instances, code_data take a repo (with an optional seq) or one crate of the ring, and a byte cap. EventLog::mir_block_payloads gives the schema-7 block payloads of a repo, derived from its code_mir rows.
• Sizes. mimir-lake <lake-dir> stat prints files, bytes on disk and rows per table (EventLog::table_sizes).

Storage: Parquet, zstd level 3, 256 k-row row groups (log.rs write_props); every write of a release goes through one commit lock and one skade atomic commit.