Your browser sucks, JavaScript disabled! Gunnar's console needs JavaScript and WebAssembly. Git over SSH on port 2222 does not.

nordisk/nornir

crates/mimir-ingest/src/lib.rs

//! # mimir-ingest — git as the change log, skade as the event log
//!
//! Mímir v2 lanes 4+5 (`PLAN-MIMIR-V2-2026-10-12.md` §3, §11, §12).
//!
//! ```text
//! events  = first-parent history of the tracked branch, per repo
//! offset  = last SHA consumed, per repo, stored in the log (consumer_offsets)
//! loop:   classify (docs-only → ledger row, no cargo)
//!         → checkout sha in the consumer's own clone (only changed paths)
//!         → dump (pluggable; a crate that does not compile is `partial`,
//!           never a block)
//!         → delta vs the open set → ONE skade atomic release
//!           (events + offset + ledger [+ universe manifest]) → offset = sha
//! ```
//!
//! The log is append-only CHANGE EVENTS (`+` opens/replaces a key's payload,
//! `-` closes it) because skade's iceberg commits APPEND/OVERWRITE only.
//! valid_from / valid_to are DERIVED by [`EventLog::fold`]: the state at
//! (repo, seq ≤ N) is, per key, the last event, open iff it is `+`. The open
//! set the delta runs against is that fold, answered from the READ-FAST
//! zone where a repo has one ([`readfast`]: interval tables + the event
//! tail after the anchor) and from the events elsewhere.
//!
//! Identity is SEMANTIC: node = id, edge = (from, to, kind) with a site
//! `count`; positions (node spans, edge sites) are their own table
//! `code_span_events`, so moved code writes span events only.
//!
//! TWO producer streams, never sharing a table (LAW 5, one writer per table):
//! the SOURCE stream (mimir-driver dumps → `code_node_events`,
//! `code_edge_events`, `code_span_events`) and the BINARY stream (mimir-binary
//! over every linked artifact, `--with-binary` → `bin_node_events`,
//! `bin_edge_events`, `code_symbol_events`, ledger `bin_ingest_runs`; owner =
//! the artifact `name@triple:profile`, ordered by a release ordinal `rev`).
//! With the binary layer one commit is still ONE atomic release covering both;
//! a binary-only re-ingest ([`Consumer::reingest_binaries`]) is its own.
//! Build Thing products (builder 2, [`product`]) ride the same binary tables
//! under the pseudo repo `@build-thing` ([`Consumer::ingest_products`]).
//! Provenance (source_only / binary_only / both) is never stored: it is
//! derived by [`EventLog::provenance`] (join on the moniker). The toolchain's
//! std / core / alloc enter the universe ring once per toolchain
//! ([`StdDumper`]).
//!
//! The payload schemas are not defined here: they come from `mimir-schema`
//! through [`PayloadSpec`] / [`Specs`]; this crate only adds the event prefix
//! (`repo · seq · sha · op · krate · payload_hash · key_dup`).

#[cfg(test)]
mod alloc_count;
pub mod artifact;
pub mod boundary;
mod atoms;
pub mod checkout;
pub mod compact;
pub mod consumer;
pub mod delta;
pub mod driver;
pub mod dump;
pub mod git;
pub mod log;
pub mod product;
pub mod provenance;
pub mod readfast;
pub mod ring_split;
pub mod schema;
pub mod std_dump;

pub use artifact::{FoundArtifact, JoinContext};
pub use compact::{CompactOptions, CompactReport};
pub use consumer::{CommitReport, Consumed, Consumer, DryDelta, KeyDelta, PhaseMs, RepoSpec, Run, Stopped, dry_delta};
pub use delta::{Kind, OpenSet, PayloadHash, RepoOpen, Stream};
pub use driver::{forget_without_mir, local_mirror_stems, BinaryLayer, DriverDumper, DriverHost, DumpProgress, FlagPolicy, ProgressSink, RepoFlags, RunnerHandle, SuperEntry, SuperOutcome, ToolCall, ToolRunner, artifact_key, graph_units, plan_waits, Wait, run_super, super_cover_of, DriverReady, DriverSpec, DriverUnavailable, DumpedUnit, LoadFailure, OutsideTheCommit, dated_sibling_plan, load_failure, pinned_toolchain, restore_siblings};
pub use std_dump::{StdDumper, StdOutcome, StdStatus};
pub use dump::{BinaryDump, CommitDump, CrateDump, DumpPhases, Dumper, DumperPool};
pub use log::{BackfillRow, BinOffset, BinRunRow, EventLog, FeedRow, FoldDelta, FoldScope, FoldTrack, FoldedRows, LakeHeld, LakeSchemaMismatch, LogWriter, Offset, RunRow, Scan, Stamp, Unevolvable, WriterSeatTaken};
pub use product::{ArtifactHit, ArtifactQuery, PRODUCTS_REPO, ProductArtifact, ProductFacts};
pub use provenance::ProvenanceView;
pub use readfast::{IntervalDiff, MaterializeReport, MaterializeSweep, MirPayloads};
pub use ring_split::{SplitOptions, SplitReport};
pub use schema::{
    PayloadSpec, Specs, code_bin_edges_spec, code_bin_nodes_spec, code_edges_spec, code_nodes_spec, code_spans_spec,
    code_symbols_spec,
};