Mímir macro lenses: a plan
Status: M6, M1 and M7 are built (lane MACRO-LENS, 2026-11-05), with the §5b feedback loop and the architecture-report/1 output: cargo lens reflexion, arch-recover, refactor-price and arch-report, documented in lenses.md §4.27–4.30. M2, M3, M4 and M5 are built (lane MACRO-LENS-2, 2026-11-06: cargo lens layer-stack §4.31, M7 prices β with it; seam-interface §4.32; concept-scatter §4.33; ports-adapters §4.34, the §5b template), and arch-report takes its cuts from M2, its seams from M3 and a concepts section from M4, each priced by M7 (§4.30). A Gunnar target spec is a test fixture (tests/golden/gunnar-architecture.ron). How to write a spec and read the report: lens-guide.md §6–7. The weights were measured for sensitivity and are kept fixed (§5c, lane MACRO-SENS). The owner picks what gets built next. Everything named as existing (dsm, misplaced, cycles, chokepoints, reach-set, the room, the matrices) is on nornir main and described in lenses.md, README.md and compiler-to-graph.md.
0. Why
The lenses built so far mostly find micro structure. They point at one function, one trait, one enum or one struct: a 4,071-line router, a trait default that only errors, a struct whose fields are mostly Option, two enums with the same variants.
The goal for Holger and Gunnar is macro structure: what the program should look like as a whole. That means which components it has, which direction their dependencies may run, where the interfaces sit, and which component owns each concept. The Holger architecture review (the "Fable report") asked exactly those questions, and answered them by reading. This plan makes the code graph answer them as well, and compute the gap between the code and the shape we want, so the refactoring plan comes from the graph.
What exists already on the macro side only describes the shape:
| lens | what it says on Holger (h32/next 9dbe5cbf) | what it does not say |
dsm |
propagation cost of server-lib 58.7 %; core of 97 of 171 modules | which edges to cut to bring it down |
misplaced |
Leiden communities Q 0.83, against 0.59 for modules and 0.46 for crates; 407 misplaced fns | the layout to move to, beyond single functions |
cycles |
the strongly connected knots | the smallest cut that untangles them |
chokepoints |
the narrow passages | which interface belongs there |
The seven lenses below fill those gaps. M6 + M1 + M7 are the core, and the others refine.
1. Inputs and notation
From one dump the room already holds one sparse boolean or count matrix per edge kind (see compiler-to-graph.md):
A_calls,A_uses(USES_TYPE),A_reads(READS_FIELD),A_writes(WRITES_FIELD),A_impl(IMPLEMENTS),A_contains(CONTAINS),A_touch(TOUCHES_TABLE),A_sys(SYSCALLS);nnodes, with ids as SCIP monikers, and for each node its crate, its module path, its file and its span.
Notation:
sym(A) = A ∨ Aᵀ(LOR.LAND), orA + Aᵀ(PLUS.TIMES) when counts matter.- A partition of the nodes into
cgroups is ann × cboolean membership matrixP, with one 1 per row. - Cross-group counts:
B = Pᵀ · A · P(PLUS.TIMES,c × c).B[i][j]is the number of edges from groupito groupj. - Transitive closure:
A* = I ∨ A ∨ A² ∨ …, computed by repeated squaring over LOR.LAND until it no longer changes (dsmdoes this today).
Scope: "hand-written production code", the same scope as the micro lenses. That excludes tests, generated/ and derive-written impls. Dependencies stay in the graph as fixed external nodes, so edges into them count but they are never moved.
2. The lenses
M1. architecture-recovery: the layout the code wants
Question: if we grouped the code purely by how it talks to itself, which crates and modules would we get, and how far is that from today's layout?
Steps:
- Unit graph. Merge every type with its inherent methods and its fields into one "unit", using CONTAINS:
Cisn × uand maps each node to its unit. Weighted unit adjacency:
W = Cᵀ · (Σ_k w_k · sym(A_k)) · C (PLUS.TIMES)
Starting weights, which are tuned on the Holger and Gunnar goldens: calls 1.0, uses 0.5, reads/writes 0.7, implements 1.5, touch 0.3.
- Leiden at two resolutions γ (the code
misplacedalready has), with modularity
Q_γ = (1 / 2m) · Σ_ij (W_ij − γ · k_i · k_j / 2m) · δ(c_i, c_j)
γ_crate gives crate-sized groups and γ_module gives module-sized groups nested inside them. The γ values are picked from a sweep where the number of groups is stable.
-
Stability. Run 10 seeds and keep the consensus partition. Report the adjusted Rand index between seeds. A group that changes between seeds is marked "soft" and is never proposed as a crate.
-
Comparison with today.
- MoJoFM between the proposed partition and the current crate and module partitions.
- The overlap matrix
O = P_proposedᵀ · P_current(PLUS.TIMES). - A minimum-move assignment over
O(the Hungarian algorithm), which gives the move list: for each unit, from module X to group Y.
Output: the proposed crates and modules (each named after its heaviest existing module), the move list sorted by weight moved, Q for today and for the proposal, and MoJoFM.
M2. layer-stack: the dependency direction
Question: can the code be stacked into layers where every dependency points down, and which edges stop that?
Steps:
- At unit or module level, collapse the strongly connected components (Tarjan, already in
lenses/order.rs) into a DAGDwhose edges are weighted byW. - Minimum weighted feedback arc set inside each SCC. The exact problem is NP-hard, so use the Eades–Lin–Smyth greedy order, followed by local improvement. The edges that point backwards in the found order are the cut list.
- Layers: the longest-path depth in the acyclic graph that remains. Layer 0 holds the nodes with no outgoing edges inside scope (pure leaves: types and helpers).
-
Score:
- the back-edge share
β = Σ w(cut) / Σ w(all); - the number of layers;
- for each cut edge: its file:line call sites, and the layer it jumps from and to.
- the back-edge share
Output: "cut these N edges (weight w, β = x %) and the code is L layers", with the layer of each module, and the cut list sorted by weight.
M3. seam-interface: which trait is missing between two groups
Question: where do two components talk through many concrete calls, which means an interface (a trait) is missing between them?
Steps:
-
Take the groups from M1 (or the components from M6), as membership
P. -
Cross-calls
B = Pᵀ · A_calls · P. For each pair(i, j)withB[i][j] ≥ k_min, collect the callee setS_ij: the functions injcalled fromi. -
Narrowness:
ν_ij = |S_ij| / B[i][j]. A small set called often means a narrow, stable seam, which is a good trait.- The share of
S_ijthat is already behind a trait, viaA_impl.
-
If
ν_ijis low,|S_ij| ≥ 2and less than half ofS_ijis behind a trait, proposetrait <j>Port { S_ij }. The signatures come from the H11 envelope code. Also list the callers inithat would hold thedyn <j>Portor a genericT: <j>Port.
Output: the proposed traits, each with its methods, the callers and the number of concrete call sites it would replace.
M4. concept-scatter: who owns a concept
Question: which concepts (a type family, an enum family) are touched by many components, so that no component owns them?
Steps:
-
A concept
κis a node set: an enum and its variants, a struct and its impl, or a taxonomy family from H9 (for example the four "format" enums in Holger). -
Touch counts per group:
t_κ = Pᵀ · (A_uses + A_reads + A_writes)ᵀ · 1_κ(a vector of length c). -
Scatter:
p = t_κ / Σ t_κ;- the entropy
H(κ) = −Σ p_i · log₂ p_i; - the number of groups touching it,
g(κ) = |{i : t_κ[i] > 0}|.
-
The proposed owner is
argmax_i t_κ[i]. Every other touch is either a use through the owner's interface (fine) or knowledge leaking out of the owner (listed).
Output: concepts ranked by H. For Holger we expect ArtifactFormat and its siblings at the top, with the owner the format crates should be.
M5. ports-and-adapters: where domain code does I/O
Question: is the I/O (network, files, processes, database, syscalls) at the edges, or mixed into the domain logic?
Steps:
-
Seed I/O nodes: functions with a row in
A_sysorA_touch, or a call into a known I/O API set (std::fs,std::net,std::process,tokio::net/fs/process,hyper,reqwest,gix). This uses the B10 reach-set engine. -
Classify each unit:
- adapter: its I/O is direct, and it is called from outside its group;
- domain: it is reached from entry points and has no direct I/O;
- glue: the rest.
-
Violation: a unit in a group whose share of domain units is ≥ 70 % that does direct I/O. Score per group:
io_share = |io units| / |units|.
Output: the groups with their adapter, domain and glue shares, and the violations with file:line, ranked by how many domain callers sit above each one.
M6. reflexion: conformance to a target architecture (the core lens)
Question: given the architecture we want, where does the code agree with it, where does it break it, and what is missing?
This is the reflexion model of Murphy, Notkin and Sullivan, on our matrices.
- The target model, written by the owner or taken from a review's "as it should be" diagram, as a small RON file next to the repo:
(
components: [
(name: "formats", map: ["crate:holger-znippy-*", "mod:server/lib/src/formats/**"]),
(name: "intake", map: ["mod:server/lib/src/promote/**", "mod:server/lib/src/pull_intake/**"]),
(name: "gate", map: ["fqn:*::IntakeChain*", "mod:server/lib/src/admitted/**"]),
(name: "store", map: ["crate:holger-storage*"]),
(name: "http", map: ["mod:server/lib/src/exposed/**"]),
],
allowed: [
("http", "intake"), ("intake", "gate"), ("gate", "formats"),
("gate", "store"), ("formats", "store"),
],
expected: [ ("intake", "gate") ], // MUST exist: an absence is a finding
)
-
Map
M(n × h): each node goes to the first component whose pattern matches. Unmapped nodes are reported, never dropped. -
Implied component graph:
R = Mᵀ · A · M(PLUS.TIMES), one matrix per edge kind or summed over them. -
The three results:
- convergence:
R[a][b] > 0and(a, b)is allowed; - divergence:
R[a][b] > 0and(a, b)is not allowed. Each one is listed with its concrete edges (file:line); - absence:
(a, b)is expected andR[a][b] = 0.
- convergence:
-
Score: the conformance
κ = Σ w(convergent) / (Σ w(convergent) + Σ w(divergent)), plus the number of absences.
Why it is the core: the micro lenses tell us what is wrong in one place. M6 tells us how far the whole program is from the shape we chose, and every divergence is a concrete edge to remove or redirect. With M7 the same model prices each step.
M7. refactor-price: the value of a refactoring before doing it
Question: if we did refactoring row R2 (or any move list), how much would the structure improve?
Steps:
-
A move list (from M1, M3, a review's §6 row, or written by hand) changes the memberships (
P,M). A "cut" removes edges, and an "interface" turns direct edges into edges to and from one trait node. -
Recompute the following on the edited graph, without touching the code:
- propagation cost
PC = (Σ_ij A*_ij) / n², as indsm; - the SCC count and size;
- the M2 back-edge share β;
- the M1 modularity Q;
- the M6 conformance κ and its divergences.
- propagation cost
-
The delta table for each row is what it costs (the units moved, the call sites changed) against what it gains (ΔPC, Δβ, Δκ, the divergences closed).
Output: the refactoring rows ranked by measured gain per changed call site. This replaces ranking them by gut feeling.
3. Dataflow
mimir dump (Arrow) ──▶ room: A_calls, A_uses, A_reads, A_writes, A_impl, A_contains, A_touch, A_sys
│
├─▶ C (CONTAINS) ─▶ W = Cᵀ (Σ w_k sym A_k) C ─▶ M1 Leiden ─▶ P_proposed, move list
│ │
target model (RON) ──────────┼─▶ M (component map) ─▶ R = Mᵀ A M ─▶ M6 reflexion ─▶ conv / div / absence, κ
│ │
├─▶ SCC (Tarjan) ─▶ DAG ─▶ M2 feedback arc set ─▶ layers, cut list, β
│
├─▶ B = Pᵀ A_calls P ─▶ M3 seams ─▶ proposed traits (signatures via H11)
├─▶ t_κ = Pᵀ (A_uses+A_reads+A_writes)ᵀ 1_κ ─▶ M4 scatter H(κ), owner
└─▶ B10 reach (I/O seeds) ─▶ M5 adapter / domain / glue
│
move list / cut / interface ─▶ edited graph ─▶ M7 recompute PC, β, Q, κ ─▶ ranked refactoring rows
Every step is a sparse matrix product or a graph walk on matrices the room already holds. There is no second graph engine (no petgraph) and no rayon.
4. A worked example (constructed)
Six units, with calls:
http ─▶ intake ─▶ gate ─▶ store
http ─▶ store (the router writes the store directly)
intake ─▶ formats ─▶ intake (a cycle: formats calls back into intake)
Target model: allowed http→intake, intake→gate, gate→formats, gate→store; expected intake→gate.
-
M6:
Rhas the edgeshttp→intake,intake→gate,gate→store,http→store,intake→formatsandformats→intake.- Convergent: 3 (
http→intake,intake→gate,gate→store). - Divergent: 3 (
http→store,intake→formats,formats→intake). - Absences: none.
- With unit weights,
κ = 3 / 6 = 0.50.
-
M2: one SCC
{intake, formats}. The cheapest cut isformats→intake(weight 1), so β = 1/7. Layers after the cut: store 0, formats 0, gate 1, intake 2, http 3. -
M7, row "route http→store through intake":
- The divergence
http→storecloses, and κ rises to 3/5 = 0.60. - Cutting
formats→intakeas well removes the SCC, so PC drops. - Making
intake→formatsgo throughgategives κ = 4/4 = 1.0. - The table ranks these three changes by Δκ per changed call site.
- The divergence
5. Order of work, and acceptance
| step | lens | why in this order | acceptance |
| 1 | M6 reflexion | the core; it needs only a target model and the room | on Holger, with the target model written from the review's "as it should be" diagram, the divergences include the review's R1 (raw paths around IntakeChain) and R2 (format logic in server/lib), with file:line |
| 2 | M1 recovery | proposes the shape when nobody wrote one | stable over seeds (ARI ≥ 0.8 on the crate level); on Holger its format group matches the review's proposed format crate |
| 3 | M7 price | turns M1 and M6 into a ranked plan | ranks the Holger review's §6 rows; the top 3 agree with the review or the difference is explained |
| 4 | M2 layers | dependency direction | cut list on Holger server-lib with β, checked by hand on 10 edges |
| 5 | M3 seams | the missing traits | proposes the review's Format write/pull trait from the server/lib → format-crate seam |
| 6 | M4 scatter | concept ownership | ArtifactFormat and its siblings rank in the top 5 on Holger |
| 7 | M5 ports | I/O at the edges | violations checked by hand on 10 rows |
For every lens, as for the micro ones:
- a fixture test, written first;
- a Holger golden row and a Gunnar golden row. The Gunnar architecture review is being written now and is the second target;
- a docs section in
lenses.mdwith the formulas, the semiring of each step, a dataflow picture and a worked example; - a bench of wall time and peak RSS on Holger and Gunnar.
5b. Review round 1 (Gemini, 2026-11-05): the engine M6 + M1 + M7
The review agrees that M6 + M1 + M7 is the core. Bottom-up Leiden finds where code naturally clusters, but it cannot name a cluster or know what it is meant for. Top-down layer checks know what is forbidden, but not where the code wants to live. Run together, they close the loop: Mímir proposes a structure instead of only reporting debt. M2, M3, M4 and M5 become outputs or special cases of that one engine. The decisions taken from the review follow.
The reflexion core as masks
With A the composite multi-edge graph, M ∈ 𝔹^{n×k} the mapping and T ∈ 𝔹^{k×k} the target's allowed edges:
H = Mᵀ · A · M(PLUS.TIMES,k × k): the dependency volume between components.- Convergences:
H ∧ T, i.e.H⟨T⟩. - Divergences:
H⟨¬T⟩. - Absences:
T_expected⟨¬H⟩.
These are plain masked operations in SuiteSparse, with no new kernel.
M5 is a target model, not a separate algorithm. It is the standard three-part target: driving adapters → domain ← driven adapters. Domain → I/O sink is not in T, so every such edge appears in H⟨¬T⟩. M5 is therefore built as a shipped target template, plus the I/O seed set from B10.
Seam extraction as a cut (M3)
The concrete calls from component a to component b are
A_{a→b} = diag(M[:, a]) · A_calls · diag(M[:, b])
The column projection of A_{a→b} (the columns with any entry) gives the exact set of functions called across the boundary. Their signatures (via H11) are the candidate trait methods, so the trait is read off the cut edges instead of being invented by hand.
The target spec (.nornir/architecture.ron)
The spec is kept minimal: ordered layers, components mapped by globs, explicit allow or forbid rules, and the expected edges.
(
layers: ["exposed", "promote", "domain", "storage"], // top to bottom
components: {
"exposed": ["mod:holger_server_lib::exposed::**"],
"promote": ["mod:holger_server_lib::promote::**", "mod:holger_server_lib::pull_intake::**"],
"domain": ["crate:holger_traits", "mod:holger_server_lib::identity::**"],
"storage": ["crate:holger-znippy-*", "mod:server::repository::**"],
},
allow: [], // empty = derived from `layers`
forbidden: [("domain", "storage")], // the domain never talks to a concrete repository
expected: [("promote", "domain")],
)
The rules are read in order:
layers: a component may depend only on the layers below it. That gives the defaultT, an upper-triangular matrix over the layer order. A layer entry with*is a glob over component names ("svc-*"): the components it matches share that one layer and may not use each other.allow: adds edges toT.forbidden: removes edges fromT, and always wins overallow.expected: sets the absence check.- Edge exceptions (built 2026-11-06, lane MACRO-LENS-2, asked by the Holger spec):
allow_edges: [("fqn:<from>", "fqn:<to>")]excepts an item edge whose source matches the first fqn glob and target the second, andvia: ["fqn:<glob>"]excepts every edge whose source item matches (a decorator's forwarding methods, e.g.fqn:*[Decorator]*). An excepted edge counts as convergent whatever its pair rule says (an explicit edge exception beatsforbiddenandlayers), and is listed asexceptedwith its file:line. Onlyfqn:globs (or bare fqn globs) are allowed there.
A lower layer that calls an upper one is a divergence and also an M2 back edge.
How the mapping M is declared
The review's open question asked: globs in the spec, or components inferred from Cargo crates? Decision: both, in layers.
- Inferred by default. With no spec, every crate is a component and the layers come from the crate dependency DAG (the Cargo graph). That runs on any repo with zero setup.
- Globs override. A spec maps nodes by
crate:,mod:(module path, with*and**) orfqn:globs. The most specific match wins:fqnovermodovercrate, and the longer glob within a kind. - Leiden suggests the rest. A node no glob maps, in a crate the spec names more than one component for, takes the component that most of its Leiden community maps to. It is marked
inferredin the report, so the owner can pin it in the spec. - Unmapped nodes are listed, never dropped.
The spec lives in each repo, so CI can check it the way the goldens are checked. Open question 3 below becomes "yes".
The Leiden + reflexion feedback loop
For each Leiden community L (from M1) and each component c, the alignment is α(L, c) = |L ∩ c| / |L|.
If α(L, c) ≥ 0.85 for some c, every member of L that maps elsewhere is a candidate relocation to c. For example, 85 % of a community is in promote, but 3 of its functions live in exposed/http.rs. Each candidate is priced by M7 before it is proposed, and is kept only if it does not increase the divergences.
Cuts between layers (M2)
M2 uses the Eades–Lin–Smyth order on the condensation graph, weighted by W, followed by local improvement. Exact ILP is worth it only on SCCs under about 50 nodes (open question 6). The cut edges become the plan's "cut/seam" rows. Where several cut edges share a target set, M3 proposes one trait for them instead of N cuts.
What-if pricing (M7)
A proposal is a re-indexed M′, or a masked A′ = A⟨¬Cuts⟩, or both. Nothing touches git or cargo. Re-evaluating means:
ΔPC = PC(A′) − PC(A), using the transitive closure as indsm;ΔQ = Q(A′, P_proposed) − Q(A, P_current);ΔSCC = |SCC(A′)| − |SCC(A)|;Δκ, the change in the divergence count.
Closure and dsm run in under a second at 10k–15k nodes, so a full plan of dozens of rows is priced in seconds.
The output contract (one report per run, JSON plus text)
architecture-report/1
score: PC 58.7 % → 34.0 % (Δ −24.7), SCC 14 → 2, Q 0.59 → 0.81, divergences 212 → 31, conformance κ 0.62 → 0.95
moves: [ { symbol, from: module, to: component/module, weight, reason: leiden|cut|scatter } ]
seams: [ { from_component, to_component, call_edges, proposed_trait, methods: [signature], callers } ]
cuts: [ { edge: caller → callee, file:line, from_layer, to_layer, weight } ]
violations: [ { rule: forbidden|layer|absence, edge, file:line } ]
unmapped: [ symbol ] inferred: [ { symbol, component, alpha } ]
rows: [ the moves, seams and cuts grouped into refactoring rows R1..Rn, ranked by gain per changed call site ]
Each row carries its ΔPC, ΔSCC and Δκ, so a refactoring "earns its ticket before anyone opens an editor".
5c. Review round 2 (Gemini): weights fixed, sensitivity
The second review asked us to keep the macro-lens edge weights as they are and to show that they sit in a stable basin, rather than fitting them. The weights are calls 10, uses 5, reads 7, writes 7, implements 15, ×10 integers, plus the call-site cap of 20 (arch_recover.rs, ArchRecoverArgs::default). Lane MACRO-SENS (2026-11-06) measured this and changed no lens code. The raw numbers are in the lane's handback (plan/handback/macro-sens.md).
Where the weights enter
Only M1 reads the kind weights: arch_recover::units builds W = Cᵀ · (Σ_k w_k · sym(A_k)) · C with w_k · min(sites, cap) per item edge. The other lenses use plain counts:
- M6 (reflexion,
H = Mᵀ·A·M), M2 (layer-stack, the cut weights and β) and M7 (refactor-price, closed weight per changed site) all use the raw site counts: call sites for CALLS and dump edges for the other kinds. They use no kind weights and no cap. reflexion::sym_graphcaps at 20 but has no kind weights. It feeds only the inferred mapping (0 inferred items on both targets) and M7's Q column, which does not rank.- M5 (
ports_adapters) has the M1 weights hard-coded, butarch-reportdoes not call it.
So in arch-report, the weights reach the ranking only through M1's consensus partition. That partition is what the loop rows (L<n>, the §5b feedback loop) are built from. M6 and M2 cannot move, and M7 can move only through its L rows.
Method
-
Targets: Holger h32 (
dumps/holger-s6) withtests/golden/holger-architecture.ronandholger-review-rows.json; Gunnar c17a6ef4 (gunnar-default) withgunnar-architecture.ronandgunnar-review-rows.json. -
Runs: full
arch-report(top 0), 10 seeds. The weights and the cap were passed through a local patch that reads environment variables and was not committed. -
Brackets:
- the baseline (10,5,7,7,15);
- "conservative" (10,3,5,5,20) and "behavioral" (15,3,5,5,10);
- each weight ±25 % on its own. These run at ×4 scale so the values stay integers; the ×4 baseline reproduces the ×1 baseline exactly, with ARI 1 and every list identical;
- the cap at 10, 50 and none.
-
Measures against the baseline:
- Spearman ρ of the M7 ranking, over the rows present in both runs. It is computed both with tie-averaged ranks of the score and with the report's rank positions, and the table shows the lower. L rows are keyed by module + from + to.
- Jaccard of the top 10 for the M7 rows, the M6 divergent pairs and the M2 cuts.
- The ARI of M1's crate-level and module-level consensus partitions.
- The noise floor: the ARI between the consensus from seeds 0–9 and a replicate from seeds 1000–1009 under the same weights (M1's
replicate_ari). A bracket whose ARI to the baseline is close to that floor has moved no further than reseeding would move it.
-
Gunnar at a fixed γ: Gunnar's γ_crate pick is close to a boundary (see below), so its brackets were also run with γ_crate forced to 0.2, the baseline's pick.
Brackets
| bracket | weights / cap | Holger ARI crate / module | Holger seed-replicate ARI | Holger M7 ρ, top 10 | Gunnar γ picked | Gunnar ARI crate / module | Gunnar ARI at γ 0.2 | Gunnar M7 ρ, top 10 |
| baseline | 10,5,7,7,15 · 20 | 1 / 1 | 0.893 | 1, same | 0.2 | 1 / 1 | 1 | 1, same |
| conservative | 10,3,5,5,20 | 0.879 / 0.910 | 0.933 | 0.998, same | 0.2 | 0.897 / 0.915 | 0.897 | 0.998, same |
| behavioral | 15,3,5,5,10 | 0.821 / 0.835 | 0.951 | 1.000, same | 0.2 | 0.900 / 0.837 | 0.900 | 1.000, same |
| calls −25 % | 7.5 | 0.893 / 0.927 | 0.923 | 0.999, same | 0.2 | 0.909 / 0.923 | 0.909 | 1.000, same |
| calls +25 % | 12.5 | 0.854 / 0.919 | 0.887 | 0.998, same | 0.3 | 0.776 / 0.910 | 0.888 | 1.000, same |
| uses −25 % | 3.75 | 0.943 / 0.924 | 0.913 | 1.000, same | 0.2 | 0.905 / 0.946 | 0.905 | 0.998, same |
| uses +25 % | 6.25 | 0.917 / 0.936 | 0.881 | 1.000, same | 0.2 | 0.980 / 0.941 | 0.980 | 1.000, same |
| reads −25 % | 5.25 | 0.880 / 0.899 | 0.914 | 0.998, same | 0.2 | 0.868 / 0.917 | 0.868 | 0.998, same |
| reads +25 % | 8.75 | 0.894 / 0.919 | 0.942 | 1.000, same | 0.2 | 0.855 / 0.946 | 0.855 | 1.000, same |
| writes −25 % | 5.25 | 0.854 / 0.930 | 0.937 | 0.999, same | 0.2 | 0.881 / 0.944 | 0.881 | 1.000, same |
| writes +25 % | 8.75 | 0.873 / 0.891 | 0.853 | 0.998, same | 0.2 | 0.902 / 0.970 | 0.902 | 0.998, same |
| implements −25 % | 11.25 | 0.932 / 0.971 | 0.876 | 1.000, same | 0.2 | 0.993 / 0.978 | 0.993 | 1.000, same |
| implements +25 % | 18.75 | 0.968 / 0.959 | 0.894 | 1.000, same | 0.2 | 0.987 / 0.955 | 0.987 | 1.000, same |
| cap 10 | 0.843 / 0.919 | 0.903 | 0.999, same | 0.3 | 0.768 / 0.926 | 0.984 | 1.000, same | |
| cap 50 | 0.914 / 0.927 | 0.922 | 1.000, same | 0.3 | 0.688 / 0.933 | 0.907 | 1.000, same | |
| no cap | 0.861 / 0.920 | 0.923 | 1.000, same | 0.3 | 0.726 / 0.942 | 0.904 | 1.000, same |
"same" means the top 10 M7 rows are identical and in the same order (Jaccard 1).
The rest held in all 38 runs (19 per target, the ×1 baseline included):
-
M6 gave the identical divergent pair list, weight for weight.
- Holger has 15 pairs: checks→server 323, server→doors 282, checks→formats 209, gate→server 153, gate→doors 127, …
- Gunnar has 50: server→git-engine 91, doors→git-engine 52, control-plane→git-engine 49, server→control-plane 49, svc-access→control-plane 42, …
-
M2 gave the identical cut list.
- Holger: β 2.62 %, 29 layers, cut weight 495; the top cut is jobs → import_door, 23.
- Gunnar: β 1.14 %, 20 layers, cut weight 222; the top cut is storage → storage::znippy_engine, 14.
-
κ stayed at 0.861 on Holger and 0.949 on Gunnar.
-
M1's Q at γ = 1 on each bracket's own
W: Holger 0.768–0.787 (baseline 0.781); Gunnar 0.682–0.711 at γ 0.2 (baseline 0.711), and 0.742–0.763 where the pick moved to 0.3. Today's crates score 0.402 on Holger and 0.675 on Gunnar.
What moves is only the tail of the loop rows:
- Holger: the baseline has 6 L rows, at ranks 19–25 of 41. Depending on the bracket, 0–3 of them drop out (topology server→gate, index_rows::npm formats→doors, scanner::provenance checks→doors) and 0–2 new ones appear, at ranks 20–24. L4 (image_sbom checks→formats), L2 (handlers server→formats) and L3 (job_runner doors→checks) are kept in every bracket.
- Gunnar: L1 (
gunnar_serverboot→server) stays rank 1 in every bracket. Five brackets add one or two rows (storage / storage::znippy_engine git-engine→core) at ranks 22–23.
The M7 rows from the review, the seams (S), the cuts (C) and the concepts (K) do not depend on the weights at all.
The site cap
The cap is applied per item edge (one caller fn → one callee fn), before Cᵀ·S·C sums the edges into unit pairs. Both distributions are given:
| edges / pairs | p50 | p90 | p99 | max | > 10 | > 20 | > 50 | site mass clipped at 20 | |
| Holger, CALLS item edges | 12,058 | 1 | 2 | 6 | 156 | 44 | 11 | 1 | 1.6 % |
| Gunnar, CALLS item edges | 13,490 | 1 | 2 | 6 | 23 | 40 | 3 | 0 | 0.05 % |
| Holger, directed unit pairs (Σ sites) | 6,182 | 1 | 4 | 17 | 156 | 153 | 42 | 7 | (5.4 %)* |
| Gunnar, directed unit pairs (Σ sites) | 6,762 | 1 | 4 | 16 | 370 | 150 | 47 | 8 | (10.6 %)* |
* This is what a per-pair cap would clip; the per-edge cap does not clip it.
At cap 10, 50 and none:
- M7: the top 10 is the same, in the same order, with ρ ≥ 0.998.
- M6 and M2 do not change.
- Setting the sym-graph cap as well (M1 cap only vs both caps) gives identical results on both targets.
- Holger: M1 ARI is 0.84–0.91, at the seed floor.
- Gunnar: the γ pick moves from 0.2 to 0.3 for every cap other than 20. At a forced γ 0.2 the ARI is 0.90–0.98.
Is 20 a good default? Yes.
- It sits above p99 of the item edges on both targets.
- It clips 11 edges on Holger and 3 on Gunnar: the outliers, such as Holger's 156-site edge, which would otherwise dominate a unit pair.
- No finding depends on it.
- A cap of 10 would clip 4–13× as many edges and buy nothing.
Verdict: STABLE
In every bracket, ±25 % on any single weight as well as the two alternative weight sets, on both targets:
- the top 10 M7 rows are identical and in the same order;
- ρ ≥ 0.998;
- the M6 divergences and the M2 cuts are identical.
M1's consensus partition moves by about as much as reseeding moves it. The ARI to the baseline is 0.821–0.968 on Holger, against a seed-replicate ARI of 0.853–0.951, and 0.855–0.993 on Gunnar at a fixed γ (seed-replicate 0.829–0.972). The largest single-weight effects come from calls and reads/writes (minimum ARI 0.854–0.855); implements has the least effect (≥ 0.93).
The one sensitivity found is not a weight. It is M1's γ pick on Gunnar. The sweep gives 19 groups at γ 0.2 and 22–23 at γ 0.3, and the picker (closest count to today's crates, on a plateau) sits between these two grid points, so calls +25 % or any cap other than 20 tips the pick to 0.3. Even then, M7's top 10 is unchanged.
Recommendation
- Keep the weights and the cap fixed as compiled (10, 5, 7, 7, 15; cap 20). They do not need fitting.
- Do not add the optional per-spec
weights:override (item 3 of the review). The measurement gives no reason for it: no finding moves, and an override would add a knob to the spec format that the results do not respond to. - The γ-pick boundary on Gunnar is noted as an observation, not a plan row. M1's partition flips between two neighbouring grid points there. This changes only the loop rows in the tail of M7. If the owner wants M1's partition itself to be reproducible across small weight changes, the place to look is the γ picker (for example, pinning γ_crate in the spec, which
ArchRecoverArgs::gamma_cratealready supports), not the weights.
6. Open questions (for review)
- Weights. Should the edge weights in M1's
Wbe learned, by fitting them so that M1 reproduces the reviews' hand-made target shapes, or fixed by hand? If learned, how do we avoid fitting to Holger only? - Level. Is the unit level (a type with its methods) right, or should M1 and M6 run on modules? Rust modules are often too large in Holger and Gunnar, and units may be too small.
- Target models. Should the target model live in each repo (for example
.nornir/architecture.ron) and be checked in CI the way the micro goldens are, so a divergence fails the build? - Generics. Calls through a generic bound (
T: Trait) are monomorphised edges in MIR. Should M3 and M6 see them as the trait edge, the impl edge, or both? - Cross-repo. Holger's sandbox against Heimdall, and OSV parsing against modgunn: should M4 and M6 run on a dump that covers several repos, with the other repos as components?
- Feedback arc set. Is greedy Eades–Lin–Smyth plus local improvement good enough at Holger's size, or is an exact solver (ILP) on the small SCCs worth it?
7. References
- Murphy, Notkin, Sullivan: Software reflexion models: bridging the gap between source and high-level models, SIGSOFT FSE 1995. https://doi.org/10.1145/222124.222136
- Traag, Waltman, van Eck: From Louvain to Leiden: guaranteeing well-connected communities, Scientific Reports 9:5233, 2019. https://doi.org/10.1038/s41598-019-41695-z
- Eades, Lin, Smyth: A fast and effective heuristic for the feedback arc set problem, Information Processing Letters 47(6), 1993. https://doi.org/10.1016/0020-0190(93)90079-O
- MacCormack, Rusnak, Baldwin: Exploring the structure of complex software designs: an empirical study of open source and proprietary code, Management Science 52(7), 2006 (propagation cost). https://doi.org/10.1287/mnsc.1060.0552
- Wen, Tzerpos: An effectiveness measure for software clustering algorithms (MoJoFM), IWPC 2004. https://doi.org/10.1109/WPC.2004.1311061
- Existing lenses and their formulas:
lenses.md, sections 4.4cycles, 4.10chokepoints, 4.11misplacedand 4.12dsm.