Your browser sucks, JavaScript disabled! Gunnar's console needs JavaScript and WebAssembly. Git over SSH on port 2222 does not.

nordisk/facett

.nornir/flowsim-spike.md

facett-flowsim — design spike

Edge-parametric particle ADVECTION on a system/pipeline DAG: particles ride each edge from source → consumer at speed = normalized throughput and density = volume, bound to a per-edge flow: f32 derived from the same ViewerState the DAG is already drawn from. GPU-less pure core (unit-testable flow→speed), a wgpu compute pass that reuses facett-core's particles.wgsl render, and a toggle/overlay with a static-arrows fallback (the fallback already exists — see §5).

Status: BUILT AND LANDED — facett d7b1f68 on main (pushed 2026-08-01, after an independent re-verification: the FC-7 GPU parity proofs ran on a real device with no self-skip, and a fresh constant-speed mutation turned 5 tests red on applied output — busy 22.64px vs idle 22.64px). All 7 build-order steps of §7 are implemented, tested and pushed. The design below is accurate except where flagged [STALE] / [CORRECTED] — read §9 (at the bottom) before trusting any line of §0–§8.


0. What already exists (so we reuse, not twin)

Reuse law consequence: all particle GPU machinery (buffers, render pipeline, CPU parity harness) STAYS in facett-core. facett-flowsim is the domain binding (flow→speed/density) + the overlay/toggle + the fallback. It never re-implements a particle system.


1. Where it lives

New leaf crate facett-flowsim at facett/facett-flowsim/, added to the workspace members list in facett/Cargo.toml:3 (next to facett-pipeline / facett-graphview).

facett-flowsim/
  Cargo.toml
  src/
    lib.rs      # FlowSim overlay + Msg/state_json + static-arrows fallback
    flow.rs     # PURE core: flow:f32 → (speed, density). No egui, no wgpu. Unit-tested.
    gpu.rs      # #[cfg(feature="wgpu")] thin driver over facett_core GpuParticles advect

Cargo.toml deps (mirror how peers path-dep, e.g. pipeline Cargo.toml:31):

[dependencies]
facett-core = { path = "../facett-core", version = "0.1" }
egui = { workspace = true }
serde = { version = "1", features = ["derive"] }
serde_json = { workspace = true }

[features]
testmatrix = ["facett-core/testmatrix"]
# GPU advection pass; OFF by default so the pure core + fallback always build
# (matches graphview's `gpu` gating and particles' `wgpu` gating).
wgpu = ["facett-core/wgpu"]

Dependency direction (no cycles): facett-flowsim depends ONLY on facett-core. It does not depend on facett-pipeline (which would cycle once pipeline consumes flowsim). It takes already-computed screen-space spline polylines + per-edge flow: f32 from its host. Hosts that own a DAG (facett-pipeline, facett-plandag, korp's dep-view) depend on facett-flowsim and drive it. No knut dependency anywhere (leaf law — knut's Spark adapter maps ViewerState → PipelineModel and stops there, pipeline model.rs:17).


2. The pure core — flow.rs (GPU-less, unit-testable)

The heart Rickard wants testable: flow → motion, no device.

/// Tuning for the flow→motion mapping. Default = calm, readable.
pub struct FlowParams {
    pub min_speed: f32,     // arc-fraction/sec at flow=0 (a trickle, never frozen)
    pub max_speed: f32,     // arc-fraction/sec at flow=1
    pub density_per_edge: u32, // particle count at volume=1 (scaled by volume)
    pub max_density: u32,   // hard cap per edge (keeps big DAGs bounded)
    pub speed_gamma: f32,   // perceptual curve on normalized throughput (e.g. 0.6)
}

/// Per-edge advection parameters — the OUTPUT of the pure core, the INPUT to
/// both the GPU pass and the fallback.
pub struct EdgeParam {
    pub edge: usize,     // index into edge_ends / the polyline list
    pub speed: f32,      // arc-fraction advanced per second (>0 always)
    pub count: u32,      // particles seeded on this edge (density = volume)
}

/// PURE: normalize a raw per-edge throughput against the busiest edge, then map
/// to (speed, count). `flow[i]` is raw throughput_rows_s (or bytes/s) of edge i;
/// `volume[i]` is a size hint (rows materialized / edge weight), 1.0 if unknown.
pub fn plan_edges(flow: &[f32], volume: &[f32], p: &FlowParams) -> Vec<EdgeParam>;

/// PURE: normalized throughput in [0,1] → speed in [min,max] via speed_gamma.
pub fn flow_to_speed(norm: f32, p: &FlowParams) -> f32;

/// PURE: advance one particle's arc param s∈[0,1) by speed·dt, wrapping. This is
/// the CPU MIRROR of the WGSL `cs_advect` s-update (FC-7 parity, §3).
pub fn advance_s(s: f32, speed: f32, dt: f32) -> f32; // (s + speed*dt).rem_euclid(1.0)

Unit tests (red-when-broken, testable-code law):

  1. flow_to_speed monotonic + clamped: norm=0 → min_speed, norm=1 → max_speed, monotone increasing between (mirrors particles.rs:411 style).

  2. plan_edges normalizes to the busiest edge: the max-throughput edge gets max_speed; a zero-throughput edge gets min_speed (never 0 — no frozen edge). Empty input → empty output.

  3. plan_edges density tracks volume + caps: `count == (density_per_edge as f32

    • volume).round()clamped tomax_density`.
  4. advance_s wraps in [0,1) and is deterministic (bit-identical twice), and a full cycle returns to the start within one period — same shape as edge_phase_wraps_and_increases (pipeline lib.rs:1363).

No egui, no wgpu in flow.rs → runs in plain cargo test, on the headless box.


3. The GPU compute pass — add cs_advect to particles.wgsl (reuse render)

The advection compute is NEW domain logic but the point render is shared, so it lands in facett-core beside the boids (reuse law: particle GPU code lives once). Add ONE compute entry + one method; the existing pt_vs/pt_fs render (particles.wgsl:110) and GpuParticles::render (particles.rs:349) are reused unchanged.

Particle field packing (reuse the 16-byte Particle, particles.rs:24)

New WGSL (append to facett-core/src/render/gpu/particles.wgsl)

// ── edge-parametric advection ─────────────────────────────────────────
struct AdvectU { a: vec4<f32>, };      // x=dt, y=count, z=_, w=_
@group(0) @binding(0) var<uniform> ADV: AdvectU;
@group(0) @binding(1) var<storage, read>       a_src: array<Particle>;
@group(0) @binding(2) var<storage, read_write> a_dst: array<Particle>;
@group(0) @binding(3) var<storage, read> edge_off: array<u32>;   // CSR: per-edge start into pts (len = n_edges+1)
@group(0) @binding(4) var<storage, read> edge_pts: array<vec2<f32>>; // concatenated normalized polylines
@group(0) @binding(5) var<storage, read> part_edge: array<u32>;  // per-particle edge id

@compute @workgroup_size(64)
fn cs_advect(@builtin(global_invocation_id) gid: vec3<u32>) {
    let i = gid.x;
    let n = u32(ADV.a.y);
    if (i >= n) { return; }
    let dt = ADV.a.x;
    let me = a_src[i];
    let s = fract(me.vel.x + me.vel.y * dt);   // advance arc param, wrap [0,1)
    let e = part_edge[i];
    let start = edge_off[e];
    let end   = edge_off[e + 1u];
    let pos = sample_polyline(start, end, s); // arc-length lookup along the edge
    a_dst[i].pos = pos;
    a_dst[i].vel = vec2<f32>(s, me.vel.y);      // keep s + speed for next step
}

sample_polyline(start,end,s) walks edge_pts[start..end] by cumulative arc-length to fraction s — the WGSL twin of the CPU point_at_arclen (pipeline lib.rs:981). Ping-pong a_src/a_dst has no read/write hazard, same pattern as cs_step (particles.wgsl comment :4) and cs_layout (layout.wgsl:2), so it's deterministic (FC-7).

New Rust method (in facett-core/src/render/gpu/particles.rs)

GPU↔CPU parity proof (self-skipping, particles.rs:456 harness reused)

Add gpu_advect_matches_cpu_reference next to gpu_step_matches_cpu_reference (particles.rs:508): seed N particles across 2 edges, one GPU advect step vs advect_cpu, assert pos within 1e-3 and that particles actually moved along the edge (Δs > 0). Reuses compute_device() (particles.rs:456) and read_particles (particles.rs:481) verbatim.

Extract-and-share (reuse-law cleanup, small): move point_at_arclen (pipeline lib.rs:981) into facett_core::edges as pub fn point_at_arclen(pts: &[Pos2], target: f32) -> Pos2 so the CPU mirror, the fallback, and pipeline's existing dashes all call ONE arc-length walk. The WGSL sample_polyline is its GPU twin.


4. Seeding — density = volume

facett-flowsim::gpu::seed(edges: &[EdgeParam], polylines) -> (Vec<Particle>, Vec<u32> /*part_edge*/): for each EdgeParam, push count particles with s spread evenly in [0,1) (deterministic seed, like ring() particles.rs:398), pos = polyline sampled at that s, vel = (s, speed), and part_edge = edge. Total N = Σ count. One shared ping-pong ParticleBuffers (particles.rs:294) for ALL edges — a single compute dispatch + a single additive render pass per frame, independent of edge count.


5. The overlay, toggle, and static-arrows fallback — lib.rs

facett-flowsim exposes an overlay a DAG host paints after its edges/nodes. It follows the facett Elm/FC contract (serializable state + Msg + state_json), same as PipelineState (pipeline lib.rs:249) and PlanDagView (plandag lib.rs:16).

pub struct FlowSim {
    params: FlowParams,
    mode: FlowMode,          // Particles | Arrows (fallback) | Off
    speed_scale: f32,        // global multiplier (UX slider)
    clock_s: Option<f64>,    // injected clock (FC-7), like PipelineState.clock_s (lib.rs:275)
    // gpu state behind #[cfg(feature="wgpu")]: GpuParticles + ParticleBuffers + FlowGeometry
}
pub enum FlowMsg { SetMode(FlowMode), SetSpeed(f32), Tick(f64), SetClock(f64) }

impl FlowSim {
    /// Feed this frame's geometry + signal. `polylines[i]` is the screen-space
    /// spline of edge i (host builds it exactly as paint_flow_edge does,
    /// pipeline lib.rs:931 spline_points). `flow`/`volume` are per-edge.
    pub fn sync(&mut self, polylines: &[Vec<Pos2>], flow: &[f32], volume: &[f32]);
    /// Paint the overlay onto `painter` for the current frame/clock.
    pub fn paint(&mut self, painter: &egui::Painter, time: f64);
    pub fn update(&mut self, msg: FlowMsg);
    pub fn state_json(&self) -> serde_json::Value; // mode, n_edges, total_particles, gpu:bool, max/min speed
}

Three-way resolution inside paint:

The fallback is not new work — it is the existing static arrow, driven by the same EdgeParam. plan_edges runs in BOTH modes, so speed/density are identical whether you see particles or chevrons.

Robot test (UI law): state_json is the introspection surface. A kittest/ robotui test builds a FlowSim, syncs a 2-edge DAG with asymmetric flow, pins the clock, and asserts: mode, total_particles scales with volume, the busy edge's speed > the idle edge's, and (GPU-present) a non-blank additive frame — mirroring particles.rs:544 additive_render_accumulates_overlap. On the headless box the GPU legs self-skip (particles.rs:509) and the Arrows-fallback + pure-core legs still assert. Record the row in the testmatrix (facett_core::testmatrix::emit, cf. l0.rs:140).


6. How the korp Pipelines DAG / dep-view turn it on

The signal is already in the model each host renders from — flowsim just needs (polylines, flow, volume) and a toggle.

A. facett-pipeline (the Pipelines DAG)

  1. Add a flowsim feature to facett-pipeline/Cargo.toml → dep:facett-flowsim (optional, like runa-adapter pipeline Cargo.toml:12).
  2. PipelineView gains flowsim: Option<FlowSim> + with_flowsim(bool) (same builder shape as with_animation, lib.rs:507) and a Msg::FlowMode / Msg::FlowSpeed pass-through.
  3. Derive per-edge flow/volume where edge_ends is walked in paint_dag (lib.rs:791): for edge (from_i, to_i), flow = nodes[from_i] .throughput_rows_s (source stage's live rate, lib.rs:167; fall back to the throughput label parse lib.rs:802), volume = nodes[from_i].rows (lib.rs, PipelineNode.rows). Collect the polylines from the SAME spline_points call paint_flow_edge already makes (lib.rs:931) — refactor it to return the sampled pts so both the dash painter and flowsim share one spline (no second Bézier — reuse).
  4. When flowsim is Some and its mode ≠ Off, paint_dag calls flowsim.sync(&polylines, &flow, &volume) + flowsim.paint(&painter, time) instead of the marching-ants block (lib.rs:936–955); the base curve + arrowhead still draw. Off → today's dashes. The existing with_animation(false) path already is the static fallback, so the two degrade consistently.
  5. The knut adapter feeding this (ViewerState → PipelineModel) already carries StageMetrics.throughput_rows_s (model.rs:122) and rows (model.rs:120); ExecutionProgress (knut progress.rs:54) is the pipeline-level flow. No knut change needed — the per-edge flow is derived host-side from data that's already in PipelineState.

B. korp dep-view / Pipelines tab

korp renders its DAG through facett-pipeline's PipelineView (korp is a host; facett-korp itself is the Ingest leaf, korp Cargo.toml:7, and does not dep pipeline). So korp turns flowsim on by:

C. facett-plandag (optional, same seam)

PlanDagView composes the same stack (plandag lib.rs:9–14) and has per-node metric bodies (rows/time). It can adopt FlowSim identically: flow from the plan's row/rate metric, volume from rows — one more host, zero new engine work.


7. Build order for the executing agent

  1. Pure core first (facett-flowsim/src/flow.rs + Cargo.toml + workspace member): FlowParams, EdgeParam, plan_edges, flow_to_speed, advance_s + the 4 unit tests (§2). Green with plain cargo test -p facett-flowsim — no GPU, no egui. This is the testable heart.
  2. Shared arc-length helper: move point_at_arclen → facett_core::edges (§3), repoint pipeline lib.rs:949–950 to it. Keep pipeline tests green.
  3. GPU pass in facett-core: append cs_advect + sample_polyline to particles.wgsl; add EdgeAdvectParams, FlowGeometry, GpuParticles::advect, advect_cpu to particles.rs; add the parity proof (§3). Extend shader_entry_points_and_uniform_sizes (particles.rs:620) to assert fn cs_advect + AdvectU size.
  4. Overlay + fallback (facett-flowsim/src/lib.rs + gpu.rs): FlowSim, FlowMsg, state_json, paint three-way (§5), seed (§4). Robot test with the Arrows fallback (GPU legs self-skip).
  5. Wire pipeline (§6.A): flowsim feature, with_flowsim, derive flow/volume, swap the dash block. Then korp toggle (§6.B).

Each step is independently green and testable; steps 1–2 need no GPU. Nothing here uses rayon/threads (ROOT LAW: GPU compute + pure functions only — no CPU thread pools introduced).


8. Contracts honored


9. WHAT IS ACTUALLY TRUE — written by the executing agent, 2026-08-01

9.1 Built vs reused

REUSED verbatim (nothing re-implemented; each verified against the tree first):

piece reused from note
spline geometry facett_core::edges::spline_points (facett-core/src/edges.rs:23) the DAG's ONE Bézier; facett-pipeline's edge_spline (lib.rs:1107) wraps it and both the dash painter and flowsim call that wrapper — no second Bézier
arc-length walk facett_core::edges::point_at_arclen (edges.rs:44) + point_at_frac (edges.rs:73) already extracted out of pipeline (spike §3's "extract-and-share" is DONE); pipeline's dashes (lib.rs:1148-1149), the flowsim CPU advect, the chevrons and advect_cpu all route through it
particle GPU machinery facett-core/src/render/gpu/particles.rs — Particle (:24), ParticleBuffers (:153), GpuParticles (:169), render (:349) untouched; flowsim's gpu.rs is a ~97-line binding
GPU advect + CPU mirror advect_cpu (particles.rs:140), FlowGeometry (:191), GpuParticles::advect (:474), cs_advect/sample_polyline (particles.wgsl:190 / :167) already in facett-core, per the reuse law
FC-7 parity harness compute_device() (particles.rs), gpu_step_matches_cpu_reference, gpu_advect_matches_cpu_reference both still green, and they did NOT self-skip — a real GPU ran them on oden
static fallback paint_flow_edge's non-animated path (facett-pipeline/src/lib.rs:1119, arrowhead at :1163) still the fallback; flowsim never re-draws the base curve

BUILT this session (all in facett-flowsim/src/{flow,lib}.rs, facett-pipeline/src/lib.rs, facett-pipeline/tests/robot_flowsim.rs):

9.2 [CORRECTED] Two real defects the spike's design contained

  1. §2 said a zero-throughput edge gets min_speed — "a trickle, never frozen". Taken literally that makes the overlay decoration: a pipeline that has stopped glows exactly like a running one. Corrected: speed still reports min_speed (the mapping is over relative load), but count is 0 for a genuinely zero raw throughput, so nothing is drawn. count == 0 now means dead (is_flowing()); a flowing-but-thin edge is clamped up to 1 particle so thin can never masquerade as dead.
  2. §6.A step 3 said volume = nodes[from_i].rows. plan_edges reads volume as an absolute multiplier on density_per_edge, so raw row counts pin every real edge at max_density — 8 × 48 000 clamps to 64 and so does 8 × 120 000. Density carried zero information. Corrected in edge_flow_volume(): normalise rows against the busiest edge first. Measured effect on the 120k/48k/12k/4k fixture: per-edge particles went from [64, 64, 64] to [8, 2, 1].

9.3 [STALE] in §0–§8

9.4 LAW 2 — the guards were blind, proven RED

The pre-existing 8-test flowsim suite passed 8/8 with every particle frozen at the edge start, and again with the flow→speed mapping replaced by a constant. The paint_is_deterministic_under_pinned_clock test compared vertex counts, which are identical wherever the dots land — it could never go red.

Four mutations were run against the new suite; each produced real failure text:

mutation red tests actual text
particles frozen at pts[0] 3 a particle must MOVE: 0.000px on edge 0; an unpinned clock must move the dots ON SCREEN, not just their s
constant speed, throughput ignored 2 100× the throughput must show as a MUCH longer travel — busy 20.90px vs idle 20.90px; edge 0 particle travelled 26.12px, the plan says 7.38px
chord placement instead of the spline 2 particle on edge 0 at s=0.031 is 6.669px OFF the spline
zero-flow anchor reverted 2 no particles on a dead DAG: [EdgeParam { edge: 0, speed: 0.05, count: 8 }, …]; a dead DAG advects nothing

9.5 Visual proof — REAL wgpu frames, measured

facett-pipeline/tests/robot_flowsim.rs (cargo test -p facett-pipeline --features flowsim --test robot_flowsim -- --nocapture). Every claim is paired with its counterexample; each DAG is compared against a frame of itself, so labels cancel and the diff is the overlay alone:

live DAG: overlay changed 486 px vs the same frame with it Off (11 particles, derived floor 165)
dead DAG: overlay changed 0 px                     ← the anchor: a stopped pipeline does not glow
live DAG: 292 px changed between t=0.0 and t=0.42  (60% of the overlay footprint)
dead DAG: 0 px changed between clocks              ← a dead pipeline does not animate
dead DAG: 66 983 px of edge art vs an edgeless DAG ← base curve + arrowhead survive

PNGs: facett-pipeline/tests/snapshots/flowsim_{live,dead,arrows}.png. The live shot shows dense warm particles on the busiest edge and sparse cool ones on the slow edges (pt_vs colours by speed, particles.wgsl:127 — free throughput encoding).

9.6 What is NOT done


10. §9.6's first item, CLOSED — the host lane, 2026-08-01

facett-flowsim::gpu::host + FlowSim::render_gpu + the call in facett-pipeline's paint_dag (feature flowsim-gpu). It was a wiring job exactly as recorded: the compute pass, the CPU parity reference and the additive render all already existed and were already proven; nothing called them per frame.

Shape is facett-core's graphcloud::cloud, reused rather than re-invented — a persistent FlowRenderer in callback_resources, install_flow_renderer once at startup, and ONE fail-safe entry point (draw_flow) that returns true only if a frame really went, so a host may skip its CPU painter only when the GPU lane can honestly claim the frame. FlowSim::render_gpu holds the gate, so korp and plandag inherit the protocol instead of copying it.

Measured (facett-pipeline/tests/flowsim_gpu_lane.rs, real wgpu on oden)

GPU lane:  1123 px overlay vs the Off frame, 1 advect+draw frame on the device
CPU lane:   486 px overlay,                  0 device frames        (attribution)
alignment: 100.0% of CPU dots have a GPU dot within 4 px, and vice versa
extent:    GPU 453x19 px vs CPU 447x13 px    (GPU sprites are ~2x the CPU dot)
advect:     572 px moved between t=0.0 and t=0.42 — 50% of the footprint
dead DAG:     0 px overlay,                  0 device frames        (the anchor)

Derived floor 165 px = 11 planned particles x 15. flowsim_gpu_failsafe.rs (its own test binary, so it cannot lose its premise to test ordering) proves opting in without installing falls back to the CPU painter with 0 px of drift.

Three defects the work surfaced, all found by measurement

  1. gpu::seed was a partial twin of the CPU painter. It composed the seed spread and the advance itself as s = j / count, with no clock and no speed_scale — so the two lanes agreed only at t = 0 and scale = 1.0, the identity value. Both now route through flow::particle_s. Worst particle: 8.16 px -> 0.0000 px at t = 0.37, scale = 2.5.

  2. The 32 px space bug, and it is the one worth remembering. The lane first normalised to the SCREEN, reasoning that the shared blit stretches its offscreen over the whole surface before scissoring to the pane. Wrong: egui_wgpu sets the pass viewport to the callback's own rect, so the "fullscreen" blit triangle covers the widget. The overlay came out 32.6 px below the edges it was supposed to ride — and every pixel assertion was green for it, because an offset overlay is present, is animated, is GPU-drawn and differs from the CPU frame. It was found by looking at the PNG. The missing guard is now the central one: coverage of the CPU overlay by the GPU overlay within 4 px, which reads 0.0% for that bug.

    • The centroid was tried first and rejected — the lanes weight pixel mass differently along an edge (a 22 px x-centroid gap with the bounding boxes 3 px apart), so it would have been a false red AND blind to a shift along an edge.
  3. A hollow guard, built while hunting hollow guards — the third session running. "No particles to draw" was written three times (draw_flow, prepare, paint), so deleting the real gate changed no pixel and moved no counter. One writer now; the mutation is red. The duplicates existed because GpuParticles::make_buffers panicked on an empty slice while FlowGeometry::new ten lines below already padded for that exact reason; make_buffers now pads too.

The residual, stated rather than hidden

cs_advect walks cumulative arc length over the normalised poly-line, and screen -> [0,1]² is anisotropic on a non-square pane. So between reseeds the device parameterises an edge slightly differently from the CPU painter's screen-space walk. It is a reparameterisation, never an off-curve error — a particle is always exactly on its edge — and seed walks the screen curve so every reseeded frame is exact. Removing it entirely needs an aspect term in sample_polyline, i.e. a change to the proven cs_advect/advect_cpu pair, which is not worth it for a sub-dot effect.

9.7 Pre-existing breakage on origin/main — NOT from this work

cargo check --workspace --all-targets fails at 2a7e314 on a pristine worktree, before any change of mine:

Everything this session touched is green: cargo test -p facett-flowsim -p facett-core -p facett-pipeline and the same with --features flowsim / facett-flowsim/wgpu both exit 0.

Gotcha for the next agent: a facett worktree must live under /home/rickard/git/. facett-core path-deps ../../znippy-zoomies, so a worktree under /home/rickard/scratch/ cannot resolve its manifest at all.