nordisk/edda

Upstream report, ready to file: in_parallel_with_slice appends a whole periodic() interval to every call

FILED 2026-08-04: https://github.com/GitoxideLabs/gitoxide/pull/2877 (as Ignalina) — a PR, not just a report: the fix is 32 lines in one function and cargo test -p gix-features --features parallel passes on it upstream, including the two tests that exercise the error arms the unpark calls were added to.

Filed under the Ignalina account — same rule and same precedent as .nornir/upstream-gix-base-size.md (https://github.com/GitoxideLabs/gitoxide/issues/2868, filed 2026-08-03) and .nornir/upstream-gix-stepwise-serial.md.

Found 2026-08-04 measuring gunnar's receive path. Unlike the other two, gunnar is blocked on this one and carries a local patch until it lands: see vendor/gix-features/PATCH.md and the [patch.crates-io] tables in Cargo.toml and bench/Cargo.toml. It cost every index-pack 50 ms of sleeping.

Title

parallel::in_parallel_with_slice's watcher thread sleeps out its whole interval after the work is finished, adding it to every call

Body

in_parallel_with_slice spawns an interrupt-watching thread whose body is:

// gix-features/src/parallel/in_parallel.rs
move || loop {
    if stop_everything.load(Ordering::Relaxed) {
        break;
    }
    match periodic() {
        Some(duration) => std::thread::sleep(duration),
        None => {
            stop_everything.store(true, Ordering::Relaxed);
            break;
        }
    }
}

stop_everything is set at the end of the enclosing std::thread::scope, after every worker has been joined:

for thread in threads { … }          // all workers joined here

stop_everything.store(true, Ordering::Relaxed);
Ok(results)

The watcher is inside that same scope, so std::thread::scope cannot return until the watcher wakes on its own. The sleep is therefore appended to the call, not overlapped with it. in_parallel_with_slice cannot return in less than one periodic() interval no matter how little work it was given, and the whole of that interval is dead time: every worker has already finished and every result is already collected.

What it costs

gix-pack's delta traversal is the caller, and it asks for 50 ms:

// gix-pack/src/cache/delta/traverse/mod.rs
|| (!should_interrupt.load(Ordering::Relaxed)).then(|| std::time::Duration::from_millis(50)),

Tree::traverse is unconditional in index::File::write_data_iter_to_stream, so every index-pack pays 50 ms, including the smallest one. MEASURED on Linux 6.x / x86-64, gix-pack 0.73, gix-features 0.49, release build, an 8-object 490-byte pack (a five-file commit — five blobs, two trees, one commit):

strace -f -c -e trace=clock_nanosleep over one call: exactly one clock_nanosleep(CLOCK_MONOTONIC, {tv_nsec=50000000}) before, and none after.

The before-column's flatness is the whole point. Those 101 samples were taken on a 32-core box at loadavg 317; the spread is 50.22–55.01 ms, because the sleep dominates everything the machine is doing. The after-column's spread (2.99–43 ms) is that same load, now visible. Process CPU is unchanged: nothing was made faster, a wait was removed.

Suggested fix

Park instead of sleeping, and unpark once there is nothing left to watch.

let watcher = std::thread::Builder::new()
    .name("gitoxide.in_parallel_with_slice.watch-interrupts".into())
    .spawn_scoped(s, move || loop {
        if stop_everything.load(Ordering::Relaxed) { break; }
        match periodic() {
            Some(duration) => {
                let deadline = std::time::Instant::now() + duration;
                loop {
                    if stop_everything.load(Ordering::Relaxed) { break; }
                    let remaining = deadline.saturating_duration_since(std::time::Instant::now());
                    if remaining.is_zero() { break; }
                    std::thread::park_timeout(remaining);
                }
            }
            None => { stop_everything.store(true, Ordering::Relaxed); break; }
        }
    })
    .expect("valid name");
let watcher = watcher.thread().clone();

// … then `watcher.unpark();` on each of the three exits of the join loop:
// normal completion, `Ok(Err((err, is_cause)))`, and the panic arm.

Three properties worth calling out, because they are why this shape rather than a smaller one:

• The inner loop preserves the interval exactly. park_timeout may return spuriously; re-parking for the remaining time means a spurious wake-up cannot shorten the polling cadence periodic() asked for, so callers that do real work in periodic are unaffected.
• No mutex, and no race. A thread's park token is remembered: an unpark that lands before the park_timeout makes that park_timeout return immediately. There is no window in which the joiner signals and the watcher then sleeps anyway.
• The error and panic arms need it too. Without unpark there, a failing traversal still holds its error open for one whole interval before the scope can return.

The diff is +30/−2 lines, all inside one function, no signature change and no new dependency.

Why not just lower gix-pack's 50 ms

That also removes the floor, and it was rejected: 50 ms is a perfectly good latency for noticing a Ctrl-C, and an interval that has to be small enough to hide a bug will wake a thread a thousand times a second for the length of a large index. The interval is not the defect; an uninterruptible wait is.

Environment

• gix-features 0.49.0 (parallel on), published gix-pack 0.73.0
• Linux 6.x, x86-64, 32 cores
• observed through gix_pack::Bundle::write_to_directory

CORRECTION, found while checking this against upstream main before filing

The sentence "Tree::traverse is the only caller of in_parallel_with_slice in gitoxide" was true of the published crate and false of main, and the PR body now says so.

• Published gix-pack 0.73.0 is the latest release on crates.io (checked against the sparse index: 0.69 → 0.73, nothing newer) and it does drive in_parallel_with_slice from Tree::traverse at 50 ms. That is gunnar's reality and the whole basis of P-030's numbers.
• Upstream main has rewritten that traversal to resolve::all, which does not call in_parallel_with_slice at all. The floor is already gone there, by a different route. main's gix-pack is still numbered 0.73.0, so the rewrite is unreleased.
• The two callers left on main are gitoxide-core/src/repository/odb.rs:248 and gitoxide-core/src/corpus/engine.rs:213, and both pass 100 ms. The odb.rs closure never returns None, so it adds a flat 100 ms to every run that reaches it.

Two things follow and both matter to gunnar. First, gunnar cannot upgrade out of this: there is no released gix-pack with the rewrite, so the patch is the only route and stays until one ships. Second, when that release does come, check gix-pack before checking gix-features — the floor may lift from the gix-pack side first, and crates/gunnar-pack/tests/index_floor.rs will go green without this patch at all. That is the moment to delete vendor/.

What gunnar did, and why it is a patch and not a workaround

Carried, in vendor/gix-features, reached by [patch.crates-io]. There is no caller-side lever: the interval is a hard-coded literal in gix-pack, and the one knob that reaches the watcher — should_interrupt, through periodic() returning None — aborts the traversal, which is the opposite of what is wanted. thread_limit: Some(1) pays the floor too, because the watcher is spawned regardless of the worker count.

The patch is at gix-features rather than at gix-pack's literal deliberately, so that gix-pack stays stock and one vendored crate covers it. It costs gunnar's dependency graph nothing — same crate, same version, same dependencies, appliance profile 280 crates / 267 third-party before and after — and is deleted the day this lands.