nordisk/skade

skade-katalog benchmark: vs Nessie & Apache Polaris

Mirrors holger's vs-Nexus harness. Every competitor implements iceberg::Catalog (skade-katalog via RedbCatalog, Nessie/Polaris via the Iceberg REST client), so the scenarios in src/scenarios.rs are identical across all three.

This is a detached crate (own [workspace]): building it does not touch the published skade-katalog crate's dependency tree.

Targets

A REST-fronted nornir (axum shim) column — for an apples-to-apples REST-vs-REST comparison — is the remaining piece (see plan.md); the embedded number is the "no network hop" reference.

Run

# embedded — works out of the box, fully self-contained
BENCH_QUICK=1 cargo run --release -- embedded        # smoke (200 tables)
cargo run --release -- embedded                      # full (10k tables)

# Nessie (RocksDB) in a container, then bench against it
./containers/nessie_up.sh
BENCH_QUICK=1 cargo run --release -- nessie http://localhost:19120/iceberg warehouse
./containers/nessie_down.sh

# Polaris — needs OAuth2 + catalog bootstrap; polaris_up.sh prints the steps
./containers/polaris_up.sh
# …follow the printed token + create-catalog steps, then:
BENCH_REST_PROPS="credential=root:s3cr3t,scope=PRINCIPAL_ROLE:ALL" \
  cargo run --release -- polaris http://localhost:8181/api/catalog warehouse
./containers/polaris_down.sh

Output is JSONL (one scenario per line) with target, ops_sec, and p50/p99/p999_us — ready to append to bench_history.jsonl and fold into the README table, exactly like holger.

Storage coupling (important)

load_table makes the client read the metadata file the server wrote. So server and client must share the warehouse. The container scripts bind-mount a host dir at the same path inside the container and use a file:// warehouse, so the client's LocalFs FileIO reads the identical bytes. For a realistic object-store run, stand up MinIO and point both sides at s3://… (requires the iceberg S3 FileIO feature; not wired by default).

Ægir transport: ureq (default) vs uring (io_uring)

The S3 data plane runs on aegir (bench/aegir/), our pure-Rust S3 client (SigV4 over a pluggable byte transport, zero-copy Bytes reads). Two transports:

• ureq (default) — pure-Rust HTTP over rustls; handles TLS + plaintext.
• uring (opt-in feature) — an io_uring TCP transport for the plaintext RustFS hot path (:9000, no TLS). Built on the pure-Rust io-uring crate (libc FFI + bitflags + cfg-if only — no C, no bindgen, no OpenSSL). Connect via std (resolver); the request/response data path runs through io_uring Send/Recv SQE/CQE loops, over a per-thread keep-alive connection pool (so it matches ureq's pooled-agent model — one TCP handshake amortised over many ops). SigV4 + request/response semantics are identical; only the byte transport swaps. TLS endpoints always fall back to ureq.

Select per-client (Client::with_transport(Transport::Uring)) or via AEGIR_TRANSPORT=uring; compile with --features uring.

# RustFS up, then the transport micro-bench (HEAD/GET/PUT, isolates the byte path)
cargo run --release --features uring --example aegir_transport
# or the full data plane on either transport:
AEGIR_TRANSPORT=uring cargo run --release --features uring -- data skade-s3

ureq vs uring — plaintext RustFS micro-bench (8k iters, this box)

With keep-alive pooling, uring reaches parity with ureq (HEAD marginally ahead on both ops/s and p50); single-connection serial latency is dominated by RustFS server processing + RTT, so both transports converge once the per-call TCP handshake is removed. Without pooling, uring was ~30–40 % slower (one handshake per op) — the pool closes that gap. uring's win is doing it with no TLS stack in the path on the plaintext hot path, pure-Rust io_uring throughout.

Scenarios (src/scenarios.rs)

• load_table.latency — warm single-thread load_table (the trait method).
• resolve_metadata.latency — nornir-only fast path: returns Arc<TableMetadata> and skips Table::build(). This is the true catalog read speed.
• create_table.latency — write path.
• load_table.throughput.t{N} — concurrent load_table at N threads.

Reference numbers (this dev box, local NVMe, BENCH_QUICK)

Tri-modal (embedded / REST / gRPC) — same RedbCatalog, three fronts (2026-07-15, Loki, BENCH_QUICK). gRPC load_table p50 110.7 µs is on par with REST's 109 µs: over loopback the response is dominated by the metadata read + round-trip, so protobuf-vs-JSON is marginal — they tie, and gRPC's edge shows under concurrency (its throughput scales t1→t16 4.4×). ⚠️ TCP_NODELAY fix (this commit): before it, gRPC load_table/create_table stalled at a pathological ~41 ms (24 ops/s) — Nagle × delayed-ACK on the multi-segment metadata response. tonic's builder-level .tcp_nodelay(true) is ignored on the serve_with_incoming path, so each accepted socket now gets set_nodelay(true) explicitly (grpc/src/lib.rs). That took load_table 41 ms → 110 µs (371×), create_table 42 ms → 950 µs (44×). hyper/axum sets NODELAY by default, which is why REST never showed it.

load_table (embedded) p50 is 1.5 µs since the L1.5 built-Table handle cache (src/table_cache.rs) + location-first lookup — down from ~15 µs. The old cost was iceberg-rust building a fresh per-Table moka ObjectCache on every call (~14 µs, and disable_cache does not avoid it — see examples/cache_ab.rs); caching the built Table by immutable metadata_location and cloning it (shares the Arc<ObjectCache>, ~100 ns) removes it and lets load_table scale concurrently (t8 1.53M, t32 1.81M ops/s). resolve_metadata (no Table at all) is the ~0.5 µs floor. nornir-rest adds ~90–100 µs of HTTP+JSON over embedded and is the apples-to-apples REST baseline.

REST-vs-REST vs Nessie & Polaris — table_exists (pure catalog RPC, storage-free)

Full four-way run (2026-06-06, this box, full 10k-table scale):

REST-vs-REST nornir is ~8.0× lower p50 / ~9.2× more throughput than Polaris, and ~4.1× / ~4.4× vs Nessie; embedded is ~900× vs Polaris, ~460× vs Nessie (table_exists is answered from the lock-free pointer mirror — no redb read, no mutex). Polaris trails Nessie (per-request OAuth/RBAC). The JVMs' multi-ms p999 is the GC tail; nornir-rest's p999 stays at 87 µs.

Why table_exists and not loadTable for Nessie/Polaris: both reject (Nessie) or insecure-block (Polaris) a local file: warehouse for the table write path, and iceberg-rust 0.9.1 ships no S3 FileIO, so MinIO is blocked client-side too. table_exists is a pure catalog RPC with no storage, so it runs on every backend and isolates server latency. Polaris bootstrap (FILE feature flags + readiness override + OAuth + catalog create) is automated in containers/polaris_up.sh.

Durability sweep (BENCH_DURABILITY=immediate|eventual|none, create_table)

13.7k / 13.6k / 10.6k ops/sec — barely moves, because the metadata-file write dominates the redb fsync (see ../REDB_WISHLIST.md).