nordisk/znippy

Arrow 58 + znippy-iceberg — for dummies

You ran cargo build and it pulled an iceberg crate out of a folder called iceberg-arrow58 next door. Here's what that's about, in plain words. The precise version is arrow58-iceberg-migration.md.

What happened, in one breath

znippy uses Arrow (a way of holding tables of data in memory). Arrow has major versions, and two of them don't mix — everything in one build has to agree. Most of znippy was already on the new Arrow 58. One crate, znippy-iceberg (the bit that writes your archive's metadata as a real on-disk Iceberg table), was lagging on the old Arrow 57, because it leans on two outside libraries — iceberg and skade — that were still on 57.

This change drags znippy-iceberg up to 58 so the whole workspace finally agrees. The trick: point its iceberg / skade dependencies at Arrow-58 copies sitting in sibling folders, instead of the (still Arrow-57) versions on the internet.

Who was already on 58, who wasn't

So if you only ever compress / decompress / list / get, nothing about your world changed. The Arrow-58 fork dance only matters when you build the Iceberg backend.

"What do I need on disk to build it?"

Two sibling folders next to your znippy checkout:

git/
├── znippy/                 ← you are here
├── iceberg-arrow58/        ← the Arrow-58 copy of iceberg
└── skade/                  ← the Arrow-58 skade (znippy points at ../skade/skade)

The wiring is one block in the root Cargo.toml:

[patch.crates-io]
iceberg       = { path = "../iceberg-arrow58" }
skade         = { path = "../skade/skade" }
skade-katalog = { path = "../skade" }

[patch.crates-io] just means "when something wants iceberg, hand it the copy in this folder instead of downloading it." That's the whole mechanism.

"Will it build?"

Yes:

$ cargo check -p znippy-iceberg
    Finished `dev` profile [unoptimized + debuginfo] target(s)

You'll see three warning: … to_naive_date … deprecated lines. Those come from inside the iceberg fork, not from znippy — Arrow 58 renamed a date function. They're warnings, not errors. Ignore them.

"Does the published crate care?"

No. [patch.crates-io] is local-only — cargo publish strips it. The znippy-iceberg on crates.io still names the normal iceberg 0.9.1 + skade 0.4.x. Only your local checkout gets the Arrow-58 forks.

"When does this go away?"

When the real upstream iceberg ships an Arrow-58 release and a new Arrow-58 skade is published. Then someone deletes those three [patch] lines, bumps the version numbers, and the sibling folders can be thrown away. Until then, keep the folders.

Where to go next

• The exact diff + verification → arrow58-iceberg-migration.md
• Where the Iceberg sink fits the archive format → design.md
• The fork itself → ../iceberg-arrow58/.nornir/README.md