Your browser sucks, JavaScript disabled! Gunnar's console needs JavaScript and WebAssembly. Git over SSH on port 2222 does not.

nordisk/skade

.nornir/guide.md

skade — guide (f0)

Class: guide · Function axis: f0 (the whole end-to-end session). Per-function how-tos: catalog-guide, read-path-guide, commit-guide, time-travel-guide, builder-guide. The what is in design.md; the why in reasoning.md.

f0 — zero to a working embedded lakehouse

[dependencies]
skade-katalog = "0.1"
iceberg = { package = "iceberg-arrow58", version = "0.9.1" }
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }

Six moves take you from nothing to committed, time-travellable tables:

  1. Open a catalog — one redb file + a warehouse root (builder-guide):
use std::sync::Arc;
use iceberg::CatalogBuilder;
use iceberg::io::LocalFsStorageFactory;
use skade_katalog::RedbCatalogBuilder;

let catalog = RedbCatalogBuilder::default()
    .db_path("wh/catalog.redb")
    .warehouse_location("file:///abs/path/wh")
    .with_storage_factory(Arc::new(LocalFsStorageFactory))
    .load("app", Default::default())
    .await?;
  1. Make a namespace — catalog.create_namespace(&ns, props).await? (catalog-guide).

  2. Create tables — catalog.create_table(&ns, creation).await?.

  3. Read them back — warm and lock-free — load_table / resolve_metadata / table_exists hit the L1/L0/L1.5 caches (read-path-guide).

  4. Commit changes — update_table for one table, or atomic_release([...]) to advance several tables all-or-nothing (commit-guide).

  5. Travel in time — load_table_at(&ident, snapshot_id) or the batch resolve_many(&ident, &ids) (time-travel-guide).

The whole lakehouse is the warehouse directory: catalog.redb plus the Parquet files it points at. Copy the directory to copy the lakehouse.

The iceberg::Catalog contract

RedbCatalog is an iceberg::Catalog, so anything generic over that trait (a scan planner, a REST shim, a downstream tool) works against it unchanged. Use the trait methods for portability; reach for the skade-only methods (resolve_metadata, commit_table, atomic_release, *_at, resolve_many) when you want the fast path or a capability the trait doesn't expose.

When to reach for which lever

you want… call
just check a table is there, cheapest table_exists
metadata (schema/snapshots) without building a Table resolve_metadata
a full Table handle load_table
commit N tables together, all-or-nothing atomic_release
max write throughput on a burst many concurrent update_table (auto group-commit) + relaxed durability
read one historical snapshot load_table_at / resolve_metadata_at
plan over many snapshots at once resolve_many