.nornir/tri-modal-catalog-guide.md · nordisk/skade

Serving the catalog three ways — guide

Class: guide · Function: tri-modal. The what/why is in tri-modal-catalog-design.md; the zero-knowledge version is tri-modal-catalog-for-idiots.md.

skade-katalog is one iceberg::Catalog over a single-file redb (RedbCatalog). The same catalog can be reached three ways. Pick by who is calling:

front	crate	wire	reach for it when
embedded	`skade-katalog`	none (link + call)	your code is co-located with the catalog — no hop, ~2.9M `table_exists`/s
REST	`skade-katalog-rest`	Iceberg OpenAPI / HTTP-1 + JSON	a foreign engine (Spark, PyIceberg, DuckDB, Trino) must read it
gRPC	`skade-katalog-grpc`	protobuf / HTTP-2	a nordisk Rust service (nornir, dwarves, korp, modgunn) wants the fast wire

All three are thin fronts over one RedbCatalog. The serving process owns the redb write-lock and serializes every commit; remote clients are lock-free.

The one-command server

$ skade-katalog serve --warehouse /var/lib/skade/wh --rest 0.0.0.0:8181 --grpc 0.0.0.0:8182
skade-katalog: REST on 0.0.0.0:8181, gRPC on 0.0.0.0:8182 (warehouse /var/lib/skade/wh)

One process, one RedbCatalog, both wires. A table created over gRPC is instantly loadable over REST — they are the same catalog, not two copies. Give it just one of --rest / --grpc to serve a single wire.

Embedding a front in your own process

Both fronts are libraries. serve blocks; spawn returns the bound address after binding (handy for tests / ephemeral ports):

use skade_katalog::RedbCatalogBuilder;
# async fn run(cat: skade_katalog::RedbCatalog) -> anyhow::Result<()> {
// REST (Iceberg OpenAPI)
let rest_addr = skade_katalog_rest::spawn(cat.clone(), "127.0.0.1:0".parse()?).await?;
// gRPC (skade.katalog.v1.Catalog)
let grpc_addr = skade_katalog_grpc::spawn(cat, "127.0.0.1:0".parse()?).await?;
# Ok(()) }

RedbCatalog is cheaply Clone (an Arc inside), and clones share the one redb file — so handing cat.clone() to each front keeps a single lock-owner and a single group-commit path.

Talking to it

REST: any stock Iceberg-REST client. With iceberg-rust: RestCatalogBuilder::default().load("c", {uri: http://host:8181, warehouse: …}).
gRPC: the generated skade.katalog.v1.Catalog client (skade_katalog_grpc::pb::catalog_client::CatalogClient), 11 RPCs mirroring the REST endpoints one-to-one (Config, {List,Create,Get}Namespace, NamespaceExists, ListTables, CreateTable, LoadTable, TableExists, DropTable, CommitTable).

How metadata + commits ride the wire

Iceberg's TableMetadata is not Serialize, but the on-disk metadata file is the canonical wire form — so both fronts splice the raw metadata JSON bytes into their load/commit responses rather than re-serializing. CommitTable carries the Iceberg requirements + updates (REST as a JSON body; gRPC as one canonical-JSON string per item) straight into RedbCatalog::commit_table, which checks every requirement against current metadata and persists through the optimistic group-commit path. A stale requirement (e.g. assert-table-uuid) is rejected as HTTP 409 (REST) / gRPC ABORTED (gRPC) — the optimistic-concurrency guard.