Your browser sucks, JavaScript disabled! Gunnar's console needs JavaScript and WebAssembly. Git over SSH on port 2222 does not.

nordisk/skade

.nornir/builder-guide.md

Builder & configuration — guide

Class: guide · Function: builder. The what is in builder-design.md.

The typed builder

use std::sync::Arc;
use iceberg::CatalogBuilder;
use iceberg::io::LocalFsStorageFactory;
use skade_katalog::{RedbCatalogBuilder, WriteDurability};

let catalog = RedbCatalogBuilder::default()
    .db_path("wh/catalog.redb")                     // required
    .warehouse_location("file:///abs/wh")           // required
    .with_storage_factory(Arc::new(LocalFsStorageFactory))  // required
    .metadata_cache_bytes(256 * 1024 * 1024)        // optional (default 128 MiB)
    .table_handle_cache_bytes(512 * 1024 * 1024)    // optional (default 256 MiB)
    .durability(WriteDurability::Immediate)         // optional (default Immediate)
    .load("app", Default::default())
    .await?;

db_path's parent directory is created for you. Opening a fresh file initialises all five redb tables; opening an existing one warm-starts the L1 pointer mirror and the Ragnar index from what's already there.

Configuring from a property map

Everything is also a property, so you can drive it entirely from load(name, props) — handy when config comes from a file or env:

use skade_katalog::*;
let props = HashMap::from([
    (REDB_CATALOG_PROP_DB_PATH.into(), "wh/catalog.redb".into()),
    (REDB_CATALOG_PROP_WAREHOUSE.into(), "file:///abs/wh".into()),
    (REDB_CATALOG_PROP_METADATA_CACHE_BYTES.into(), "268435456".into()),
    (REDB_CATALOG_PROP_DURABILITY.into(), "eventual".into()),
]);
let catalog = RedbCatalogBuilder::default()
    .with_storage_factory(Arc::new(LocalFsStorageFactory))
    .load("app", props).await?;

A property overrides the matching typed setter. Numeric / enum properties are validated at load — a bad value fails with DataInvalid naming the offending input (e.g. a non-integer cache size, or a durability that isn't immediate|eventual|none).

Choosing durability

use case durability
normal operation, crash-safety matters Immediate (default)
high write throughput, can lose the last few commits on a hard crash Eventual
one-shot bulk import / benchmarking None (note: the file grows — redb only frees pages at higher levels)

Disabling a cache

Pass 0: metadata_cache_bytes(0) disables L0 (every metadata read hits FileIO); table_handle_cache_bytes(0) disables L1.5 (every load_table rebuilds the handle). The L1 pointer mirror is always on and cannot be disabled — it is the catalog's source of latest-state truth in memory.