nordisk/skade

skade pluggable object store — skade::ObjectStore

skade::ObjectStore is the backend-agnostic bytes-IO abstraction for the Iceberg data/metadata files a Warehouse reads and writes. It decouples skade from any single S3 client so a consumer (e.g. an isolated Njord mail-server) can run skade → Iceberg on a shared MinIO with the backend of its choice. skade/src/object_store.rs, wired in skade/src/warehouse.rs.

Why it exists

Before this, the published skade crate hardwired LocalFsStorageFactory in Warehouse::open — local disk only. The real S3 work (the s3 feature, the configurable MinIO endpoint) lived in the unpublished bench/ crate (bench/src/s3_storage.rs, backed by our own aegir client). A direct consumer of the crate had no way to point skade at an object store, and no way to pick which S3 client to use.

The Njord mail-server is isolated and already links aws-sdk-s3; it must write to a shared MinIO. Pulling a second S3 client (rust-s3) into that build is wasteful, and rust-s3 doesn't even compile in every workspace (see below). So skade needs a trait the consumer can satisfy with the SDK it already has.

The trait (skade/src/object_store.rs)

#[async_trait]
pub trait ObjectStore: Debug + Send + Sync {
    async fn put(&self, key: &str, bytes: Bytes) -> Result<()>;
    async fn get(&self, key: &str) -> Result<Bytes>;
    async fn list(&self, prefix: &str) -> Result<Vec<String>>;
    async fn delete(&self, key: &str) -> Result<()>;
    async fn exists(&self, key: &str) -> Result<bool>;

    // Provided (overridable) — S3 backends do a ranged GET / HEAD.
    async fn get_range(&self, key: &str, range: Range<u64>) -> Result<Bytes>;
    async fn size(&self, key: &str) -> Result<Option<u64>>;
}

Keys are opaque, slash-separated path-like strings. Result is skade::Result (one error type over the stack).

Backends (feature-gated)

default = [] — no S3 client is pulled unless asked for, and Warehouse::open is byte-for-byte unchanged (still LocalFsStorageFactory). Existing users and the existing local-path path are untouched.

Note on "rustfs"

The rustfs crate on crates.io (0.0.x) is a full distributed object-storage server, not an embeddable client — useless as a library dep. What "local object store" actually means for a consumer is a persistent store with no S3 server and no extra crates, so the rustfs feature ships LocalFsStore: keys → files under a root dir, std-only.

Note on rust-s3 / maybe-async

rust-s3 uses maybe-async. In a workspace that also links gix (which flips maybe-async to is_sync globally) rust-s3 will not compile — this is exactly why the bench crate used our own aegir client instead. The published skade crate has its own detached workspace, so the s3 feature builds cleanly for direct consumers. A consumer who does hit the gix conflict (or who already links aws-sdk-s3, like Njord) should use the aws-s3 backend.

How the warehouse uses it (the iceberg bridge)

iceberg-rust drives all blob IO through its own Storage / StorageFactory extension points (FileIOBuilder::new(factory)). skade bridges the trait into that machinery rather than re-plumbing the writer/reader:

ObjectStore (trait)
   └─ ObjectStoreBridge   : impl iceberg::io::Storage      (read/write/list/delete → trait)
        ▲
ObjectStoreFactory        : impl iceberg::io::StorageFactory (build() → bridge)
        │ given to
RedbCatalogBuilder::with_storage_factory(...)  →  FileIO  →  every data/metadata blob

The ObjectStore bridge: one trait, fronted into iceberg's Storage / FileIO, sends every data + metadata blob to a feature-gated backend while the redb catalog stays local.

Two constructors (both additive — open is unchanged):

// 1. From a live store (in-process; e.g. Njord's own aws-sdk-s3 client wrapped).
Warehouse::open_with_store(db_path, "s3://bucket/warehouse", Arc::new(store)).await?;

// 2. From a serializable config (settings/env-driven; backend built lazily).
Warehouse::open_with_object_store_config(
    db_path, "s3://bucket/warehouse",
    ObjectStoreConfig::AwsS3(S3Config { endpoint: "http://minio:9000".into(), .. }),
).await?;

The redb catalog file stays on local disk at db_path (it is a single-file ACID mmap, not an object); only the Iceberg data + metadata blobs go through the ObjectStore. The warehouse_uri is the location iceberg stamps into table metadata and hands back to the store on every IO.

typetag / serialization caveat (honest)

iceberg's StorageFactory and Storage are #[typetag::serde] — a built storage must survive a config round-trip. A live SDK client is not serializable, so ObjectStoreFactory serializes as a small ObjectStoreConfig enum and rebuilds the backend in build(), exactly as iceberg's own S3 FileIO does. Consequence: open_with_store (live store) is for the in-process flow; if iceberg ever serializes the factory and rebuilds it, the live store is dropped and a MemoryStore is reconstructed. For a durable, restart-safe S3 warehouse, prefer open_with_object_store_config (config-built, fully reconstructible).

Pointing at MinIO / rustfs / aws

use skade::{Warehouse, ObjectStoreConfig, S3Config};

let cfg = S3Config {
    endpoint: "http://minio:9000".into(),  // empty → real AWS
    region: "us-east-1".into(),
    bucket: "njord-mail".into(),
    access_key_id: "minio".into(),
    secret_access_key: "minio123".into(),
    path_style: true,                       // required for MinIO
};

// rust-s3 backend (feature "s3")
let wh = Warehouse::open_with_object_store_config(
    "/var/lib/njord/catalog.redb", "s3://njord-mail/warehouse",
    ObjectStoreConfig::RustS3(cfg.clone())).await?;

// aws-sdk-s3 backend (feature "aws-s3") — Njord reuses its own SDK
let wh = Warehouse::open_with_object_store_config(
    "/var/lib/njord/catalog.redb", "s3://njord-mail/warehouse",
    ObjectStoreConfig::AwsS3(cfg)).await?;

// embedded local store (feature "rustfs")
let wh = Warehouse::open_with_object_store_config(
    "/var/lib/njord/catalog.redb", "file:///var/lib/njord/store",
    ObjectStoreConfig::LocalFs { root: "/var/lib/njord/store".into() }).await?;

Flow note — Njord mail-server on shared MinIO

Njord (isolated)                    shared MinIO
  ├─ aws-sdk-s3 (already linked) ───────┐
  ├─ skade [features = ["aws-s3"]]      │
  │    Warehouse::open_with_object_store_config(
  │        catalog.redb (local disk),   │   redb stays local
  │        "s3://njord-mail/warehouse", │
  │        AwsS3(S3Config{ endpoint: minio })) 
  │            └─ AwsS3Store ──put/get/list──▶ s3://njord-mail/warehouse/...
  └─ table.append(batch) → parquet + metadata land in MinIO via the ONE SDK

One S3 SDK in the build, the catalog ACID on local disk, the lake on shared MinIO. That is the unblock.

Tests (inject-assert) — skade/tests/object_store.rs

No live MinIO required (CI-safe):

• memory_store_roundtrips / local_fs_store_roundtrips — inject 3 objects (incl. an empty one and a 4 KiB body), assert get returns the exact bytes, get_range slices exactly, list(prefix) returns exactly the matching keys, size matches the injected length, delete is precise + idempotent.
• warehouse_writes_and_reads_through_memory_store — a real Iceberg create + append(3 rows) + read where every blob lands in the injected MemoryStore (asserts a .parquet and a metadata key are present), and 3 rows come back through the trait.
• warehouse_roundtrips_through_local_fs_config (rustfs) — same end-to-end table round-trip via a serializable ObjectStoreConfig::LocalFs, asserting a parquet file landed under the configured store root (endpoint/root honored) and 2 rows read back.

cargo test --test object_store                  # MemoryStore + warehouse-over-memory
cargo test --features rustfs --test object_store # + LocalFsStore + config round-trip

The live-MinIO S3 backends (s3, aws-s3) are build-verified in CI; their wire behavior is exercised against a real MinIO out of band (they share the same ObjectStoreBridge + trait contract the memory/local tests pin).