nordisk/skade

How to fold skade-sql back into skade

This crate is temporary by design. It exists for one external reason, and when that reason goes away it should be deleted, not maintained.

The one condition

Apache publishes a datafusion release on arrow 59.

Check it in one command — no guessing, no waiting on an announcement:

cargo add datafusion --dry-run   # or: look at the newest version on crates.io

then resolve it and read the arrow major it actually pulls:

cd /tmp && cargo new dfprobe && cd dfprobe
cargo add datafusion
cargo tree -e normal | grep -oE 'arrow-array v[0-9]+'

arrow-array v59 → fold back. arrow-array v58 → not yet, leave this crate alone.

Measured 2026-08-14: newest published datafusion is 54.1.0 and it resolves arrow-array v58.4, which is exactly why this crate exists. Also measured: iceberg-datafusion-arrow59 cannot be published either, for the same root cause — cargo publish refuses a git dependency with no version requirement.

Why it was pulled out at all

skade is on arrow 59. The only datafusion on arrow 59 is apache/datafusion git main, pinned by rev. A git dep with no version makes the whole crate unpublishable:

error: all dependencies must have a version requirement specified when publishing.
       dependency `datafusion` does not specify a version

An unpublishable skade is not a cosmetic problem. knut forbids sibling path deps (crates/knut-tracing/tests/no_sibling_path_deps.rs — a path that escapes the repo aborts resolution on any bare clone), so knut consumes skade from crates.io. While skade could not publish, knut kept resolving the old arrow-58 skade and carried two arrow_array majors in its graph.

So the datafusion edge moved out of skade's manifest, and skade ships again.

The fold-back, step by step

1. Move the two impls back. TableSql::sql returns to skade/src/table.rs and WarehouseSql::{session, sql} to skade/src/warehouse.rs, each as an inherent impl inside a #[cfg(feature = "sql")] mod sql { … } — that is the exact shape they had before, and git history has the originals:

git log --diff-filter=D --oneline -- skade/src/table.rs   # find the commit that removed them
git show <that commit>^:skade/src/table.rs

1. Restore the error variant. Put back in skade/src/error.rs:

#[cfg(feature = "sql")]
#[error("datafusion: {0}")]
DataFusion(#[from] datafusion::error::DataFusionError),

and drop skade_sql::df(), the helper that currently folds DataFusion errors into SkadeError::Other because the typed variant could not exist.

1. Restore the manifest. In skade/Cargo.toml:

datafusion = { version = "<the arrow-59 release>", optional = true }
iceberg-datafusion = { package = "iceberg-datafusion-arrow59", version = "…", optional = true }

[features]
sql = ["dep:datafusion", "dep:iceberg-datafusion"]

Note version, not git — that is the whole point of folding back. Publish iceberg-datafusion-arrow59 first; it becomes publishable at the same moment this crate becomes unnecessary, for the same reason.

1. Restore the re-exports in skade/src/lib.rs:

#[cfg(feature = "sql")]
pub use datafusion;
#[cfg(feature = "sql")]
pub use iceberg_datafusion;

and put sql back in the "Feature flags" section of the crate docs, replacing the "SQL is in skade-sql" section that points here.

1. Delete this crate. git rm -r skade-sql. Anything still importing skade_sql::{TableSql, WarehouseSql} deletes the use line — the methods are inherent again, so the call sites are unchanged.

2. Prove it, do not assume it. The point of the fold-back is a publishable skade with SQL:

cd skade && cargo publish --dry-run     # must reach "Uploading", not "does not specify a version"
cargo test --features sql
cargo tree --features sql | grep -oE 'arrow-array v[0-9]+' | sort -u   # must be v59 alone

What must NOT change when it folds back

The call sites. table.sql(q) and warehouse.sql(q) keep their names, signatures and behaviour; only the use skade_sql::{TableSql, WarehouseSql}; line disappears. If the fold-back tempts you into an API change, that is a separate commit — this one is a move.