nordisk/knut

networkx-export — FalkorDB → Arrow Flight SQL → pandas → NetworkX (the Python client)

Class: design. The Python-land counterpart to the Flight SQL server: where graphar-flight serves Cypher in, Arrow out, python/knut is the thin client that pulls those Arrow rows into a NetworkX graph for analysis in Python. Reuse: python/knut depends only on pyarrow/pandas/networkx — no knut Rust code, no extra graph library. Every conversion is a single library call on data that is already Arrow. Source: python/knut/client.py (KnutClient.to_networkx), example python/examples/social_graph.py, tests python/tests/test_networkx.py.

What to_networkx does

KnutClient.to_networkx(vertex_query, edge_query, …) runs two registered Cypher queries against a graphar-flight server — one returning vertices, one returning edges — and returns a NetworkX graph with node and edge attributes carried across verbatim. It is the one call that turns a live FalkorDB graph into an in-memory object you can run graph algorithms on (shortest path, centrality, components, …) without leaving Python.

It is a read convenience built on the client's query/query_df primitives; it adds no wire protocol of its own.

The path — FalkorDB → Arrow → pandas → NetworkX

No intermediate format is invented; each hop is a library boundary that already speaks the next stage's native type.

1. FalkorDB → Arrow. query(cypher) sends the Cypher as a Flight SQL CommandStatementQuery (encoded by the pure-Python _proto helper), does get_flight_info then do_get, and read_all()s the returned stream into a pyarrow.Table. The Cypher string must match a key registered in the server's SchemaRegistry (see flight-design.md, law L1).
2. Arrow → pandas. query_df is query(...).to_pandas() — zero-copy where Arrow allows.

3. pandas → NetworkX. The two frames are handed to the pure builder _build_networkx:

   • nx.from_pandas_edgelist(edge_df, source=src_col, target=dst_col, edge_attr=…, create_using=…) builds the base graph. Every edge column that is not src/dst becomes an edge attribute; a src/dst-only frame yields plain edges (edge_attr=None).
   • nx.set_node_attributes(G, vertex_df.set_index(node_id_col).to_dict("index")) attaches every remaining vertex column as a node attribute, keyed by _gar_id.

to_networkx is deliberately split into fetch (query_df ×2, the only I/O) and build (_build_networkx, pure) so the build is testable offline (see Testing) and reusable when a caller already holds frames from elsewhere.

API

from knut import KnutClient

client = KnutClient(host="localhost", port=50051)   # +tls / token / tls_root_certs for prod

G = client.to_networkx(
    vertex_query="MATCH (n:Person) RETURN n._gar_id AS _gar_id, n.name, n.age",
    edge_query=(
        "MATCH (a:Person)-[r:KNOWS]->(b:Person) "
        "RETURN a._gar_id AS src, b._gar_id AS dst, r.since"
    ),
    node_id_col="_gar_id",   # vertex column used as the NetworkX node id
    src_col="src",           # edge endpoint columns
    dst_col="dst",
    directed=True,           # DiGraph (default) or undirected Graph
)

Contract: node count = distinct ids appearing across vertices and edge endpoints; edge count = rows in the edge frame; every non-key column is preserved as an attribute; direction is honoured (a DiGraph never invents the reverse edge). _gar_id is the same 0-based id used on the write path (insert_vertices/insert_edges) — the two round-trip.

Underlying primitives, if you want raw frames instead of a graph:

• client.query(cypher) -> pyarrow.Table
• client.query_df(cypher) -> pandas.DataFrame
• KnutClient._build_networkx(vertex_df, edge_df, …) -> nx.Graph — the pure builder, no I/O.

Example

python/examples/social_graph.py is the full round-trip: insert four Person vertices and four KNOWS edges, fetch them back, build a DiGraph, then print counts, per-node in-degree centrality, and the shortest path Alice → Dave. Run it against a graphar-flight server on localhost:50051 with the two schemas from the file's header docstring registered.

When to use it

• Graph analysis in Python land — you want NetworkX's algorithm library (paths, centrality, connectivity, community detection) over a graph that lives in FalkorDB.
• Notebook / ad-hoc exploration — pull a subgraph named by a registered query and poke at it.
• A bridge to the wider PyData stack — the intermediate query_df frames drop straight into pandas/NumPy/scikit-learn.

Reach for the Flight SQL server surface directly (ADBC / Power BI / raw Arrow) when you want tabular rows rather than a graph object, or when the working set is too large to hold in memory as a NetworkX graph — NetworkX is an in-memory, single-process structure.

Testing

python/tests/test_networkx.py exercises the real to_networkx offline: a FixtureClient subclasses KnutClient, skips the real FlightClient, and overrides query to serve canned pyarrow.Tables keyed by the Cypher string. Everything above that seam — query_df.to_pandas, to_networkx, _build_networkx — is the genuine production code. The suite asserts node/edge counts, node and edge attribute preservation, directedness (the reverse edge is absent), the undirected option, custom column names, and the same Alice → Dave shortest path the example prints; the pure _build_networkx seam is also tested directly from DataFrames.

Only the gRPC/DoGet hop is stubbed. The live-FalkorDB leg (Cypher execution, registry lookup, the Flight round-trip) is covered by the Rust integration tests in crates/graphar-flight/tests and is not re-proven from Python.

PDF

knut's docs render to PDF via nornir docs book knut (see docs.toml). The <!-- nornir:class: design --> marker at the top of this file routes it into the manual book (docs/manual.pdf) alongside the other design chapters.

Two DB-free siblings

This doc's road is live FalkorDB → Flight SQL → pandas → networkx. Two sibling roads reach networkx from the rel2graph output directly, no database:

• File export — rel2graph-networkx-design.md: the GraphMutation dumped as a node-list + edge-list + oracle JSON that demo/knut_networkx.py loads into a MultiDiGraph and asserts against.
• Arrow Flight — flight-graph-server-design.md: a plain (non-SQL) Arrow Flight server (knut-flight-server, :50055) that streams the same nodes/edges as Arrow to the same loader.

All three end in a networkx graph; the two siblings additionally prove they agree with knut's correctness oracle. Runbook for both: graph-to-python-guide.md.

See also

• The server this client talks to: flight-design.md / flight-guide.md.
• The whole-system map: design.md (f0) · workflow: guide.md (f0).