2 posts tagged with "Performance"

A single-row INSERT into an Iceberg table is a strange thing to benchmark. The row is a few dozen bytes; the commit that lands it rewrites metadata, swaps a pointer, and waits for the catalog to say yes. On our own cluster that costs about 712 ms, and no amount of tuning changes the shape of it — you are paying for a catalog transaction, once per statement, whatever the statement contains.

That is fine for the workload Iceberg was built for. It is not fine for an application that edits one order, appends one comment, or increments one counter and then wants to read it back.

Gnok now takes a different path for those writes, and turning it on is one line of DDL:

ALTER TABLE orders SET TBLPROPERTIES ('gnok.write.mode' = 'command');

That write now acknowledges in 5–7 ms.

We recently needed to land a large pile of JSON documents — roughly 50 GB across ~70 datasets, including a couple of 15+ GB monsters — into Gnok so it could be queried with SQL alongside everything else. The documents were schemaless and deeply nested; Gnok tables are columnar Iceberg. This post walks through the pipeline we landed on, and the handful of gotchas that shaped it.