One ALTER TABLE for Millisecond Writes

A single-row INSERT into an Iceberg table is a strange thing to benchmark. The row is a few dozen bytes; the commit that lands it rewrites metadata, swaps a pointer, and waits for the catalog to say yes. On our own cluster that costs about 712 ms, and no amount of tuning changes the shape of it — you are paying for a catalog transaction, once per statement, whatever the statement contains.

That is fine for the workload Iceberg was built for. It is not fine for an application that edits one order, appends one comment, or increments one counter and then wants to read it back.

Gnok now takes a different path for those writes, and turning it on is one line of DDL:

ALTER TABLE orders SET TBLPROPERTIES ('gnok.write.mode' = 'command');

That write now acknowledges in 5–7 ms.

Where the time goes

Three paths exist, and the difference between them is when the write is durable enough to acknowledge.

Path	What it does	p50
direct (default)	Commits to the catalog per statement	712 ms
staged	Buffers small writes, commits in the background	12–15 ms
command	Commits to a replicated log, publishes to Iceberg after	5–7 ms

The last one is not a cache and not a queue you can lose. The write is replicated before it is acknowledged; Gnok publishes it into Iceberg in the background shortly after. What changed is the order of those two facts, not whether both happen.

The 712 ms is per statement, not per row — a bulk load of ten million rows pays it once and does not care. This is about the small write, and the small write is what an application does all day.

What the table has to say

The mode lives on the table, because it is a fact about the data:

Mode	Behavior
direct (default)	Ordinary Iceberg DML
command	Low-latency path; ordinary DML still permitted
command_only	Low-latency path; out-of-band DML refused

The middle value is the one that makes a migration possible. While you are moving writers over, both paths must work on the same table. When the last writer has moved, command_only shuts the door so nothing can modify the table behind the engine's back.

There is no service setting to change and nothing to keep in step elsewhere: every part of Gnok reads the mode from the same place you wrote it.

What a fast write has to prove

A write that is acknowledged before it reaches the table has to be replayable exactly once, and has to know which row it belongs to. So the engine asks three things of the statement:

BEGIN;
SET LOCAL gnok.idempotency_key = '01a06867-04e0-7f34-a545-d82c7e40e656';
SET LOCAL gnok.command_type    = 'order.edit';      -- optional label
SET LOCAL gnok.expected_version = 4;                -- optional compare-and-swap

MERGE INTO orders x
USING (SELECT CAST(1001 AS BIGINT) AS order_id,
              CAST(99.50 AS DECIMAL(10,2)) AS amount) s
   ON x.order_id = s.order_id
 WHEN MATCHED THEN UPDATE SET amount = s.amount
 WHEN NOT MATCHED THEN INSERT (order_id, amount) VALUES (s.order_id, s.amount);

COMMIT RETURNING;

An explicit transaction with an idempotency key — that key is the opt-in, and a retry with the same key returns the original answer rather than writing twice. A full-primary-key MERGE with every column assigned, so the row's identity and its post-image are both known without reading the table first. And a target the write path owns.

The expected_version carrier is optional and does what a compare-and-swap does: if the row moved since you read it, the write is refused as a stale version and your retry loop re-reads.

Reads see their own writes

The obvious objection to acknowledging early is that the next query will not see the row. It does. A read against a command table merges in the rows that are ordered but not yet published, so read-your-writes holds without waiting for the catalog. COMMIT RETURNING hands back a commit token if you want to require a specific write to be visible on a particular read.

What gets refused, and why that is the point

Some statements cannot be replayed exactly once, and the engine says so instead of quietly taking the slow path:

• a MERGE that leaves columns unassigned — the post-image is unknown;
• a MERGE whose ON clause is not the whole primary key — the row's identity is unknown;
• an UPDATE … WHERE over a predicate — it may match anything;
• two writes to the same table in one transaction — the second would have to read the first's uncommitted effect;
• any transaction touching a table outside the write path — refused whole, before anything is admitted, because an acknowledged write that cannot be published would block every write behind it.

Each refusal names what could not be proven and what to change. A write that is admitted is a write the engine can land.

When to leave it alone

direct is still right for most tables. Bulk loads amortize the catalog commit to nothing. Analytics tables often have no primary key, so there is no identity to key a command on. The low-latency path is for the transactional core of an application — the tables it edits one row at a time and reads back immediately — and for those, the difference between 712 ms and 6 ms is the difference between a background job and an interactive one.