User-Defined Functions (UDFs)

Gnok supports user-defined functions written in WebAssembly (WASM) or registered as ONNX models. UDFs extend SQL with custom scalar and aggregate logic that runs inside the query engine with near-native performance.

Supported Languages

Language	Type	Description
WASM	Scalar	WebAssembly Component Model modules — write in Rust, Go, C/C++, or any language that compiles to WASM
ONNX	Scalar	Pre-trained ML models registered via CREATE MODEL for in-query inference
Native	Scalar	Built-in Rust functions shipped with Gnok

Writing a WASM UDF

1. Write the Function

Use the gnok-udf-guest library to write a UDF in Rust:

use gnok_udf_guest::ipc::{decode_input_batch, encode_output_column};
use arrow::array::{ArrayRef, StringArray};

pub fn evaluate(input_ipc: Vec<u8>) -> Result<Vec<u8>, String> {
    let args = decode_input_batch(&input_ipc)?;
    let emails = args[0].as_any().downcast_ref::<StringArray>()
        .ok_or("expected string array")?;

    let result: StringArray = emails.iter()
        .map(|v| v.map(|s| s.trim().to_lowercase()))
        .collect();

    encode_output_column(&result)
}

The guest library handles Arrow IPC serialization — your function receives Arrow arrays and returns an Arrow array.

2. Compile to WASM

cargo component build --target wasm32-wasip2 --release

3. Register the Function

Encode the .wasm binary as base64 and register it:

CREATE FUNCTION normalize_email(TEXT) RETURNS TEXT
LANGUAGE wasm
VOLATILITY immutable
AS 'AGFzbQEAAAA...base64-encoded-wasm-bytes...';

4. Use in Queries

SELECT normalize_email(email) AS clean_email
FROM customers;

CREATE FUNCTION

CREATE [OR REPLACE] FUNCTION name(arg_type1, arg_type2, ...)
RETURNS return_type
LANGUAGE wasm
[VOLATILITY immutable | stable | volatile]
AS 'base64-encoded-wasm-bytes';

Volatility

Volatility tells the optimizer how the function behaves:

Volatility	Meaning	Optimizer Behavior
immutable	Same inputs always produce the same output	Can be constant-folded, pushed down through joins
stable	Consistent within a single query	Can be cached per-query, pushed into scans
volatile	May return different results each call (default)	No optimization — re-evaluated every time

Supported Argument and Return Types

BOOLEAN, INT8, INT16, INT32, INT64, FLOAT32, FLOAT64, TEXT / UTF8 / STRING, DATE, TIMESTAMP

CREATE MODEL (ONNX)

Register a pre-trained ONNX model as a SQL function for in-query inference:

CREATE MODEL price_predictor(FLOAT64, FLOAT64, FLOAT64)
RETURNS FLOAT64
AS 'base64-encoded-onnx-model';

Use it like any scalar function:

SELECT
    property_id,
    price_predictor(sqft, bedrooms, bathrooms) AS predicted_price
FROM listings;

ONNX models execute via the ort runtime. Input columns are converted to float tensors, and output tensors are converted back to Arrow arrays. See ONNX Inference for GPU acceleration and batch inference details.

Aggregate UDFs

Gnok also supports aggregate UDFs with distributed state. Aggregate UDFs implement an Accumulator interface with:

• update_batch() — process a batch of input rows
• merge_batch() — merge partial state from another worker (for distributed aggregation)
• evaluate() — produce the final result
• state() — serialize partial state for shuffle between workers

DROP FUNCTION

DROP FUNCTION normalize_email;
DROP FUNCTION IF EXISTS normalize_email;

SHOW FUNCTIONS

List all registered functions:

SHOW FUNCTIONS;

Returns: function_name, language, return_type, volatility, arg_types.

Module Caching

WASM modules are compiled once and cached using a content-addressed (SHA-256) module cache. Subsequent calls reuse the pre-compiled module with near-native performance via Wasmtime. An optional disk-based AOT (ahead-of-time) cache avoids recompilation across restarts.

Sandboxing and Resource Limits

WASM UDFs run in isolated sandboxes with strict resource limits:

Limit	Default	Description
Memory per instance	64 MB	Max memory a single UDF invocation can use
Execution timeout	30 seconds	Epoch-based timeout enforcement
Max module size	50 MB	Maximum size of a .wasm binary

Per-tenant quotas can further restrict UDF usage:

Quota	Default	Description
Max functions	100	Maximum registered functions per tenant
Max storage	500 MB	Total WASM module storage per tenant

Distributed Execution

When a query using a WASM UDF runs on a distributed cluster, the coordinator sends the WASM module bytes inline with the fragment execution request via gRPC. Workers compile and cache the module locally on first use. ONNX models are distributed the same way.