Vector search and RAG

Embed support documents, inspect vector-index metadata, retrieve relevant text, and check a grounded answer.

The practical problem and model

A support assistant needs to answer a refund question from the company's current policy. Literal word matching can miss a relevant passage when the question uses different wording. An embedding represents text as a numeric vector so related passages can be compared by distance.

Retrieval-augmented generation (RAG) first selects source documents, then asks a language model to answer from that context. Retrieval and answer generation are separate quality checks: a fluent answer can still be wrong if the retrieved material is irrelevant or outdated.

This example uses five synthetic documents. The refund and shipping policies are the two support documents, allowing you to verify both a metadata filter and the final answer.

Before you run

Complete the shared setup. This walkthrough uses synthetic data and recreates its tutorial objects. Use a sandbox tenant or schema, and run the steps in order.

Studio: Tutorial: Vector Search & RAG. Download the complete SQL.

EMBED() uses Gnok's embedding provider, within your organization's monthly embedding allowance; it needs no setup. The final AI_AGG answer also needs your organization's AI Governance. An organization administrator opens Operations → AI Governance in Studio, saves a monthly token budget as Available, and enables a rollout policy that allows provider anthropic, model claude-haiku-4-5-20251001, and operation ai_complete, with egress and retention approved. See AI Governance.

Step 1: Create the document corpus

Each row has text, metadata, and an initially empty embedding. The setup creates the catalog as well as the schema.

CREATE CATALOG IF NOT EXISTS tutorial;

CREATE SCHEMA IF NOT EXISTS tutorial.rag;

CREATE OR REPLACE TABLE tutorial.rag.documents (
    doc_id     BIGINT,
    title      VARCHAR,
    body       VARCHAR,
    category   VARCHAR,
    published  DATE,
    embedding  ARRAY<FLOAT>
);

INSERT INTO tutorial.rag.documents (doc_id, title, body, category, published) VALUES
    (1, 'Refund Policy',        'Customers may request a refund within 30 days of purchase by contacting support. Refunds are issued to the original payment method within 5 business days.', 'support',  DATE '2026-01-15'),
    (2, 'Shipping FAQ',         'Standard shipping takes 3-5 business days. Express shipping is 1-2 business days. International orders take 7-14 days.', 'support',  DATE '2026-02-01'),
    (3, 'Q4 Earnings Summary',  'Revenue grew 24% year-over-year driven by enterprise expansion. EBITDA margin improved to 18%.', 'finance', DATE '2026-02-10'),
    (4, 'Privacy & GDPR',       'We process customer data per GDPR Article 6 lawful bases. Right-to-erasure requests are honored within 30 days.', 'legal',  DATE '2025-11-22'),
    (5, 'Security Incident Q1', 'On 2026-01-08 a brute-force login attempt was detected and blocked. No customer data was accessed.', 'security', DATE '2026-01-12');

Step 2: Embed the documents and register an index

EMBED calls Gnok's embedding provider. All document vectors must use the same model and dimension. HNSW describes a graph-based approximate-neighbor index; the listing confirms that the index is registered.

UPDATE tutorial.rag.documents
SET embedding = EMBED(title || E'\n\n' || body)
WHERE embedding IS NULL;

CREATE VECTOR INDEX IF NOT EXISTS tutorial.rag.idx_documents_embedding
ON tutorial.rag.documents (embedding)
USING HNSW
WITH (m = 16, m_max_0 = 32, ef_construction = 200, metric = 'cosine');

SHOW VECTOR INDEXES;

Step 3: Compare semantic search with filtered retrieval

Smaller cosine distance means greater similarity. The first query considers five documents; the filtered query can return only the two support documents. Inspect EXPLAIN rather than assuming a particular physical access path.

SELECT
    doc_id,
    title,
    COSINE_DISTANCE(embedding, EMBED('How do I get a refund?')) AS distance
FROM tutorial.rag.documents
ORDER BY distance ASC
LIMIT 5;

EXPLAIN
SELECT doc_id, title, COSINE_DISTANCE(embedding, EMBED('How do I get a refund?')) AS distance
FROM tutorial.rag.documents
ORDER BY distance ASC LIMIT 5;

SELECT
    doc_id,
    title,
    published,
    COSINE_DISTANCE(embedding, EMBED('How do I get a refund?')) AS distance
FROM tutorial.rag.documents
WHERE category = 'support'
  AND published >= DATE '2026-01-01'
ORDER BY distance ASC
LIMIT 5;

Step 4: Generate an answer from retrieved passages

The CTE provides source text to AI_AGG. The prompt asks the model to stay within that text, but the answer must still be checked against the policies.

WITH retrieved AS (
    SELECT title, body
    FROM tutorial.rag.documents
    WHERE category = 'support'
    ORDER BY COSINE_DISTANCE(embedding, EMBED('How do I get a refund?')) ASC
    LIMIT 5
)
SELECT AI_AGG(
    'Answer the user question using ONLY the provided documents. If the documents do not contain the answer, say so. User question: How do I get a refund?',
    title || E'\n' || body
) AS answer
FROM retrieved;

What to check in the results

• All five rows receive embeddings; distance results must be finite and ordered from smallest to largest.
• SHOW VECTOR INDEXES includes tutorial.rag.idx_documents_embedding. The scoped form SHOW VECTOR INDEXES IN tutorial.rag is also supported.
• The filtered search returns only support documents. LIMIT 5 cannot create five results from two matching rows.
• A supported refund answer mentions the 30-day purchase window and contacting support. The policy also specifies the original payment method and processing within five business days; the answer must not invent a different policy.

Result correctness and index acceleration

The recorded tutorial run used exact distance scans. Its successful results and index lifecycle checks do not certify ANN acceleration or a VectorIndexScan plan. Check the plan and benchmark recall and latency on your deployment before making an acceleration claim.

Adapt it to real data

Split long documents into useful passages and retain source IDs, effective dates, and access controls. Build a question set with expected supporting passages to measure retrieval quality independently from answer quality. Re-embed consistently when changing embedding models.

Apply authorization before selecting context. Treat retrieved text as data, including any embedded instructions, and review answers for unsupported claims. Index size, recall, update frequency, and latency need representative workloads beyond this five-document fixture.

Related reading

Vector operations · BM25 and hybrid retrieval

All tutorials · Setup and troubleshooting