RAG, Vector Indexing & Reranking

GalaxDB handles the entire retrieval pipeline - embed on write, index, search, cache - in a single SQL statement. No external vector database, no embedding API, no application-side join.

RAG

The standard RAG pattern maps directly to a single SEMANTIC_MATCH query. Store document chunks in a table with an EMBEDDING MODEL column, then retrieve by semantic similarity combined with any SQL filter.

-- 1. Create the table with an embedding column
CREATE TABLE docs (
    id     INT PRIMARY KEY,
    source TEXT,
    chunk  TEXT EMBEDDING MODEL 'sentence-transformers/all-MiniLM-L6-v2' DIM 384
);

-- 2. Insert chunks - embeddings are computed automatically
INSERT INTO docs (id, source, chunk)
VALUES (1, 'handbook', 'TLS certificates must be rotated every 90 days.');

-- 3. Retrieve: semantic search + metadata filter in one query
SELECT id, chunk
FROM docs
WHERE SEMANTIC_MATCH(chunk, 'how do I renew a TLS cert', 0.5)
  AND source = 'handbook'
LIMIT 5;

On step 3 the engine embeds the query text locally (no API call), runs HNSW beam search on the embedding column, applies the source filter in the same pass, and returns the top-5 by cosine similarity - one SQL call, no extra infrastructure.

Asymmetric encoding

Models like BGE-M3, Qwen3-Embedding, EmbeddingGemma, and LFM2.5-Embedding apply different instruction prefixes to stored documents vs search queries. Using the wrong prefix degrades recall. GalaxDB handles this automatically:

• On INSERT, the sidecar embeds with is_query: false (document prefix).
• On SEMANTIC_MATCH, the sidecar embeds with is_query: true (query prefix).

Symmetric models like all-MiniLM-L6-v2 ignore the flag. See Embedding Models for the full list.

Semantic cache

For repeated or near-identical RAG queries, the semantic cache avoids re-embedding and re-running HNSW. A cache hit returns the same results as a miss, skips the sidecar call and HNSW traversal entirely, and increments galaxdb_semantic_cache_hits_total on /metrics.

-- Serve queries within cosine similarity 0.97 of a cached query
-- Cache entries expire after 5 minutes (300 seconds)
CREATE SEMANTIC CACHE FOR TABLE docs SIMILARITY 0.97 TTL 300;

-- Remove the cache
DROP SEMANTIC CACHE FOR TABLE docs;

Cache entries are invalidated automatically on INSERT, UPDATE, or DELETE to the table, so stale results are never returned.

Vector indexing

HNSW (default)

Every EMBEDDING MODEL column uses HNSW by default - no extra DDL needed. The index is mutable: inserts land in a WAL-backed in-memory delta buffer first, then merge into the base HNSW graph when the buffer hits max(10,000, total_vectors × 1%).

Recall and latency on SIFT-1M (M=16, ef_construction=200, AWS c6id.4xlarge):

Quantization options: SQ8 (4x compression with AVX2/SIMD), FP16, RaBitQ. Build throughput: 15,295 vec/sec on 16 vCPUs (65.4 s for 1M vectors).

DiskANN (opt-in, v0.7+)

For vector sets larger than available RAM. The Vamana graph and full-precision vectors live on disk as fixed-size node records - only the nodes visited during beam search are loaded into a bounded in-memory cache, so RAM use stays constant regardless of index size.

Incremental inserts work via a FreshDiskANN-style in-memory delta: a new row is immediately findable in search without a full rebuild. consolidate() folds the delta into the disk graph periodically.

HNSW remains the default; DiskANN is strictly opt-in. Verified recall@10 ≥ 0.90 (cosine) and ≥ 0.85 (L2) against exact brute-force ground truth on real clustered data.

Adaptive planner

The query planner picks the search path automatically based on filter selectivity:

• Brute-force scan - when the SQL filter leaves fewer than 1,000 candidates (or less than 0.1% of the table). Exact cosine over a tiny set is faster than HNSW traversal.
• HNSW with post-filter - for everything else.

-- Update statistics used by the planner
ANALYZE docs;

Reranking

Every SEMANTIC_MATCH does a two-stage rerank internally - no extra step needed:

1. HNSW approximate search returns a candidate set (default: 200 candidates) with approximate cosine distances from the graph.
2. Exact rerank - the engine fetches the raw float32 vectors for those candidates from PAX blocks and recomputes exact cosine distance for each. Delta buffer candidates are already exact (brute-force). The union step deduplicates by row ID; the rerank step corrects the HNSW approximation error.

The final top-k is exact, not approximate. You do not need a separate cosine reranking call.

Historical vector search

For reproducible RAG - auditing what data a model was trained or evaluated on - you can run a semantic search as of a past snapshot:

-- Pin the current state
CREATE VERSION TAG 'rag-eval-v1' FOR TRAINING WITH TRAINING PRECISION 'float32';

-- Run the same RAG query against the historical state
-- Rows inserted after the snapshot are excluded
SELECT id, chunk
FROM docs
WHERE SEMANTIC_MATCH(chunk, 'how do I renew a TLS cert', 0.5)
AT VERSION 'rag-eval-v1' CONSISTENCY 'SEMANTIC_SNAPSHOT'
LIMIT 5;

The live index is never mutated. See Time-Travel Queries for more on version snapshots.