xVault-RAG: Encrypted Enterprise RAG

Executive Summary

xVault-RAG brings information-theoretic security to retrieval-augmented generation. Every vector embedding and source document is XorIDA-split into K-of-N shares distributed across independent storage nodes — no single node holds enough data to reconstruct any document or its semantic representation.

Five modules handle the complete RAG pipeline: redact.ts strips PII before ingestion. embed.ts generates vector embeddings on-device (no data leaves the boundary). vault.ts performs XorIDA threshold sharing across all embeddings and documents. rag.ts orchestrates chunking, embedding, similarity search, and context assembly. store.ts provides a SQLite-backed PersistentKnowledgeBase with WAL mode for concurrent reads during background indexing.

When you query the knowledge base, similarity search reconstructs K-of-N shares to compute cosine similarity rankings, returns top-k chunks, and assembles context — all while maintaining information-theoretic security. An attacker who compromises any single storage node learns nothing about the corpus — not computationally hard to break, but mathematically impossible.

101 tests passing (75 core + 26 persistence). Production-ready SQLite storage with Write-Ahead Logging enables background indexing without blocking search queries. Typical RAG latency overhead: 10-55ms for split-search-reconstruct pipeline.

Developer Experience

xVault-RAG provides progress tracking and 30+ structured error codes to help developers build reliable, debuggable RAG systems with split-storage security.

Progress Callbacks

Both ingest() and search() operations support onProgress callbacks for tracking long-running operations, especially useful for large document corpora.

const kb = new PersistentKnowledgeBase({
  dbPath: './knowledge.db',
  threshold: { k: 2, n: 3 }
});

// Ingest with progress tracking
await kb.ingest(documents, {
  onProgress: async (event) => {
    switch (event.stage) {
      case 'redacting':
        console.log('Redacting PII...');
        break;
      case 'embedding':
        console.log(`Embedding chunk ${event.current}/${event.total}...`);
        break;
      case 'splitting':
        console.log('XorIDA splitting...');
        break;
      case 'storing':
        console.log('Writing to SQLite...');
        break;
      case 'complete':
        console.log('Ingestion complete');
        break;
    }
  }
});

// Search with progress tracking
const results = await kb.search(query, {
  topK: 5,
  onProgress: async (event) => {
    if (event.stage === 'reconstructing') {
      console.log(`Reconstructing ${event.current} chunks...`);
    }
  }
});

Structured Error Handling

xVault-RAG uses a Result<T, E> pattern with detailed error structures. Every error includes a machine-readable code, human-readable message, actionable hint, and documentation URL.

interface ErrorDetail {
  code: string;         // e.g., 'INVALID_THRESHOLD'
  message: string;      // Human-readable description
  hint?: string;        // Actionable fix suggestion
  field?: string;       // Field that caused the error
  docsUrl?: string;     // Link to documentation
}

// Example: handle ingestion errors
const result = await kb.ingest(documents);
if (!result.ok) {
  const { code, message, hint } = result.error.details;
  console.error(`${code}: ${message}`);
  if (hint) console.log(`Hint: ${hint}`);
}

Common Error Codes

The Problem

Enterprise RAG systems store sensitive corporate documents as vector embeddings in centralized databases, creating a single point of data exfiltration.

Vector databases contain semantic representations of every document in the corpus. A breach exposes not just the documents but their relationships, making corporate knowledge extraction trivial.

Encrypting the vector database at rest doesn't help — the database must be decrypted to perform similarity search. There is no standard way to search encrypted embeddings without decrypting them first.

The Old Way

Use Cases

Architecture

xVault-RAG provides a drop-in replacement for standard vector stores with XorIDA split-storage and threshold-reconstructed similarity search.

The New Way

Core Modules

The implementation consists of five main modules, each providing a critical piece of the RAG pipeline:

Persistence Layer

xVault-RAG uses SQLite with Write-Ahead Logging (WAL mode) for durable split-storage:

RAG Pipeline

Complete Flow

End-to-end RAG pipeline from document ingestion to context retrieval.

Ingestion Flow

1. Redaction — PII patterns (SSN, email, phone, credit card) stripped from document text
2. Chunking — Document split into 512-token chunks with 128-token overlap for context continuity
3. Embedding — Each chunk embedded on-device using local model (384-768 dimensions)
4. XorIDA Split — Embedding vector + chunk text split into K-of-N shares with HMAC-SHA256 tags
5. Storage — Shares written to SQLite with atomic transaction (all-or-nothing)

Search Flow

1. Query Embedding — User query embedded on-device using same model as corpus
2. Share Retrieval — Fetch K shares for each chunk from SQLite (concurrent reads via WAL)
3. HMAC Verification — Verify integrity of each share before reconstruction
4. Reconstruction — XOR K shares to recover original embedding vector
5. Similarity Ranking — Compute cosine similarity between query and reconstructed embeddings
6. Top-K Selection — Return highest-ranked chunks
7. Context Assembly — Concatenate chunk texts into final context for LLM

Integration

import { PersistentKnowledgeBase } from '@private.me/vaultrag';

// Create persistent knowledge base with SQLite + WAL mode
const kb = new PersistentKnowledgeBase({
  dbPath: './knowledge.db',
  threshold: { k: 2, n: 3 }
});

// Ingest documents (automatic chunking, embedding, XorIDA split)
await kb.ingest([
  { id: 'doc1', text: 'Corporate policy...' },
  { id: 'doc2', text: 'Technical manual...' }
]);

// Search with threshold reconstruction
const results = await kb.search('What is the security policy?', { topK: 5 });

// Close database connection
await kb.close();

Key API Methods

Security Properties

Benchmarks

Performance characteristics measured on Node.js 22, Apple M2. xVault-RAG adds 10-55ms to standard RAG pipelines while achieving information-theoretic protection of all retrieved context.

Test Coverage

xVault-RAG has 101 tests passing across the full implementation:

• 75 core tests — redaction, embedding, RAG pipeline, vault splitting, threshold reconstruction
• 26 persistence tests — SQLite storage, WAL mode, concurrent reads, atomic operations, database integrity

All tests verify both correctness (HMAC verification, reconstruction accuracy) and performance (sub-millisecond splitting, minimal search overhead).

RAG Architecture Comparison

Honest Limitations

Five known limitations documented transparently. xVault-RAG provides practical encrypted retrieval with explicit trade-offs.

Ship Proofs, Not Source

xVault-RAG generates cryptographic proofs of correct execution without exposing proprietary algorithms. Verify integrity using zero-knowledge proofs — no source code required.

Use Cases

Error Hierarchy

xVault-RAG uses a structured error taxonomy with actionable hints.

class VaultError extends Error {
  public readonly details: ErrorDetail;
  constructor(message: string, code: string, hint?: string, field?: string) {
    super(message);
    this.details = { code, message, hint, field, docsUrl: `https://private.me/docs/xvault-rag#${code}` };
  }
}

Error Categories

• Configuration Errors — INVALID_THRESHOLD, INVALID_CHUNK_SIZE, MISSING_DB_PATH
• Ingestion Errors — EMPTY_DOCUMENT, REDACTION_FAILED, EMBEDDING_FAILED, SPLIT_FAILED
• Storage Errors — DB_ERROR, WRITE_FAILED, TRANSACTION_FAILED
• Search Errors — RECONSTRUCTION_FAILED, HMAC_MISMATCH, QUERY_EMBEDDING_FAILED
• Validation Errors — MISSING_DOCUMENT_ID, INVALID_SEARCH_PARAMS, INVALID_TOP_K

Full API Surface

Complete TypeScript interface for PersistentKnowledgeBase.

interface PersistentKnowledgeBase {
  /** Ingest documents with automatic PII redaction, chunking, embedding, and XorIDA split-storage */
  ingest(documents: Document[], opts?: IngestOptions): Promise<Result<void, VaultError>>;

  /** Search for semantically similar chunks with threshold reconstruction */
  search(query: string, opts?: SearchOptions): Promise<Result<SearchResult[], VaultError>>;

  /** Delete all chunks and shares for a document (atomic) */
  delete(docId: string): Promise<Result<void, VaultError>>;

  /** Close database connection (flush WAL, release locks) */
  close(): Promise<void>;
}

interface Document {
  id: string;            // Unique document identifier
  text: string;          // Document content (will be redacted + chunked)
  metadata?: Record<string, any>; // Optional metadata (not embedded)
}

interface IngestOptions {
  chunkSize?: number;     // Token count per chunk (default: 512)
  overlap?: number;       // Token overlap between chunks (default: 128)
  onProgress?: ProgressCallback;
}

interface SearchOptions {
  topK?: number;          // Number of results to return (default: 5)
  onProgress?: ProgressCallback;
}

interface SearchResult {
  docId: string;          // Source document ID
  chunkIndex: number;     // Chunk position in document
  text: string;           // Reconstructed chunk text
  score: number;          // Cosine similarity score (0-1)
  metadata?: Record<string, any>; // Document metadata
}

Error Taxonomy

Complete list of error codes with triggers and fixes.

Codebase Stats

xVault-RAG is production-ready TypeScript with comprehensive test coverage.

Module Breakdown

Ready to deploy xVault-RAG?

Talk to Sol, our AI sales engineer, or book a live demo with our team.