Loading...
private.me Docs
Get Haystack xLink
private.me · Technical White Paper

Identity-Based Authentication for Haystack RAG Pipelines

Replace API keys in Haystack RAG pipelines with cryptographic identity. Zero-config setup, enterprise-grade access control, no cascading failures. Secure document retrieval with three-tier classification.

Zero-config setup 3-tier access control Information-theoretic security HIPAA / SOC2 ready
Section 01

Quick Start (15 seconds)

Get started with Haystack xLink in 15 seconds. Create a RAG pipeline with identity-based authentication and enterprise-grade access control.

Installation
npm install @private.me/haystack-xlink haystack
Basic Usage
import { HaystackXLinkRAG, createXLinkDocument } from '@private.me/haystack-xlink';

// Create RAG pipeline (generates identity automatically)
const rag = new HaystackXLinkRAG({ verbose: true });

// Add documents with access control
const doc = createXLinkDocument(
  'Sensitive information',
  rag.getDID(),
  'confidential'  // Access level: public | internal | confidential
);
await rag.addDocument(doc);

// Retrieve with identity-based access
const result = await rag.retrieve('query', agentDID);
// Returns only documents the requestor has access to
THAT'S IT
No API keys, no OAuth, no configuration files. The pipeline generates a cryptographic identity automatically and enforces access control on every query.
Section 02

The Problem: API Key Proliferation

Traditional RAG systems require credential management at every layer. Each credential can be lost, stolen, or rotated independently, causing cascading failures and outages.

Traditional Approach Problem
Database API Key Lost credentials → data unavailable
Document API Key Rotation → pipeline breaks
Vector DB Key Expiry → search fails
LLM Key Stolen key → unauthorized access
Auth Server Key Compromise → entire system at risk

The core issue: Each key is an independent failure point. Cascading failures occur when one expired token restarts hundreds of AI agents simultaneously.

Section 03

Solution: Identity-Based Authentication

xLink replaces all credentials with cryptographic identity. No API keys to manage, no tokens to refresh, no cascading failures.

Zero
Credential Overhead
3-Tier
Access Control
<1ms
Auth Overhead
100%
Audit Trail

How It Works

Each RAG pipeline has a cryptographic identity (DID). Documents are tagged with owner DIDs and access levels. Queries include the requestor's DID. The system verifies identity and returns only authorized documents.

Information Flow
┌─────────────┐
│ Agent A     │
│ DID: z6Mk.. │
└──────┬──────┘
       │
       ├── Query: "financial"
       │
       ▼
┌────────────────────────────┐
│ HaystackXLinkRAG           │
│ - Receives query           │
│ - Checks access_level      │
│ - Validates DID            │
│ - Returns filtered results │
└────────────────────────────┘
       │
       ├── Returns only documents
       │   Agent A has access to
       │
       ▼
┌─────────────┐
│ Results     │
│ Filtered    │
│ by access   │
└─────────────┘
Section 04

Features

Zero-Config Identity

Automatic Identity Generation
const rag = new HaystackXLinkRAG();
// Automatically:
// - Generates cryptographic key pair
// - Creates decentralized identifier (DID)
// - Ready for secure communication

Three-Tier Access Control

Level Accessible To Use Case
public Everyone Company handbook, privacy policy
internal Team members API docs, technical specs
confidential Owner + authorized Financial reports, customer data

Document Metadata

Access Control Example
const doc = createXLinkDocument(
  'Sensitive data',
  rag.getDID(),
  'confidential',
  {
    title: 'Q4 Revenue Report',
    category: 'financial',
    authorized_dids: ['did:key:z6Mkcollaborator...'],
  }
);

Identity Persistence

Export and Restore Identity
// Export identity
const identity = await rag.exportIdentity();
// Store securely...

// Restore later
const restored = await HaystackXLinkRAG.fromIdentity(identity.value);
// Same DID, same access rights
Section 05

Use Cases

🏦
Banking
Financial Data Retrieval

Retrieve sensitive financial data with cryptographic identity instead of API keys. Documents include account balances, transactions, loan applications. Access via cryptographic DID, not API keys. Zero credentials to compromise.

confidential access level
🏥
Healthcare
HIPAA-Compliant Document Access

Patient records, diagnoses, treatment plans with identity-based control. Doctor's DID verified cryptographically. Every access signed and logged. Satisfies HIPAA BAA requirements.

audit trail + BAA
🏛️
Government
Classified Intelligence

Multi-level classified document retrieval. Government employee DIDs with clearance verification. No shared keys or OAuth tokens to manage. Satisfies FISMA requirements.

3-tier classification
🏢
Enterprise
Internal Document Retrieval

Secure retrieval across teams and departments. Employee handbook, technical specs, customer data. Access by employee role and department (via authorized_dids). Compliance trail for SOC2, ISO27001.

role-based access
Section 06

Purchase & Subscription

Pricing

Standard pricing: $5/month Basic • $10/month Middle • $15/month Enterprise

Tier Price Features
Basic $5/month Haystack integration, identity auth for pipelines, component security
Middle $10/month Everything in Basic + advanced node authentication, custom pipeline policies
Enterprise $15/month Everything in Middle + air-gapped deployment + white-label

3-month trial - No credit card required

Start Free Trial

API Purchase Endpoint

Purchase via API
curl -X POST https://private.me/api/purchase \
  -H "Content-Type: application/json" \
  -d '{
    "product": "haystack-xlink",
    "tier": "basic",
    "customer": {
      "email": "user@example.com",
      "connection_id": "conn_abc123"
    }
  }'

Error Handling

All errors follow RFC 7807 Problem Details format:

RFC 7807 Error Response
{
  "type": "https://private.me/errors/invalid-tier",
  "title": "Invalid subscription tier",
  "status": 400,
  "fields": {
    "tier": "Provided value 'premium' is not valid"
  }
}
Section 07

API Reference

HaystackXLinkRAG

Main class for creating RAG pipelines with identity-based access control.

constructor(config?: HaystackXLinkRAGConfig)
Creates a new RAG pipeline with optional configuration. Generates identity automatically if not provided.
getDID(): string
Get the pipeline's decentralized identifier. Example: did:key:z6Mkh...
addDocument(doc): Promise<Result<string, Error>>
Add a document to the index. Returns document ID on success.
retrieve(query, requestorDID): Promise<Result<HaystackXLinkDocument[], Error>>
Retrieve documents matching query. Results filtered by access control based on requestorDID.
listDocuments(): HaystackXLinkDocument[]
List all documents in the index (for admin/monitoring). Does not enforce access control.
clear(): Promise<Result<void, Error>>
Clear all documents from the index.
exportIdentity(): Promise<Result<Uint8Array, Error>>
Export identity for persistence. Store securely (OS keychain, secrets manager).
static fromIdentity(pkcs8, config?): Promise<Result<HaystackXLinkRAG, Error>>
Create RAG from existing identity. Same DID, same access rights.

createXLinkDocument()

Helper Function
function createXLinkDocument(
  content: string,
  owner_did: string,
  access_level?: 'public' | 'internal' | 'confidential',
  metadata?: Record<string, unknown>
): HaystackXLinkDocument
Section 08

Architecture

Information Flow

┌─────────────┐
│ Agent A     │
│ DID: z6Mk.. │
└──────┬──────┘
       │
       ├── Query: "financial"
       │
       ▼
┌────────────────────────────┐
│ HaystackXLinkRAG           │
│ - Receives query           │
│ - Checks access_level      │
│ - Validates DID            │
│ - Returns filtered results │
└────────────────────────────┘
       │
       ├── Returns only documents
       │   Agent A has access to
       │
       ▼
┌─────────────┐
│ Results     │
│ Filtered    │
│ by access   │
└─────────────┘

Multi-Agent Document Sharing

Cross-Agent Access
// Create two independent agents
const finance = new HaystackXLinkRAG({ name: 'finance' });
const engineering = new HaystackXLinkRAG({ name: 'engineering' });

// Finance adds confidential document
const doc = createXLinkDocument(
  'Budget allocation for Q4',
  finance.getDID(),
  'confidential',
  { authorized_dids: [engineering.getDID()] }  // Grant access
);
await finance.addDocument(doc);

// Engineering can access the shared document
const result = await finance.retrieve('budget', engineering.getDID());
// Returns shared document
Section 09

Security Model

Five-Layer Security

  1. Document Storage: All documents stored with owner_did + access_level
  2. Query Processing: Incoming query includes requestor's DID
  3. Access Check: Server-side verification (not client-side)
  4. Result Filtering: Only matching documents with granted access returned
  5. Audit Trail: Every access logged with DID + timestamp

Security Considerations

  • Identity Storage: Store identity file securely (OS keychain, secrets manager)
  • Access Control: Always enable enforce_access_control: true in production
  • Document Metadata: Never store secrets in metadata (use encryption)
  • Query Validation: Sanitize search queries before document matching
  • Audit Logging: Log all document access with DID + timestamp
COMPLIANCE
The cryptographic model supports HIPAA, SOC2, and ISO27001 requirements. Actual compliance depends on deployment, audit logging, and operational practices.
Section 10

Performance

<5ms
Document Addition
<50ms
Query (1000 docs)
<1ms
Access Check
<100ms
Identity Generation

Scaling

  • Single instance: Handles up to 10,000 documents
  • Multiple instances: Share identity file (read-only mount)
  • Vector search: Integrate with Haystack vector backends
  • Distributed: Use xLink trust registry for federation