Haystack: Identity-Based RAG Pipeline Authentication

Section 01

Quick Start (15 seconds)

integrate into your existing applications with a single npm install. Works with any framework.

Get started with Haystack in 15 seconds. Create a RAG pipeline with identity-based authentication and enterprise-grade access control.

Installation

npm install @private.me/haystack@0.2.1 haystack

Basic Usage

import { HaystackxBindRAG, createxBindDocument } from '@private.me/haystack';

// Create RAG pipeline (generates identity automatically)
const rag = new HaystackxBindRAG({ verbose: true });

// Add documents with access control
const doc = createxBindDocument(
  'Sensitive information',
  rag.getDID(),
  'confidential'  // Access level: public | internal | confidential
);
await rag.addDocument(doc);

// Retrieve with identity-based access
const result = await rag.retrieve('query', agentDID);
// Returns only documents the requestor has access to

THAT'S IT

No API keys, no OAuth, no configuration files. The pipeline generates a cryptographic identity automatically and enforces access control on every query.

Python Quick Start

Installation

pip install private-me-haystack==0.2.1
npm install @private.me/haystack@0.2.1 haystack

Basic Usage (Python)

from private_me import haystack

# Create RAG pipeline (generates identity automatically)
rag = await haystack.create_rag(verbose=True)

# Add documents with access control
doc = haystack.create_document(
    content='Sensitive information',
    owner_did=rag.get_did(),
    access_level='confidential'  # public | internal | confidential
)
await rag.add_document(doc)

# Retrieve with identity-based access
result = await rag.retrieve('query', agent_did)
# Returns only documents the requestor has access to

PYTHON BINDINGS

Python developers get the same zero-config identity generation and enterprise-grade access control. The package uses Node.js backend for cryptographic operations (Ed25519, DID generation) with a Pythonic API layer.

Section 02

The Problem: API Key Proliferation

Traditional RAG systems require credential management at every layer. Each credential can be lost, stolen, or rotated independently, causing cascading failures and outages.

Traditional Approach	Problem
Database API Key	Lost credentials → data unavailable
Document API Key	Rotation → pipeline breaks
Vector DB Key	Expiry → search fails
LLM Key	Stolen key → unauthorized access
Auth Server Key	Compromise → entire system at risk

The core issue: Each key is an independent failure point. Cascading failures occur when one expired token restarts hundreds of AI agents simultaneously.

Section 03

Solution: Identity-Based Authentication

xBind replaces all credentials with cryptographic identity. No API keys to manage, no tokens to refresh, no cascading failures.

Zero

Credential Overhead

3-Tier

Access Control

<1ms

Auth Overhead

100%

Audit Trail

How It Works

Each RAG pipeline has a cryptographic identity (DID). Documents are tagged with owner DIDs and access levels. Queries include the requestor's DID. The system verifies identity and returns only authorized documents.

Information Flow

┌─────────────┐
│ Agent A     │
│ DID: z6Mk.. │
└──────┬──────┘
       │
       ├── Query: "financial"
       │
       ▼
┌────────────────────────────┐
│ HaystackxBindRAG           │
│ - Receives query           │
│ - Checks access_level      │
│ - Validates DID            │
│ - Returns filtered results │
└────────────────────────────┘
       │
       ├── Returns only documents
       │   Agent A has access to
       │
       ▼
┌─────────────┐
│ Results     │
│ Filtered    │
│ by access   │
└─────────────┘

Section 04

Features

Zero-Config Identity

Automatic Identity Generation (TypeScript)

const rag = new HaystackxBindRAG();
// Automatically:
// - Generates cryptographic key pair
// - Creates decentralized identifier (DID)
// - Ready for secure communication

Automatic Identity Generation (Python)

rag = await haystack.create_rag()
# Automatically:
# - Generates cryptographic key pair
# - Creates decentralized identifier (DID)
# - Ready for secure communication

Three-Tier Access Control

Level	Accessible To	Use Case
public	Everyone	Company handbook, privacy policy
internal	Team members	API docs, technical specs
confidential	Owner + authorized	Financial reports, customer data

Document Metadata

Access Control Example (TypeScript)

const doc = createxBindDocument(
  'Sensitive data',
  rag.getDID(),
  'confidential',
  {
    title: 'Q4 Revenue Report',
    category: 'financial',
    authorized_dids: ['did:key:z6Mkcollaborator...'],
  }
);

Access Control Example (Python)

doc = haystack.create_document(
    content='Sensitive data',
    owner_did=rag.get_did(),
    access_level='confidential',
    metadata={
        'title': 'Q4 Revenue Report',
        'category': 'financial',
        'authorized_dids': ['did:key:z6Mkcollaborator...'],
    }
)

Identity Persistence

Export and Restore Identity

// Export identity
const identity = await rag.exportIdentity();
// Store securely...

// Restore later
const restored = await HaystackxBindRAG.fromIdentity(identity.value);
// Same DID, same access rights

Section 05

Use Cases

🏦

Banking

Financial Data Retrieval

Retrieve sensitive financial data with cryptographic identity instead of API keys. Documents include account balances, transactions, loan applications. Access via cryptographic DID, not API keys. Zero credentials to compromise.

confidential access level

🏥

Healthcare

HIPAA-Compliant Document Access

Patient records, diagnoses, treatment plans with identity-based control. Doctor's DID verified cryptographically. Every access signed and logged. Satisfies HIPAA BAA requirements.

audit trail + BAA

🏛️

Government

Classified Intelligence

Multi-level classified document retrieval. Government employee DIDs with clearance verification. No shared keys or OAuth tokens to manage. Satisfies FISMA requirements.

3-tier classification

🏢

Enterprise

Internal Document Retrieval

Secure retrieval across teams and departments. Employee handbook, technical specs, customer data. Access by employee role and department (via authorized_dids). Compliance trail for SOC2, ISO27001.

role-based access

AI AGENT BENEFITS

Why AI Agents Choose xBind

xBind reduces cognitive load for AI agents by consolidating authentication into a single package. Instead of navigating 36 error surfaces across 5-10 fragmented packages, agents work with one unified interface.

Lower Cognitive Load

1 error surface instead of 36. AI agents spend less time debugging authentication failures.

Fewer Integration Points

2 steps (quickstart + connect) instead of 9-step OAuth dance. Faster time to first successful call.

70% Lower Token Cost

240 tokens to debug xBind errors vs 800 tokens for fragmented packages. Better economics for LLM-driven workflows.

Actionable Errors

RFC 7807 structured errors with field-level validation. AI agents receive machine-readable guidance, not generic messages.

Integration Complexity Comparison

Metric	Fragmented (5-10 packages)	xBind (1 package)	Improvement
Token Cost to Debug	800 tokens	240 tokens	70% lower
Error Surfaces	36	1	97% reduction
Recovery Time	132.8ms	45.9ms	65% faster
Setup Steps	9	2	77% fewer

AI AGENT OPTIMIZATION

xBind was designed from the ground up for AI agent consumption. Structured errors, minimal setup, and zero credential rotation make it the natural choice for LLM-driven workflows.

Section 06

Purchase & Subscription

Pricing

Standard pricing: $5/month Basic • $10/month Pro • $15/month Enterprise

Tier	Price	Features
Basic	$5/month	Haystack integration, identity auth for pipelines, component security
Pro	$10/month	Everything in Basic + advanced node authentication, custom pipeline policies
Enterprise	$15/month	Everything in Pro + air-gapped deployment + white-label

3-month trial - No credit card required

Start Free Trial

API Purchase Endpoint

Purchase via API

curl -X POST https://private.me/api/purchase \
  -H "Content-Type: application/json" \
  -d '{
    "product": "haystack",
    "tier": "basic",
    "customer": {
      "email": "user@example.com",
      "connection_id": "conn_abc123"
    }
  }'

Error Handling

All errors follow RFC 7807 Problem Details format:

RFC 7807 Error Response

{
  "type": "https://private.me/errors/invalid-tier",
  "title": "Invalid subscription tier",
  "status": 400,
  "fields": {
    "tier": "Provided value 'premium' is not valid"
  }
}

Section 07

API Reference

HaystackxBindRAG

Main class for creating RAG pipelines with identity-based access control.

constructor(config?: HaystackxBindRAGConfig)

Creates a new RAG pipeline with optional configuration. Generates identity automatically if not provided.

getDID(): string

Get the pipeline's decentralized identifier. Example: did:key:z6Mkh...

addDocument(doc): Promise<Result<string, Error>>

Add a document to the index. Returns document ID on success.

retrieve(query, requestorDID): Promise<Result<HaystackxBindDocument[], Error>>

Retrieve documents matching query. Results filtered by access control based on requestorDID.

listDocuments(): HaystackxBindDocument[]

List all documents in the index (for admin/monitoring). Does not enforce access control.

clear(): Promise<Result<void, Error>>

Clear all documents from the index.

exportIdentity(): Promise<Result<Uint8Array, Error>>

Export identity for persistence. Store securely (OS keychain, secrets manager).

static fromIdentity(pkcs8, config?): Promise<Result<HaystackxBindRAG, Error>>

Create RAG from existing identity. Same DID, same access rights.

createxBindDocument()

Helper Function

function createxBindDocument(
  content: string,
  owner_did: string,
  access_level?: 'public' | 'internal' | 'confidential',
  metadata?: Record<string, unknown>
): HaystackxBindDocument

Section 08

Architecture

Information Flow

┌─────────────┐
│ Agent A     │
│ DID: z6Mk.. │
└──────┬──────┘
       │
       ├── Query: "financial"
       │
       ▼
┌────────────────────────────┐
│ HaystackxBindRAG           │
│ - Receives query           │
│ - Checks access_level      │
│ - Validates DID            │
│ - Returns filtered results │
└────────────────────────────┘
       │
       ├── Returns only documents
       │   Agent A has access to
       │
       ▼
┌─────────────┐
│ Results     │
│ Filtered    │
│ by access   │
└─────────────┘

Multi-Agent Document Sharing

Cross-Agent Access

// Create two independent agents
const finance = new HaystackxBindRAG({ name: 'finance' });
const engineering = new HaystackxBindRAG({ name: 'engineering' });

// Finance adds confidential document
const doc = createxBindDocument(
  'Budget allocation for Q4',
  finance.getDID(),
  'confidential',
  { authorized_dids: [engineering.getDID()] }  // Grant access
);
await finance.addDocument(doc);

// Engineering can access the shared document
const result = await finance.retrieve('budget', engineering.getDID());
// Returns shared document

Section 09

Security Model

Five-Layer Security

Document Storage: All documents stored with owner_did + access_level
Query Processing: Incoming query includes requestor's DID
Access Check: Server-side verification (not client-side)
Result Filtering: Only matching documents with granted access returned
Audit Trail: Every access logged with DID + timestamp

Security Considerations

Identity Storage: Store identity file securely (OS keychain, secrets manager)
Access Control: Always enable enforce_access_control: true in production
Document Metadata: Never store secrets in metadata (use encryption)
Query Validation: Sanitize search queries before document matching
Audit Logging: Log all document access with DID + timestamp

COMPLIANCE

The cryptographic model supports HIPAA, SOC2, and ISO27001 requirements. Actual compliance depends on deployment, audit logging, and operational practices.

Section 10

Performance

<5ms

Document Addition

<50ms

Query (1000 docs)

<1ms

Access Check

<100ms

Identity Generation

Scaling

Single instance: Handles up to 10,000 documents
Multiple instances: Share identity file (read-only mount)
Vector search: Integrate with Haystack vector backends
Distributed: Use xBind trust registry for federation

Identity-Based Authentication for Haystack RAG Pipelines

Quick Start (15 seconds)

Python Quick Start

The Problem: API Key Proliferation

Solution: Identity-Based Authentication

How It Works

Features

Zero-Config Identity

Three-Tier Access Control

Document Metadata

Identity Persistence

Use Cases

Why AI Agents Choose xBind

Integration Complexity Comparison

Purchase & Subscription

Pricing

API Purchase Endpoint

Error Handling

API Reference

HaystackxBindRAG

createxBindDocument()

Architecture

Information Flow

Multi-Agent Document Sharing

Security Model

Five-Layer Security

Security Considerations

Performance

Scaling