Loading...
private.me Docs
Get xTrace
PRIVATE.ME · Technical White Paper

Xtrace: Tamper-Evident Audit Trails

HMAC-chained distributed audit logs with optional XorIDA split storage. Each entry is cryptographically chained to its predecessor via SHA-256 hashing, HMAC-signed for independent verification, and optionally split across storage nodes for compliance and fault tolerance. Append-only JSONL format with zero npm dependencies.

v0.1.0 Building Block Crypto 0 npm deps <10ms chain append Info-theoretic split
Section 01

Executive Summary

Xtrace creates audit trails where tampering with any entry breaks the cryptographic chain for all subsequent entries, making unauthorized modifications immediately detectable.

Every audit entry is HMAC-SHA256 signed for independent verification. Each entry's hash depends on the previous entry's hash via SHA-256(prevHash + JSON(entry)), creating a tamper-evident chain. Modifying any entry invalidates the chain from that point forward.

For distributed compliance architectures, XorIDA split storage divides each entry into threshold shares across N storage nodes. Any k-of-N nodes can reconstruct the entry, but fewer than k shares reveal zero information about the plaintext — not computationally hard to break, but mathematically impossible.

Append-only JSONL storage ensures immutability at the filesystem level. Zero npm runtime dependencies. Works anywhere the Web Crypto API is available — Node.js, Deno, Bun, Cloudflare Workers.

Section 02

The Problem

Traditional audit logs are mutable text files. An attacker with write access can delete entries, modify timestamps, or rewrite entire logs without leaving evidence.

Logs are often unverified. Standard logging libraries write plaintext to disk. There's no built-in proof that log entries haven't been altered since creation.

Centralized logs are single points of failure. One compromised log server means the entire audit trail can be tampered with or destroyed.

Compliance requires tamper-evidence. HIPAA, SOC 2, ISO 27001, and SEC 17a-4 all require audit trails that detect unauthorized modifications. Standard logs don't provide cryptographic proof.

Distributed logs lack integrity. Replicating logs across nodes for availability doesn't prevent tampering if all replicas can be modified in sync.

Property Standard Logs Signed Logs Blockchain Xtrace
Tamper detection No Single sigs Yes Chained HMAC
Distributed storage Replicas Replicas P2P consensus XorIDA split
Single-node compromise Full loss Full loss Partial Zero info
Verification speed N/A Per-entry Full chain <1ms/entry
Independent verification No HMAC Requires sync HMAC + chain
Setup complexity Trivial Moderate High 5 lines
npm dependencies 1-5 10+ 50+ 0
Why not blockchain?
Blockchain provides tamper-evidence via consensus but requires peer-to-peer synchronization, proof-of-work or stake, and heavyweight dependencies. Xtrace provides the same tamper-evidence guarantee with SHA-256 chaining and HMAC signatures — no consensus protocol, no mining, zero npm dependencies.
Section 03

Real-World Use Cases

Six scenarios where tamper-evident audit trails are compliance or security requirements.

🏥
Healthcare
HIPAA Access Logs

Every PHI access generates a chained audit entry. HIPAA requires tamper-evident logs. Split storage across 3 nodes ensures no single compromise reveals patient data.

splitAuditEntry() + 2-of-3
💹
Financial
SEC 17a-4 Compliance

Trade order modifications logged in an immutable chain. SEC requires WORM (write-once-read-many) audit trails. Xtrace provides cryptographic WORM via chained hashes.

appendToChain() + verifyChain()
🔒
Enterprise
SOC 2 Audit Trails

Key rotation events, permission changes, and access grants logged with tamper detection. SOC 2 requires demonstrable log integrity. HMAC chains provide cryptographic proof.

verifyEntryHMAC() + chain validation
🏛
Government
FISMA Audit Requirements

Federal systems require tamper-evident audit logs for compliance. Split storage across classified and unclassified networks with information-theoretic security.

XorIDA split + HMAC verification
Legal
eDiscovery Chain of Custody

Document access and modifications tracked in a tamper-evident chain. Legal teams can prove when and by whom evidence was accessed or altered.

ChainedAuditEntry + timestamp validation
📡
IoT
Device Firmware Updates

Firmware deployment events logged with cryptographic proof. Detect unauthorized firmware modifications across fleet of 10,000+ devices.

appendToChain() + GENESIS_HASH
Section 04

Solution Architecture

Four composable functions for building tamper-evident audit systems.

Verification
verifyChain() + verifyEntryHMAC()
Recomputes hashes for full chain
Validates prevHash links
Independent HMAC verification per entry
<1ms per entry verification
Split Storage
splitAuditEntry()
XorIDA threshold sharing over GF(2)
Defaults to 3 shares, threshold 2
Each share independently HMAC-signed
Information-theoretic security
Reconstruction
reconstructAuditEntry()
XorIDA reconstruction from k shares
HMAC verification before deserialization
PKCS#7 unpadding
JSON parse with validation
Composable Design
Use chain building alone for single-node tamper detection. Add split storage for distributed compliance architectures. Each function works independently or in combination.
Section 04a

HMAC-Chained Audit Trail

Each entry's hash depends on the previous entry's hash. Tampering with any entry breaks the chain for all subsequent entries.

Building a chain
import { appendToChain, verifyChain, GENESIS_HASH } from '@private.me/auditlog';

// First entry starts from GENESIS_HASH
const entry1 = {
  entryId: 'e1',
  timestamp: Date.now(),
  actor: 'admin@corp.com',
  action: 'key.rotate',
  resource: 'master-key-v3',
};

const chained1 = await appendToChain(entry1, GENESIS_HASH);
if (!chained1.ok) throw new Error(chained1.error.message);

// Second entry chains from first
const entry2 = {
  entryId: 'e2',
  timestamp: Date.now(),
  actor: 'admin@corp.com',
  action: 'user.create',
  resource: 'bob@corp.com',
};
const chained2 = await appendToChain(entry2, chained1.value.hash);

// Verify the entire chain
const valid = await verifyChain([chained1.value, chained2.value!]);
console.log('Chain valid:', valid.ok); // true
GENESIS 0000...0000 Entry 1 key.rotate SHA-256(prev + data) HMAC-SHA256 Entry 2 user.create SHA-256(prev + data) HMAC-SHA256 Entry 3 permission.grant SHA-256(prev + data) HMAC-SHA256 TAMPER-EVIDENT CHAIN Each hash depends on previous hash — modifying any entry breaks the chain
Chain Integrity
Modifying entry 1 changes its hash. Entry 2's hash depends on entry 1's hash, so entry 2's hash becomes invalid. All subsequent entries fail verification. Tampering is immediately detectable.
Section 04b

XorIDA Split Storage

Distribute audit entries across N storage nodes. Any k-of-N nodes can reconstruct an entry. Fewer than k shares reveal zero information.

Split storage (3 nodes, 2 needed)
import { splitAuditEntry, reconstructAuditEntry } from '@private.me/auditlog';

// Split entry into 3 shares, require 2 for reconstruction
const splits = await splitAuditEntry(chainedEntry, 3, 2);
if (!splits.ok) throw new Error(splits.error.message);

// Store each share on a different node
await storeOnNode('node-1', splits.value[0]);
await storeOnNode('node-2', splits.value[1]);
await storeOnNode('node-3', splits.value[2]);

// Reconstruct from any 2 shares
const restored = await reconstructAuditEntry([splits.value[0], splits.value[1]]);
console.log('Reconstructed:', restored.ok); // true
Chained Audit Entry actor: admin@corp.com | action: key.rotate XorIDA split Share 1 Node 1 (US-East) HMAC: abc...123 data: [encrypted share] Share 2 Node 2 (EU-West) HMAC: def...456 data: [encrypted share] Share 3 Node 3 (AP-South) HMAC: ghi...789 data: [encrypted share] Any 2 shares reconstruct the entry — single node compromise reveals ZERO info
Information-Theoretic Security
XorIDA threshold sharing over GF(2) ensures any k-1 or fewer shares reveal zero information about the entry content. Not computationally hard to break — mathematically impossible. No quantum computer can break information-theoretic security.
Section 05

Integration

Five lines to add tamper-evident audit logging to any system.

Basic integration
import { appendToChain, verifyChain } from '@private.me/auditlog';
import { readFileSync, appendFileSync } from 'fs';

// Read the existing chain from JSONL
const lines = readFileSync('audit.jsonl', 'utf-8').trim().split('\n');
const chain = lines.map(line => JSON.parse(line));
const lastHash = chain[chain.length - 1]?.hash || GENESIS_HASH;

// Append new entry
const entry = {
  entryId: crypto.randomUUID(),
  timestamp: Date.now(),
  actor: 'system',
  action: 'backup.complete',
  resource: 'database-2026-04-10',
};
const chained = await appendToChain(entry, lastHash);
if (chained.ok) {
  appendFileSync('audit.jsonl', JSON.stringify(chained.value) + '\n');
}

Integration with Other ACIs

Xtrace composes with other PRIVATE.ME ACIs for comprehensive compliance architectures:

Authorize
Access Control
Log every authorization decision
Tamper-evident permission grants
Audit role assignments
Backup
Data Protection
Log every backup operation
Verify restore operations
Detect unauthorized data access
Xlink
M2M Identity
Log agent-to-agent messages
Audit DID registrations
Track scope changes
Xredact
AI Inference
Log redaction requests
Audit PII detection events
Track model usage
Section 06

Security Guarantees

Five cryptographic guarantees that make Xtrace tamper-evident.

1. Tamper-Evident Chain

Each entry's hash depends on the previous entry's hash via SHA-256(prevHash + JSON(entry)). Modifying any entry changes its hash, which breaks the chain for all subsequent entries. Verification recomputes hashes and checks prevHash links.

2. HMAC Integrity

Every entry is independently HMAC-SHA256 signed. Individual entries can be verified without replaying the entire chain. The HMAC key is embedded in the hmacSig field (format: base64(key):base64(signature)).

3. HMAC Before Reconstruct

When reconstructing split entries, HMAC verification executes on the padded data before deserialization. Corrupted data is rejected before JSON parsing. This prevents injection attacks via malformed JSONL.

4. Information-Theoretic Security

XorIDA threshold sharing over GF(2) ensures any k-1 or fewer shares reveal zero information about the entry content. Not computationally hard to break — mathematically impossible. Quantum-proof by construction.

5. No Math.random()

All randomness uses crypto.getRandomValues() via @private.me/crypto. No predictable pseudo-random number generation. HMAC keys and XorIDA shares use cryptographically secure random sources.

Transport Security
The HMAC key is embedded in the hmacSig field and in split shares' hmac field. Transport-layer encryption (TLS) should protect entries and shares in transit. Xtrace provides tamper-evidence, not transport encryption.
Section 07

Performance Benchmarks

Measured on M1 MacBook Pro, Node.js 20.11.0, 100-iteration average.

<5ms
appendToChain (1KB entry)
<1ms
verifyEntryHMAC
~2ms
verifyChain (10 entries)
~8ms
splitAuditEntry (3 shares)
~6ms
reconstructAuditEntry (2 shares)

Scalability

Operation 100 entries 1,000 entries 10,000 entries
Build chain ~500ms ~5s ~50s
Verify chain ~100ms ~1s ~10s
Split all entries (3 shares) ~800ms ~8s ~80s
Production Optimization
Chain verification can be parallelized by storing periodic checkpoints. Verify the last N entries since the last checkpoint instead of the entire chain. Split storage can be batched — split multiple entries in parallel before writing to storage nodes.
Section 08

Honest Limitations

What Xtrace does NOT provide.

1. Chain Order Requirement

Chain verification requires entries in order. Out-of-order entries will fail prevHash checks. Distributed systems must ensure append-only ordering or use timestamp-based conflict resolution.

2. HMAC Key Exposure

The HMAC key is embedded in the hmacSig field (and in split shares' hmac field). Transport-layer encryption (TLS) should protect entries and shares in transit. Xtrace does NOT encrypt the entry content — only the chain integrity is protected.

3. No Automatic Uniqueness

appendToChain() does not enforce uniqueness of entryId. The caller is responsible for generating unique IDs (e.g., crypto.randomUUID()).

4. Metadata JSON Serialization

Metadata is stored as Record<string, unknown> and is included in the hash computation via JSON.stringify(). Objects with circular references or non-serializable values will fail. Use only JSON-compatible data types.

5. No Built-in Timestamp Validation

Xtrace does not validate that entry timestamps are monotonically increasing. The caller must implement timestamp validation if required for compliance (e.g., SEC 17a-4 requires timestamps synchronized to NIST time servers).

6. No Built-in Storage Backend

Xtrace provides the cryptographic primitives for tamper-evident chains. It does NOT include a storage backend. The caller must implement JSONL append-only file writes, database inserts, or object storage uploads. See the Enterprise CLI for reference implementations.

Not a Complete Audit System
Xtrace is a building block for tamper-evident audit trails. A complete audit system requires: timestamp synchronization, uniqueness enforcement, access control, retention policies, backup/restore, and compliance reporting. Xtrace provides the cryptographic chain integrity layer.
Section 09

Enterprise CLI

Production-ready HTTP server with JSONL file backend, REST API, and health checks. Part of the PRIVATE.ME Enterprise CLI Suite.

Install and run
# Install from private npm registry
pnpm add @private.me/auditlog-cli

# Start server (port 4100)
pnpm --filter @private.me/auditlog-cli start

# Or run directly
node node_modules/@private.me/auditlog-cli/src/server.ts

CLI Endpoints

POST /append
Append a new entry to the audit chain. Returns the chained entry with hash and HMAC signature.
GET /verify
Verify the integrity of the entire chain. Returns success if all prevHash links and HMACs are valid.
GET /chain
Retrieve the full audit chain (or last N entries via query param). Returns JSONL array.
POST /split/:entryId
Split a chained entry into XorIDA shares. Returns array of SplitAuditEntry objects.
POST /reconstruct
Reconstruct a chained entry from threshold shares. Returns the original entry after HMAC verification.
GET /health
Health check endpoint. Returns 200 OK if the server is running.
Production Features
JSONL file backend with atomic appends. CORS enabled for cross-origin requests. Request validation via Zod schemas. Structured error responses with error codes and hints. Rate limiting ready (connect to Redis-backed rate limiter). Docker-ready with health checks.
Section 10

Complete ACI Surface

Five public functions, five types, five error codes.

Functions

appendToChain(entry: AuditEntry, prevHash?: string): Promise<Result<ChainedAuditEntry, AuditError>>
Creates a chained audit entry. Computes SHA-256 hash of prevHash + JSON(entry) and generates HMAC signature. Defaults to GENESIS_HASH if no previous hash provided.
verifyChain(chain: readonly ChainedAuditEntry[]): Promise<Result<void, AuditError>>
Verifies the integrity of an ordered chain of entries. Recomputes each hash and checks prevHash links. Returns error on first broken link.
verifyEntryHMAC(entry: ChainedAuditEntry): Promise<boolean>
Verifies the HMAC integrity of a single chained entry. Returns true if the HMAC is valid, false otherwise.
splitAuditEntry(entry: ChainedAuditEntry, totalShares?: number, threshold?: number): Promise<Result<SplitAuditEntry[], AuditError>>
Splits a chained entry into XorIDA shares for distributed storage. Defaults to 3 shares with threshold 2.
reconstructAuditEntry(shares: readonly SplitAuditEntry[]): Promise<Result<ChainedAuditEntry, AuditError>>
Reconstructs a chained entry from threshold-many split shares. HMAC verification runs before the data is returned.

Constants

GENESIS_HASH: string
The initial hash for the first entry in a chain. Value: '0'.repeat(64) (64-character zero string).
Advanced Topics

Error Codes, Storage Backends, Wire Format

Deep dives into error taxonomy, pluggable storage implementations, and JSONL serialization format.

Appendix A1

Error Taxonomy

Five error codes covering validation, chain integrity, HMAC failures, split operations, and reconstruction.

Error Code Returned By Cause
INVALID_ENTRY appendToChain Entry is missing required fields (actor, action, or resource is empty/falsy).
CHAIN_BROKEN verifyChain An entry's prevHash does not match the previous entry's hash, or recomputed hash does not match the stored hash.
HMAC_FAILURE reconstructAuditEntry HMAC-SHA256 verification of the reconstructed padded data failed.
SPLIT_FAILED splitAuditEntry totalShares < 2, threshold < 2, threshold > totalShares, or XorIDA split threw.
RECONSTRUCT_FAILED reconstructAuditEntry No shares provided, insufficient shares, XorIDA failed, PKCS#7 unpad failed, or JSON parse failed.
Error Structure
All errors follow the AuditError interface: { code: AuditErrorCode; message: string }. Use the Result<T, E> pattern for error handling: check .ok, access .value or .error.
Appendix A2

Storage Backends

Three reference implementations: JSONL file, SQLite, and S3 object storage.

1. JSONL File Backend (Reference Implementation)

Append-only JSONL file
import { appendFileSync, readFileSync } from 'fs';

function appendEntry(entry: ChainedAuditEntry) {
  appendFileSync('audit.jsonl', JSON.stringify(entry) + '\n');
}

function loadChain(): ChainedAuditEntry[] {
  const lines = readFileSync('audit.jsonl', 'utf-8').trim().split('\n');
  return lines.map(line => JSON.parse(line));
}

2. SQLite Backend (Structured Query)

SQLite with full-text search
-- Schema
CREATE TABLE audit_chain (
  entry_id TEXT PRIMARY KEY,
  timestamp INTEGER NOT NULL,
  actor TEXT NOT NULL,
  action TEXT NOT NULL,
  resource TEXT NOT NULL,
  prev_hash TEXT NOT NULL,
  hash TEXT NOT NULL,
  hmac_sig TEXT NOT NULL,
  metadata TEXT
);

CREATE INDEX idx_timestamp ON audit_chain(timestamp);
CREATE INDEX idx_actor ON audit_chain(actor);
CREATE INDEX idx_action ON audit_chain(action);

3. S3 Object Storage (Distributed)

S3-compatible object storage
import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';

async function uploadEntry(entry: ChainedAuditEntry) {
  const s3 = new S3Client({ region: 'us-east-1' });
  await s3.send(new PutObjectCommand({
    Bucket: 'audit-logs',
    Key: `entries/${entry.entryId}.json`,
    Body: JSON.stringify(entry),
  }));
}
Backend Recommendations
JSONL file: Simple, append-only, works for <100K entries. SQLite: Structured queries, full-text search, works for <10M entries. S3: Distributed, versioned, infinite scale. Use S3 + DynamoDB for global compliance architectures.
Appendix A3

Wire Format

JSONL serialization format for ChainedAuditEntry and SplitAuditEntry.

ChainedAuditEntry Format

JSONL entry
{
  "entryId": "e1",
  "timestamp": 1712745600000,
  "actor": "admin@corp.com",
  "action": "key.rotate",
  "resource": "master-key-v3",
  "prevHash": "0000000000000000000000000000000000000000000000000000000000000000",
  "hash": "a1b2c3d4e5f6...",
  "hmacSig": "key_base64:sig_base64",
  "metadata": { "reason": "quarterly rotation" }
}

SplitAuditEntry Format

Split share
{
  "entryId": "e1",
  "shareIndex": 0,
  "shareTotal": 3,
  "shareThreshold": 2,
  "data": "base64_encoded_share_data...",
  "hmac": "key_base64:sig_base64",
  "originalSize": 512
}
Backward Compatibility
The JSONL format is append-only and versioned. Future versions may add new fields but will never remove existing fields. Older parsers can safely ignore unknown fields via JSON parsing semantics.

Deployment Options

🌐

SaaS Recommended

Fully managed infrastructure. Call our REST API, we handle scaling, updates, and operations.

  • Zero infrastructure setup
  • Automatic updates
  • 99.9% uptime SLA
  • Pay per use
View Pricing →
📦

SDK Integration

Embed directly in your application. Runs in your codebase with full programmatic control.

  • npm install @private.me/auditlog
  • TypeScript/JavaScript SDK
  • Full source access
  • Enterprise support available
Get Started →
🏢

On-Premise Enterprise

Self-hosted infrastructure for air-gapped, compliance, or data residency requirements.

  • Complete data sovereignty
  • Air-gap capable
  • Docker + Kubernetes ready
  • RBAC + audit logs included
Enterprise CLI →