TrainingGuard: AI Dataset Protection

Executive Summary

TrainingGuard protects AI training datasets from theft, poisoning, and unauthorized access by splitting them across independent custodians using XorIDA threshold secret sharing. Every split and reconstruction is recorded in an immutable provenance chain with SHA-256 hashes, actor identities, and timestamps.

Two functions cover the complete workflow: guardDataset() splits training data into N shares (default 3-of-3) with HMAC-SHA256 integrity protection per share and automatic chunking for large datasets. reconstructDataset() rebuilds the dataset from K-of-N shares, verifies the SHA-256 hash against the manifest, and appends a provenance record documenting who reconstructed it and when.

The security guarantee is information-theoretic — not computationally hard to break, but mathematically impossible. Fewer than K custodians learn exactly zero information about the training data, regardless of computing power. This protects against data theft by insiders, cloud provider breaches, and even quantum computers.

Designed for EU AI Act Article 10 compliance (data governance obligations), dataset licensing enforcement (no single licensee can reconstruct alone), and secure multi-party model training where no participant should access the full dataset.

The Problem

AI training datasets are high-value targets for theft, poisoning, and regulatory violations. Traditional protection mechanisms fail against insider threats and breaches.

Dataset theft is undetectable. A single engineer with database access can copy 50,000 labeled images to a USB drive. The dataset is gone, but audit logs show nothing suspicious — just read operations.

Data poisoning is invisible until deployment. An attacker with write access changes labels on 0.1% of samples. The model trains successfully but fails catastrophically on specific inputs.

Cloud storage is centralized trust. S3 buckets, Azure Blob Storage, and Google Cloud Storage all require trusting the cloud provider's access controls, employee background checks, and compliance certifications. One compromised admin account exposes everything.

Encryption-at-rest protects against disk theft, not authorized access. Once the application decrypts the dataset for training, it exists in plaintext memory on a server where anyone with SSH access can dump it.

The Old Way: Centralized Storage

The New Way: Threshold-Split Custody

Real-World Use Cases

Six scenarios where TrainingGuard protects AI training pipelines from theft, poisoning, and compliance violations.

How It Works

Five-step pipeline: validate, hash, chunk, split, manifest. Reconstruction reverses the flow with hash verification.

guardDataset() Pipeline

1. Validate config — Minimum 2 custodians, threshold ≤ total custodians, data not empty.
2. SHA-256 hash — Hash the entire dataset before chunking. This hash goes in the manifest and is verified on reconstruction.
3. Chunk data — Large datasets split into 2MB chunks (configurable). Each chunk processed independently.
4. Per chunk: PKCS#7 pad → XorIDA split → HMAC-SHA256 — Padding ensures chunk size divisibility, XorIDA generates N shares, HMAC signs each share.
5. Build manifest — UUID, metadata, chunk count, dataset hash, config, provenance record (action: 'split', actor, timestamp, hash).

reconstructDataset() Pipeline

1. Validate share counts — Each chunk must have ≥ threshold shares. Total chunks must match manifest.
2. Per chunk: XorIDA reconstruct → PKCS#7 unpad — Threshold shares reconstruct the chunk, padding removed.
3. Concatenate chunks — Reassemble all chunks in order, trim to original size from manifest.
4. Verify SHA-256 hash — Hash reconstructed data, compare against manifest. Mismatch = RECONSTRUCT_FAILED error.
5. Append provenance — Add 'reconstruct' provenance record (actor, timestamp, hash). Return data + updated manifest.

Integration Patterns

Three deployment patterns for different trust topologies.

Pattern 1: Multi-Cloud Distribution

import { guardDataset, reconstructDataset } from '@private.me/trainingguard';

const metadata = {
  name: 'fraud-detection-v4',
  version: '4.2.0',
  recordCount: 100000,
  format: 'parquet',
  dataSize: data.length,
  source: 'transaction-db-prod',
};

const config = { custodians: 3, threshold: 2, chunkSize: 2 * 1024 * 1024 };
const result = await guardDataset(data, metadata, config, 'ml-engineer@corp.com');

if (result.ok) {
  const { manifest, shares } = result.value;
  // shares[chunkIndex][custodianIndex]
  await uploadToS3(shares[0][0], 'custodian-1-us-east');  // AWS
  await uploadToAzure(shares[0][1], 'custodian-2-eu');    // Azure
  await uploadToGCS(shares[0][2], 'custodian-3-asia');  // GCP
}

Pattern 2: Federated Learning

// Hospital A, Research Lab B, Compliance Officer C each hold 1 share
const config = { custodians: 3, threshold: 3 }; // All 3 required

const result = await guardDataset(phiData, metadata, config, 'data-engineer@hospital.org');
// Distribute shares to 3 independent orgs

// Training time: collect shares from all 3 parties
const collectedShares = [
  await fetchFromHospital(),
  await fetchFromLab(),
  await fetchFromCompliance(),
];

const rebuilt = await reconstructDataset(manifest, collectedShares, 'trainer@lab.edu');
if (rebuilt.ok) {
  await trainModel(rebuilt.value.data); // Use reconstructed dataset
  // Provenance now shows: split (hospital) + reconstruct (lab)
}

Pattern 3: Dataset Licensing Enforcement

// Seller splits dataset 3-of-3: buyer, seller, escrow
const config = { custodians: 3, threshold: 2 }; // Any 2 of 3 can reconstruct

const result = await guardDataset(dataset, metadata, config, 'seller@datasets.ai');
if (result.ok) {
  const { manifest, shares } = result.value;
  await sendToBuyer(shares[0][0], manifest);      // Share 1 to buyer
  await keepInternal(shares[0][1]);               // Share 2 seller retains
  await sendToEscrow(shares[0][2], manifest);    // Share 3 to escrow

  // Buyer CANNOT reconstruct alone — needs seller cooperation OR escrow release
  // Seller can revoke by deleting their share (buyer now needs escrow)
}

Security

Six layers of defense protecting training data from theft, poisoning, and unauthorized access.

Threat Model

TrainingGuard assumes:

• Honest-but-curious custodians: Custodians follow the protocol but may attempt to infer information from their shares. XorIDA guarantees they learn nothing.
• Byzantine custodians (threshold - 1): Up to K-1 custodians may collude or be compromised. As long as ≥K custodians remain honest, the dataset cannot be reconstructed by attackers.
• Network adversaries: Shares transmitted over untrusted networks. HMAC integrity protects against tampering in transit.
• Storage breaches: Cloud provider breach exposes one custodian's shares. Attacker learns zero information (information-theoretic security).

TrainingGuard does NOT protect against:

• Compromise of ≥K custodians: If threshold or more custodians are compromised, the dataset can be reconstructed.
• Side-channel attacks during reconstruction: RAM dumps, timing analysis, or EM radiation during the reconstruction phase may leak information. Use hardware enclaves (SGX/SEV) for high-security scenarios.
• Social engineering: Tricking custodians into releasing shares. Multi-party authorization and provenance logging mitigate this.

Performance Benchmarks

Real-world dataset sizes from small (1MB) to large (100MB). Measured on Node.js 22, median of 100 iterations.

Node.js 22 • 100 iterations • 2-of-3 config • 2MB chunk size • Median time

Honest Limitations

TrainingGuard is not a universal solution. Here are scenarios where it does not help or where alternatives are better.

What TrainingGuard Does NOT Do

When NOT to Use TrainingGuard

• Small datasets (<1MB): Overhead of chunking and splitting exceeds the security benefit. Use simple AES-256-GCM encryption instead.
• Single-party custody: If all custodians are in the same organization under unified access control, TrainingGuard adds complexity without distributing trust. Use encryption-at-rest.
• Real-time inference pipelines: Reconstruction latency (100ms+ for large datasets) may be unacceptable for latency-sensitive inference. Cache reconstructed datasets in memory.
• Public datasets: If the dataset is already public (e.g., ImageNet, COCO), splitting provides no confidentiality benefit. TrainingGuard is for proprietary or sensitive datasets.

Regulatory Compliance

TrainingGuard supports data governance obligations under EU AI Act, GDPR, HIPAA, and financial regulations.

EU AI Act — Article 10 (Data Governance)

Article 10 requires high-risk AI systems to ensure training datasets are "relevant, representative, free of errors and complete" with documented "data governance and management practices."

HIPAA — Security Rule

45 CFR § 164.312(a)(1) requires covered entities to implement technical safeguards to protect ePHI. TrainingGuard provides:

• Access control: Threshold reconstruction prevents unauthorized access by single custodian.
• Audit controls: Provenance chain creates HIPAA-compliant audit logs (who, what, when).
• Integrity controls: SHA-256 hash verification detects unauthorized modifications.
• Transmission security: HMAC-SHA256 per share protects data in transit between custodians.

SOX — Section 404 (Internal Controls)

Financial institutions using AI for fraud detection or credit scoring must document data governance controls. TrainingGuard provenance chain demonstrates:

• Who accessed training data and when (non-repudiation via provenance)
• Data integrity verification (SHA-256 hash prevents silent poisoning)
• Multi-party authorization for dataset reconstruction (separation of duties)

GDPR — Article 32 (Security of Processing)

GDPR requires "appropriate technical and organizational measures" to protect personal data. TrainingGuard provides:

• Pseudonymization: Shares reveal zero information about the dataset (information-theoretic security).
• Confidentiality: Threshold custody prevents single-point breaches from exposing data.
• Integrity: SHA-256 + HMAC prevent unauthorized modification.
• Accountability: Provenance chain demonstrates compliance with data processing obligations.

Complete API Surface

Three core functions: guard, reconstruct, verify.

Provenance Chain

Every split, access, reconstruction, and verification is recorded in an immutable append-only log with actor identity, timestamp, and data hash.

Provenance Actions

Example Provenance Chain

[
  {
    "action": "split",
    "actor": "ml-engineer@hospital.org",
    "timestamp": "2026-04-10T14:32:18.442Z",
    "dataHash": "a3f5...d8e2"
  },
  {
    "action": "reconstruct",
    "actor": "trainer@research-lab.edu",
    "timestamp": "2026-04-11T09:15:42.112Z",
    "dataHash": "a3f5...d8e2",
    "details": "Training run batch-042"
  },
  {
    "action": "verify",
    "actor": "compliance@hospital.org",
    "timestamp": "2026-04-12T11:22:05.881Z",
    "dataHash": "a3f5...d8e2",
    "details": "HIPAA audit verification"
  }
]

Error Handling

All functions return Result<T, TrainingGuardError>. Six error codes cover validation, split, reconstruction, and provenance failures.

Error Handling Pattern

const result = await guardDataset(data, metadata, config, actor);

if (!result.ok) {
  switch (result.error.code) {
    case 'INVALID_CONFIG':
      console.error('Config error:', result.error.message);
      break;
    case 'SPLIT_FAILED':
      console.error('Crypto failure:', result.error.message);
      break;
    default:
      console.error('Unknown error:', result.error);
  }
  return;
}

// Success path
const { manifest, shares } = result.value;