AI Gateway with PII Redaction for LLM Applications

Why PII Protection Matters for LLMs

Consider a common scenario in a healthcare application:

{
  "model": "gpt-4.1",
  "messages": [
    {
      "role": "user",
      "content": "Summarize this patient record: John Smith, SSN 123-45-6789, DOB 03/15/1982, diagnosed with Type 2 diabetes on 01/10/2025. Prescribed Metformin 500mg."
    }
  ]
}

Without a gateway, this prompt — containing a real name, SSN, date of birth, and medical diagnosis — is sent unmodified to the model provider's servers.

Architecture: AI Gateway with PII Detection

OpenSourceAIHub implements a multi-stage pipeline that inspects every request before it reaches any downstream provider:

The AI Firewall Detection Engine

Unlike simple regex-based filters, the OpenSourceAIHub firewall uses a multi-layered detection engine that combines four techniques to minimize false negatives without sacrificing latency:

This combined approach adds fewer than 50 milliseconds of latency per request for text and approximately 0.5–1 second for image payloads (vision security). Every response includes x-hub-scan-ms timing headers so you can verify performance in production.

Example: PII Redaction Flow

Supported Entity Types (28 total)

Implementing PII Redaction in an AI Gateway

OpenSourceAIHub is a drop-in replacement for the OpenAI API. Point your existing SDK at the Hub endpoint and PII redaction happens automatically — no code changes beyond the base URL and API key:

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "os_hub_your_key_here",
  baseURL: "https://api.opensourceaihub.ai/v1",
});

const response = await client.chat.completions.create({
  model: "oah/gpt-4.1",
  messages: [
    {
      role: "user",
      content: "Summarize this patient record: John Smith, SSN 123-45-6789",
    },
  ],
});

// The Hub automatically:
// 1. Scans the prompt for PII entities
// 2. Redacts "John Smith" → [PERSON], "123-45-6789" → [US_SSN]
// 3. Forwards the cleaned prompt to the provider
// 4. Returns the response with x-hub-scan-ms timing header

from openai import OpenAI

client = OpenAI(
    api_key="os_hub_your_key_here",
    base_url="https://api.opensourceaihub.ai/v1",
)

response = client.chat.completions.create(
    model="oah/gpt-4.1",
    messages=[
        {
            "role": "user",
            "content": "Summarize this patient record: John Smith, SSN 123-45-6789",
        }
    ],
)

# PII is redacted before the request reaches OpenAI.
# Check response headers for scan timing and violation counts.

curl -X POST https://api.opensourceaihub.ai/v1/chat/completions \
  -H "Authorization: Bearer os_hub_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "oah/gpt-4.1",
    "messages": [
      {
        "role": "user",
        "content": "Summarize this patient record: John Smith, SSN 123-45-6789"
      }
    ]
  }'

# Response headers include:
# x-hub-scan-ms: 12
# x-hub-violations: PERSON,US_SSN
# x-hub-correlation-id: req_xxxx

Violation Response Format

When the firewall detects a BLOCK-level entity (like a prompt injection attempt), it rejects the request immediately with a structured error response:

{
  "error": {
    "message": "Security policy violation: request blocked.",
    "type": "security_violation",
    "code": 400,
    "violations": [
      {
        "entity": "PROMPT_INJECTION",
        "action": "BLOCK",
        "start": 0,
        "end": 47
      },
      {
        "entity": "US_SSN",
        "action": "REDACT",
        "start": 52,
        "end": 63
      }
    ],
    "correlation_id": "req_a1b2c3d4"
  }
}

Supported AI Providers

PII redaction works identically across all providers. Use a single gateway endpoint and virtual model names (oah/*) — the Hub handles provider routing automatically:

Benefits of an AI Security Gateway

Try It with OpenSourceAIHub

Get started in under five minutes. No credit card required for the free tier — every request is protected from your very first API call.