OpenSourceAIHub Documentation — API Reference, Guides & Model Catalog

How It Works (The Big Picture)

AI Providers (Groq, Together.ai, DeepInfra, Mistral AI, Anthropic, OpenAI, Google Gemini, xAI)

These are the 8 companies that host and run AI models on their servers. Think of them like different cell phone carriers — they all offer similar services (access to AI models) but at different prices and speeds. Groq is known for speed, Together.ai for variety, DeepInfra for cost, Mistral AI for their own models, Anthropic powers Claude, OpenAI powers GPT-4.1, Google Gemini offers Gemini models, and xAI provides Grok. You don't need accounts with any of them to use OpenSourceAIHub — unless you choose BYOK mode.

BYOK (Bring Your Own Key)

"Bring Your Own Key" means you create your own account with an AI provider (like OpenAI, Google Gemini, xAI, Groq, or Together.ai), get an API key from them, and store it in OpenSourceAIHub. When you send a request, the Hub uses YOUR key to talk to the provider. The provider bills you directly. The Hub adds the security layer for free. This is ideal if you already have provider accounts or want maximum control over costs. BYOK is also the only way to access premium closed-source models like GPT-4o, Gemini 1.5 Pro, and Grok-2.

Managed Mode (Wallet / Hub Credits)

Don't want to sign up with AI providers? No problem. In Managed Mode, you add credits to your OpenSourceAIHub wallet ($1 = 1,000,000 Hub Credits). Different models burn credits at different rates per actual token. The Hub uses its own provider keys on your behalf. The Hub charges wholesale cost plus a service fee — 25% markup for open-source models and 30% markup for closed-source models (OpenAI, Gemini, xAI) — deducted from your wallet automatically. This is the easiest way to get started — just top up and go.

Smart Routing

When you use Managed Mode, the Hub doesn't just send your request to one provider — it periodically indexes prices across all supported providers and selects the most cost-efficient route for your request, ensuring high performance at near-wholesale rates. This is best-effort optimization, not a guarantee of the absolute lowest price on every request. You don't need to do anything — it just works.

DLP (Data Loss Prevention) / AI Firewall

DLP is the security layer that scans every request for sensitive data before it reaches the AI model. It can detect 28+ types of sensitive information — from email addresses and credit cards to API keys and social security numbers. When it finds something, it either replaces the data with [REDACTED] (redact mode) or blocks the entire request (block mode), depending on your policy settings.

Wallet

Your prepaid credit balance on OpenSourceAIHub. You add credits via Stripe ($1 = 1,000,000 Hub Credits), and usage is deducted automatically each time you make an AI request in Managed Mode. Think of it like a prepaid balance for AI usage. 1,000,000 Hub Credits can cover thousands of simple AI requests.

Hub Credits

Hub Credits are the abstract currency for your wallet balance. $1.00 = 1,000,000 Hub Credits. Different AI models consume credits at different rates per actual LLM token — lightweight models like Llama 3 8B burn fewer credits per token, while premium models like GPT-4o burn more. This means your credit pool stretches further with efficient models.

API Key

A secret string (like a password) that identifies who you are when making requests. OpenSourceAIHub gives you keys that start with "os_hub_" (user-level) or "oah_" (project-level). You include this key in every request so the Hub knows it's you. Never share your API keys publicly.

1. Install the OpenAI SDK

npm install openai

OpenSourceAIHub is OpenAI-compatible. Use the standard SDK — just change the baseURL.

2. Make Your First Request

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "os_hub_your_key_here",
  baseURL: "https://api.opensourceaihub.ai/v1",
});

const response = await client.chat.completions.create({
  model: "oah/llama-3-70b",          // virtual model → smart-routed
  messages: [
    { role: "user", content: "Explain the CAP theorem." }
  ],
  max_tokens: 512,
});

console.log(response.choices[0].message.content);

// Custom headers are returned:
// x-request-id:   req_xxxx
// x-hub-latency:  hub overhead ms
// x-dlp-latency:  DLP scan ms

What Happens Behind the Scenes

Every time you send a request to the Hub, here's exactly what happens in order. The entire process typically takes under 100ms of Hub overhead:

1. 1Authenticate: The Hub verifies your API key. It checks if the key is valid, not expired, and not revoked.
2. 2Resolve Provider: Based on your mode (BYOK or Managed), the Hub determines which AI provider and credentials to use. In Managed Mode, the Smart Router selects the most cost-efficient available option.
3. 3Scan for Sensitive Data: The AI Firewall scans every message in your request for PII, credentials, and injection patterns. If your request contains images, those are OCR-scanned too.
4. 4Enforce Policy: If sensitive data is found, the Hub either replaces it with [REDACTED] or blocks the entire request — depending on your DLP policy settings.
5. 5Check Balance: In Managed Mode, the Hub estimates how much this request will cost and verifies your wallet can cover it. If not, you get a 402 error with a clear message.
6. 6Forward to AI: The cleaned, safe request is forwarded to the AI provider via the standard API. The Hub never stores your prompts or responses.
7. 7Log Metadata: Only metadata is recorded — token counts, cost, latency, entity types detected. Never the actual content of your prompts or responses.
8. 8Return Response: The AI provider's response is returned to your application with extra headers showing Hub latency and a correlation ID for debugging.

Supported & Unsupported Endpoints

The Hub implements the OpenAI /v1/chat/completions endpoint — the industry standard used by virtually all LLM applications. Some other OpenAI endpoints are not supported by design:

If your application currently uses the Threads API, you can migrate by managing the messages[] array in your own code (or database) and sending the full conversation with each request. This gives you the same “memory” behavior while keeping the security benefits of our stateless proxy.

Response Headers

Every response includes these custom headers for full transparency:

How API Keys Work

Every request to OpenSourceAIHub must include an API key in the Authorization header. This is how the Hub knows who you are, which security policies to apply, and how to bill you. API keys are like passwords for your AI requests — keep them secret and never include them in client-side code (like a website's JavaScript) where anyone can see them.

// Every request includes this header:
Authorization: Bearer os_hub_your_key_here

Your personal API key, tied to your user account. Created in the API Keys page of the dashboard. Best for individual developers, prototyping, and personal projects. The key is shown only once when created — after that, only an irreversible cryptographic hash is retained, so even our engineering team cannot retrieve your raw key.

Use this when you're a solo developer or want one key for all your projects.

Authorization: Bearer os_hub_*

A key scoped to a specific project. Each project can have its own DLP (security) policies, provider settings, and usage analytics. Multiple team members can use the same project key. Created within a project's Keys page.

Use this when you have multiple applications or teams that need separate security policies and usage tracking.

Authorization: Bearer oah_*

Security Best Practices

Routing Overrides

By default, the Smart Router picks the cheapest provider for your model. Use these headers to override that behavior.

Analytics Tags

Tag every request with metadata that flows into your project analytics dashboard. Use these headers to slice usage by feature, environment, or tenant — then filter the dashboard to see per-feature costs, latency, and PII violations.

Example: Tagged Request

Pass analytics tags alongside your chat completion request. These tags are stored with every request event and appear in your project dashboard, where you can filter charts and the request table by any combination of feature, environment, and tenant.

curl -X POST https://api.opensourceaihub.ai/v1/chat/completions \
  -H "Authorization: Bearer oah_your_project_key" \
  -H "Content-Type: application/json" \
  -H "x-feature: chatbot-v2" \
  -H "x-env: production" \
  -H "x-tenant: customer_123" \
  -d '{
    "model": "oah/llama-3-70b",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Which Model Should I Choose?

Real-World Example: Before vs. After

"messages": [
  {
    "role": "user",
    "content": "Hey, can you help me format this for my resume?\nName: Jane Smith\nEmail: jane.smith@acme.com\nSSN: 123-45-6789\nMy OpenAI key is sk-abc123xyz456"
  }
]

"messages": [
  {
    "role": "user",
    "content": "Hey, can you help me format this for my resume?\nName: [REDACTED]\nEmail: [REDACTED]\nSSN: [REDACTED]\nMy OpenAI key is [REDACTED]"
  }
]

Two Policy Actions

Custom IP Guard (Enterprise Regex)

Define custom regex patterns in your project's DLP policy to protect internal project names, proprietary terminology, internal IP ranges, or any sensitive string unique to your organization. Custom patterns run alongside the 28 built-in entities on every request.

// In your project's DLP Policy dashboard, add:
{
  "custom_regexes": [
    {
      "name": "INTERNAL_PROJECT_NAME",
      "pattern": "(?i)\\b(project[- ]?phoenix|codename[- ]?aurora)\\b",
      "action": "REDACT"
    },
    {
      "name": "INTERNAL_IP_RANGE",
      "pattern": "\\b10\\.42\\.\\d{1,3}\\.\\d{1,3}\\b",
      "action": "BLOCK"
    }
  ]
}

// These patterns are applied in real-time alongside
// the 28 built-in entity recognizers on every request.

How It Works

1. 1Detect: The Hub identifies image_url entries with data:image/* Base64 payloads in the messages array.
2. 2Extract: The Vision Security Layer extracts all readable text from the image via OCR within a 5-second timeout.
3. 3Scan: Extracted text is scanned by the Hub Governance Engine (28 entities + custom regex).
4. 4Enforce: If violations are found, the entire request is blocked with a 400 Security Violation. The image is never forwarded.
5. 5Purge: The image is processed in RAM only and immediately discarded. It is never written to disk or stored.

Total Token Budgeting

In Managed Mode, the Hub performs a pre-flight check before every request:

1. 1.Estimates the input token cost for the entire messages array (full conversation history).
2. 2.If the estimated input cost alone exceeds your wallet balance → 402 Insufficient Balance.
3. 3.Caps max_tokens (output) based on remaining balance after input cost, preventing surprise overspend.

Wallet Top-Up

Credits are purchased via Stripe one-time payments. Minimum top-up is 5M Credits ($5). Credits have no cash value, are non-transferable, and expire after 12 months of account inactivity. Wallet deductions are atomic and transaction-safe — you can never be double-charged or spend below zero, even under high concurrency.

Frequently Asked Questions

Understanding Latency Fields

┌───────────────── Total Request (elapsed) ──────────────────┐
│                                                            │
│  ┌─ Hub Overhead ─┐  ┌─── Provider (upstream) ───┐        │
│  │                 │  │                           │        │
│  │  DLP scan       │  │  Network + LLM inference  │        │
│  │  routing        │  │                           │        │
│  │  validation     │  │                           │        │
│  │                 │  │                           │        │
│  └─── latency_ms ──┘  └── upstream_latency_ms ───┘        │
│       (hub_meta)             (hub_meta)                    │
│                                                            │
│  dlp_latency_ms ⊆ latency_ms                              │
└────────────────────────────────────────────────────────────┘

latency_ms          = total_elapsed − upstream_latency_ms
dlp_latency_ms      ⊆ latency_ms  (DLP scan portion only)
upstream_latency_ms  = time spent waiting for the provider