Introducing StellarGate: Use Any LLM Without Exposing Your Data

The Problem

Every enterprise wants to use the best AI models available. GPT-4 for analysis, Claude for long-context reasoning, Gemini for multimodal tasks. The capabilities are undeniable. But there’s a catch that stops most companies at the door.

To use these models, you have to send your data to a third party. Every prompt, every document, every question — it all leaves your infrastructure and travels to servers you don’t control. For many organizations, that’s a non-starter.

Healthcare companies can’t send patient records to OpenAI. Law firms can’t pipe privileged communications through an external API. Financial institutions face strict data residency requirements that prohibit cross-border data transfers. Even companies without hard regulatory constraints often have internal policies that prevent sensitive data from reaching third-party services.

The result? Teams either avoid commercial LLMs entirely — settling for weaker local models — or they create manual workarounds where employees redact sensitive information by hand before prompting. Both approaches are slow, error-prone, and don’t scale.

How StellarGate Works

StellarGate sits between your application and any LLM provider. It automatically detects and replaces sensitive entities in your prompts before they leave your infrastructure, then restores them in the response. The LLM never sees your real data.

The process follows a four-step pipeline:

Input: "Send payment of $14,500 to John Smith at john@acme.com"

Step 1 — Detect:   [John Smith] → PERSON, [$14,500] → AMOUNT, [john@acme.com] → EMAIL
Step 2 — Tokenize:  "Send payment of [AMT_1] to [PER_1] at [EMAIL_1]"
Step 3 — Send:      LLM receives sanitized prompt, generates response
Step 4 — Resolve:   Replace tokens back → original values in final output

Step 1: Detect

StellarGate runs a local Named Entity Recognition (NER) pipeline on every incoming prompt. This isn’t a regex matcher — it’s a purpose-built model that understands context. It knows that “Apple” in “Apple Inc. reported earnings” is an organization, while “apple” in “I ate an apple” is not.

Step 2: Tokenize

Every detected entity gets replaced with a deterministic placeholder token. The same entity always maps to the same token within a session, so the LLM can still reason about relationships. If “John Smith” appears five times in a document, all five instances become the same token.

Step 3: Send

The sanitized prompt is forwarded to the target LLM. From the model’s perspective, it’s just processing a normal request — it has no idea that tokens represent redacted values. The model reasons about the structure and semantics just as effectively.

Step 4: Resolve

When the response comes back, StellarGate replaces all tokens with their original values. Your application receives a clean, fully resolved response as if the anonymization never happened.

What Gets Anonymized

StellarGate’s NER pipeline recognizes a comprehensive set of entity types out of the box:

• Person names — full names, first/last names, titles
• Email addresses — personal and corporate addresses
• Phone numbers — international formats, extensions
• Monetary amounts — currencies, ranges, written-out amounts
• Physical addresses — street addresses, cities, postal codes
• National IDs — social security numbers, tax IDs, passport numbers
• Account numbers — bank accounts, credit cards, IBANs
• Dates of birth — various date formats linked to individuals
• Organization names — companies, institutions, departments

Each entity type can be independently enabled or disabled, so you can fine-tune what gets anonymized based on your specific compliance requirements.

Integration

We designed StellarGate to require the smallest possible change to your existing code. If you’re already using the OpenAI SDK, you change one line — the base URL:

// Before
const client = new OpenAI({
	baseURL: 'https://api.openai.com/v1'
});

// After — one line change
const client = new OpenAI({
	baseURL: 'https://gate.stellarbase.ai/v1'
});

The same applies to the Anthropic SDK:

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic({
	baseURL: 'https://gate.stellarbase.ai/anthropic'
});

const message = await client.messages.create({
	model: 'claude-sonnet-4-20250514',
	max_tokens: 1024,
	messages: [{ role: 'user', content: 'Summarize this contract...' }]
});

StellarGate is fully compatible with any OpenAI-compatible API client. That includes LangChain, LlamaIndex, Vercel AI SDK, and custom HTTP clients. If it can point to a URL, it works with StellarGate.

No SDK to install. No wrapper functions. No code refactoring. Just swap the URL and your prompts are automatically protected.

Performance

Privacy shouldn’t come at the cost of speed. StellarGate adds less than 5 milliseconds of latency to each request. Here’s why it’s so fast:

• Local NER models — entity recognition runs entirely on your infrastructure, no external API calls needed for the anonymization step
• Streaming support — StellarGate handles streaming responses natively, resolving tokens on the fly as chunks arrive
• Session-level caching — entity mappings are cached per session, so repeated entities are resolved instantly
• Async pipeline — detection and tokenization happen concurrently with request preparation

In our benchmarks across 10,000 prompts of varying complexity, the median overhead was 3.2ms. The 99th percentile was 4.8ms. For comparison, a typical LLM API call takes 500ms to 3 seconds — the anonymization overhead is invisible.

What’s Next

StellarGate is available today as part of the StellarBase platform. Here’s what we’re building next:

• Custom entity types — define your own entity patterns for domain-specific data (medical codes, internal project names, proprietary terms)
• Per-field policies — different anonymization rules for different parts of your prompt (e.g., system prompt vs. user message)
• Audit logs — full traceability of every anonymization event, exportable for compliance reporting
• Multi-language NER — expanded entity recognition for Czech, German, French, and Spanish
• Policy templates — pre-built anonymization profiles for GDPR, HIPAA, and SOC 2 compliance

If you’re an enterprise that wants to use the best AI models without compromising on data privacy, StellarGate is built for you. Get started at stellarbase.ai/gate or reach out to our team for a demo.