Why use the Gateway?
You could call Anthropic or OpenAI directly. But then you have to build the compliance layer yourself.- Zero-Retention Agreements: We hold BAAs with major providers—pass this through to your users.
- Unified Audit Log: Every prompt and completion logged for compliance (optional).
- PII Redaction: Automatically strip names, SSNs, and dates before they hit the model.
- Provider Fallback: If OpenAI is down, we route to Azure or Anthropic automatically.
Quick example
Specialized Models
Choose the right model for your feature:| Provider | Models | Best for |
|---|---|---|
| Anthropic | Claude 3.5 Sonnet, Opus | Contract Review. Largest context window, best reasoning. |
| OpenAI | GPT-4o | Extraction. Reliable JSON output for structured data. |
| Gemini 1.5 Pro | Large Documents. 1M+ token context for analyzing entire case files. | |
| CaseMark | Legal-BERT / RoBERTa | Classification. Fine-tuned for specific legal taxonomy. |
Endpoints
Models
GET /llm/v1/models — List available modelsChat completions
POST /llm/v1/chat/completions — Generate AI responsesEmbeddings
POST /llm/v1/embeddings — Generate vector embeddings