315326d0a2
- Add authentication middleware with API key validation - Add request logging middleware for observability - Add audit logging middleware for admin operations - Refactor API endpoints to use centralized auth middleware - Add comprehensive unit tests for all middleware - Add API documentation and deployment guide - Update README with health endpoints and documentation links - Fix test data isolation in router tests Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
LLM Gateway
A unified LLM Gateway supporting multiple providers and API formats.
Features
- Multi-format API: OpenAI Chat Completions, OpenAI Responses API, Anthropic Messages API
- Multi-provider support: OpenAI, Anthropic, Azure OpenAI, Google Gemini, AWS Bedrock
- Model aliasing and routing: Flexible model mapping with load balancing
- Rate limiting: RPM/TPM limits at multiple levels
- Budget control: Key and Project level spending limits
- High availability: Fallback, retry, and circuit breaker
- Observability: Request logging and usage statistics
Quick Start
# Install dependencies
pip install -r requirements.txt
# Run the server
uvicorn app.main:app --reload
API Endpoints
LLM APIs
POST /v1/chat/completions- OpenAI-compatible chat completionsPOST /v1/responses- OpenAI Responses APIPOST /v1/messages- Anthropic Messages API
Admin APIs
GET|POST|PUT|DELETE /admin/providers- Provider managementGET|POST|PUT|DELETE /admin/keys- API Key managementGET|POST|PUT|DELETE /admin/projects- Project managementGET|POST|PUT|DELETE /admin/models/aliases- Model alias managementGET /admin/usage/stats- Usage statistics
Health Endpoints
GET /health- Basic health checkGET /ready- Readiness checkGET /admin/providers/{id}/health- Provider health status
Documentation
- API Documentation - Complete API reference
- Deployment Guide - Production deployment instructions
Configuration
See .env.example for configuration options.
Docker
# Build and run
docker-compose up -d
# Check health
curl http://localhost:8000/health
License
MIT