root
|
bebe8c1bb5
|
docs: add LLM Gateway design document
Design a unified LLM Gateway with:
- Multi-format API support (OpenAI, Anthropic, Responses API)
- 5 provider adapters (OpenAI, Anthropic, Azure, Gemini, Bedrock)
- Model aliasing, routing, and load balancing
- RPM/TPM rate limiting and budget control (key/project level)
- Fallback/retry with circuit breaker
- Request logging and usage statistics
- Admin API for provider/key/model management
Tech stack: Python (FastAPI) + SQLite
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
2026-05-01 14:52:53 +08:00 |
|