root
|
8f550a2100
|
docs: add LLM Gateway implementation plan
Plan includes 6 batches with 34 tasks:
- Batch 1: Project foundation (config, db, logging)
- Batch 2: Data models and Admin API (CRUD)
- Batch 3: Core services (transformer, router, rate limiter, budget)
- Batch 4: Provider adapters (OpenAI, Anthropic, Azure, Gemini, Bedrock)
- Batch 5: API endpoints (chat, messages, responses)
- Batch 6: Management and deployment
Estimated time: 19-25 hours
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
2026-05-01 14:55:18 +08:00 |
|
root
|
bebe8c1bb5
|
docs: add LLM Gateway design document
Design a unified LLM Gateway with:
- Multi-format API support (OpenAI, Anthropic, Responses API)
- 5 provider adapters (OpenAI, Anthropic, Azure, Gemini, Bedrock)
- Model aliasing, routing, and load balancing
- RPM/TPM rate limiting and budget control (key/project level)
- Fallback/retry with circuit breaker
- Request logging and usage statistics
- Admin API for provider/key/model management
Tech stack: Python (FastAPI) + SQLite
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
2026-05-01 14:52:53 +08:00 |
|