nobody/llm-gateway
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5 Commits 1 Branch 0 Tags
Commit Graph

5 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
root
681ad84674 test(router): add router unit tests and fix test data isolation 
- Add unit tests for Router model alias resolution
- Fix async fixture configuration with pytest_asyncio
- Add automatic rollback in db_session fixture for data isolation
- Fix test_get_fallback_provider chain handling

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-01 22:33:38 +08:00
root
8348520bdf feat: implement LLM Gateway with multi-provider support 
Implement a unified LLM Gateway supporting multiple API formats and providers:

Features:
- OpenAI Chat Completions, Responses API, and Anthropic Messages API
- Provider adapters for OpenAI, Anthropic, Azure OpenAI, Google Gemini, AWS Bedrock
- Model aliasing with weighted round-robin load balancing
- Virtual API keys with RPM/TPM rate limiting
- Budget control at key and project levels
- Request logging, usage statistics, and audit logs
- Fallback/retry with circuit breaker pattern
- Admin CRUD APIs for providers, projects, keys, models, usage
- Provider health checks

Tech stack:
- FastAPI with async SQLAlchemy 2.0
- SQLite with aiosqlite
- bcrypt for API key hashing, AES-256 for provider key encryption
- Docker containerization

Tests: 18 passing integration tests for admin API endpoints

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-01 15:39:21 +08:00
root
8f550a2100 docs: add LLM Gateway implementation plan 
Plan includes 6 batches with 34 tasks:
- Batch 1: Project foundation (config, db, logging)
- Batch 2: Data models and Admin API (CRUD)
- Batch 3: Core services (transformer, router, rate limiter, budget)
- Batch 4: Provider adapters (OpenAI, Anthropic, Azure, Gemini, Bedrock)
- Batch 5: API endpoints (chat, messages, responses)
- Batch 6: Management and deployment

Estimated time: 19-25 hours

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-01 14:55:18 +08:00
root
bebe8c1bb5 docs: add LLM Gateway design document 
Design a unified LLM Gateway with:
- Multi-format API support (OpenAI, Anthropic, Responses API)
- 5 provider adapters (OpenAI, Anthropic, Azure, Gemini, Bedrock)
- Model aliasing, routing, and load balancing
- RPM/TPM rate limiting and budget control (key/project level)
- Fallback/retry with circuit breaker
- Request logging and usage statistics
- Admin API for provider/key/model management

Tech stack: Python (FastAPI) + SQLite

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-01 14:52:53 +08:00
SPB Agent
2545753fca Initial commit 2026-05-01 14:37:47 +08:00