SmartRouter is a drop-in, OpenAI-compatible gateway that answers every request at the cheapest tier that can prove it's right — deterministic solvers and a free local model first, a paid model only when it's genuinely needed.
The same gateway classifies a request, then answers it wherever it can be proven correct for free — and only spends on the hard part.
Everything runs offline; solvers answer for free with no keys. Add a Fireworks key only if you want the escalation tier live.
# one container: gateway + dashboard
git clone https://github.com/kartikbhartiya/Hybrid-Token-Efficient-Routing-Agent
cd Hybrid-Token-Efficient-Routing-Agent
docker compose up --build agent→ open http://localhost:8000 · lean image: docker build -f Dockerfile.demo -t smartrouter . && docker run -p 8000:8000 smartrouter
# Python 3.11+
pip install -r requirements.txt
uvicorn src.server:app --port 8000→ open http://localhost:8000 · Windows: double-click run.bat
# 301 tests, offline, no keys needed
pip install -r requirements.txt pytest
python -m pytest tests/ -q→ 301 passed in ~14s · on Windows just double-click test.bat
40 graded tasks across all 8 challenge categories. Numbers checked by exact match, entities by required-mention, and code by executing it against unit tests.
| Setup | Accuracy | Fireworks tokens |
|---|---|---|
| SmartRouter | 100% | 0 |
| All-remote baseline (GLM-5.1 for everything) | 100% | 3,137 |
| Category | Zero-token | Graded correct |
|---|---|---|
| Math | 5 / 5 | 5 |
| Code debugging | 5 / 5 | 5 |
| Code generation | 5 / 5 | 5 |
| Logical reasoning | 5 / 5 | 5 |
| Sentiment | 5 / 5 | 5 |
| NER | 5 / 5 | 5 |
| Summarization | 1 / 5 | 1 |
| Factual | 0 / 5 | 0 |
| Total answered free | 31 / 40 | 31 / 31 |
The 9 remaining tasks (factual + long summaries) go to the free local model first — still zero Fireworks tokens — then the cheapest sufficient model only if needed.
An answer ships from a free tier only when it can be proven good — and escalates the moment it can't. Five of the six tiers cost nothing.
Ranks the allowed Fireworks models cheap→strong and stops at the first whose answer clears free verification.
what is 1+3 and write two-sum → the arithmetic is free, only the code half is paid.
Time-sensitive facts (current leaders, prices, records) go to the stronger tier; stable facts stay free.
Paraphrased repeats served free at ≥0.92 cosine — with a numeric guard so the numbers must match exactly.
Strips filler ("i'm confused, what is 2+2?"), answers greetings and "what's today's date" for free.
Learns from live traffic (promote/demote categories) and degrades to free tiers instead of failing when a budget runs out.
Every routing decision, solver, cache, and the grading contract are covered by offline tests with stubbed clients.
$ python -m pytest tests/ -q
........................................ [ 40%]
........................................ [ 80%]
.................... [100%]
301 passed in 14.2s
$ python scripts/coverage_report.py
TOTAL 31/40 zero-token · 31/31 correct