Development

Repository layout

tokirecur/
├── src/tokirecur/
│   ├── core/        # The 5-step algorithm, recycling engine
│   ├── index/       # BruteForce + HNSW index, SQLite metadata
│   ├── backend/     # llama-server HTTP adapter (slot routing on upstream endpoints)
│   ├── cache/       # in-memory slot pool; blob/tree modules are dormant
│   ├── api/         # FastAPI: OpenAI + Anthropic + admin endpoints
│   ├── bench/       # The benchmark CLI (toki bench)
│   └── cli/         # `toki` admin CLI
├── patches/         # Historical patch for older llama.cpp
├── docker/          # ROCm Dockerfile + docker-compose
├── scripts/         # installers, site build, llama.cpp helpers
├── tests/           # unit, integration, benchmarks
└── benchmark/       # configs, reports, corpus

Tests

pytest tests/unit/ -v        # fast (seconds)
pytest tests/ -v             # includes integration (needs a live llama-server)

1,200+ unit tests cover the recycling engine, the index, the rate limiter, the persistent-slot pool, the dormant .trkv format, the CLI config writers, and the OpenAI-compatible API (chat, completions, and Responses). CI runs the unit tests on Python 3.11 / 3.12 / 3.13 on Linux, and on 3.13 on macOS, Windows and arm64 Linux, every push and PR, and executes the Linux and macOS system installers end-to-end.

Benchmark harness

One released measurement: AMD Ryzen AI MAX+ 395, Qwen3.5-9B-Q4_K_M, llama-server 9413. The same run is on the benchmark page; reuse is in-memory slot routing, so no --slot-save-path is involved.

License

MIT.