Install

Toki Recur is a Python app, install it with pipx, which isolates it in its own venv and puts toki on PATH. The repository is public and MIT licensed, so every command below works anonymously: no token, no account, no rate limit to hit.

Simplest: clone, then run the installer

Clone the repo once and run the installer from the checkout. Cross-platform, no token required.

# Linux / macOS
git clone https://github.com/lhadzic377-netizen/TokiRecur.git
cd TokiRecur
./install.sh

:: Windows (cmd or PowerShell)
git clone https://github.com/lhadzic377-netizen/TokiRecur.git
cd TokiRecur
install.bat

Both install the toki CLI with pipx (auto-installed if missing; Linux/macOS falls back to a venv). The embeddings extra (sentence-transformers + torch) is included by default; skip it with TOKI_EXTRAS= ./install.sh.

Direct pipx install

pipx install "tokirecur[embeddings] @ git+https://github.com/lhadzic377-netizen/TokiRecur.git"

One-time: sudo apt install pipx or python3 -m pip install --user pipx.

Without pipx

Install into your own venv instead:

python3 -m venv .venv && source .venv/bin/activate
pip install "tokirecur[embeddings] @ git+https://github.com/lhadzic377-netizen/TokiRecur.git"

If toki is not found after install, your PATH may not include the venv's bin directory, either activate the venv or use the full path (./.venv/bin/toki). On systems where pip install -e writes a shim to ~/.local/bin that points at system Python, delete that shim and re-run pip install -e . from inside the activated venv so the shim is regenerated with the correct shebang.

curl / irm one-liner installers

The installers below are scripts/install-curl.sh and scripts/install-irm.ps1. They install tokirecur with pipx from the public repository. No token is needed; if you set TOKI_REPO_TOKEN (or GITHUB_TOKEN) it is used instead, which only raises the GitHub API rate limit.

Linux / macOS (curl)

curl -sf "https://raw.githubusercontent.com/lhadzic377-netizen/TokiRecur/main/scripts/install-curl.sh" \
  -o /tmp/toki-install.sh
bash /tmp/toki-install.sh

Windows (irm / PowerShell)

$installer = irm -Uri "https://raw.githubusercontent.com/lhadzic377-netizen/TokiRecur/main/scripts/install-irm.ps1" -ErrorAction Stop
$installer | Out-File -Encoding utf8 "$env:TEMP\toki-install.ps1"
pwsh -File "$env:TEMP\toki-install.ps1"

curl -sf (exit code 22 on HTTP errors) and irm ... -ErrorAction Stop fail fast instead of silently downloading an error page. Both installers use TOKI_REPO_TOKEN (or GITHUB_TOKEN) when it is set and fall back to the anonymous public URL when it is not.

Fetch the llama-server binary

One command installs the matching official build, it auto-detects the best backend for your machine. Pass --backend to override.

toki model install-llama                 # auto-detect backend, latest release
toki model install-llama --backend rocm  # force a backend (rocm|cuda|sycl|openvino|vulkan|metal|cpu)
toki model install-llama --backend rocm --gfx 11.5.1  # set HSA_OVERRIDE_GFX_VERSION
toki model install-llama --tag b6400     # pin a release tag
toki model install-llama --name 35b      # ...and point a registry model at it

Platforms

Toki Recur is pure Python over the llama-server HTTP API, it runs on any x86_64 or aarch64 device running Linux, macOS or Windows, with whatever GPU backend your llama-server build has. The only per-platform piece is the llama-server binary itself:

The GPU backends are provided by the llama-server binary itself, Toki Recur talks to it over HTTP and is backend-agnostic. Every GPU install verifies the vendor runtime and a real device line from llama-server --list-devices before reporting success, so a missing driver cannot silently degrade to CPU: CUDA checks its paired cudart/cublas archive and CUDA0; ROCm checks libamdhip64.so / amdhip64*.dll and ROCm0; SYCL checks the oneAPI runtime, oneMKL and Level Zero on Linux and bundled sycl*.dll on Windows; OpenVINO checks the core runtime and the Intel GPU plugin; Vulkan checks the driver loader. Linux SYCL archives are not self-contained, so oneAPI is a real prerequisite there. CUDA 13.x needs driver 580.65+; the Linux 12.8 build needs 570.26+ (Windows 12.4 needs 551.61+). macOS ships one universal Metal build (there is no separate CPU asset); CPU-only mode is -ngl 0 / --device none, and a Mac or VM without Metal runs that build on CPU with an explicit warning.

System daemon installers

Optional, run as a system daemon (dedicated user + service manager unit + log rotation, auto-start at boot). install.sh/install.ps1 bootstrap the full stack themselves (own venv at /opt/toki-recur, package + llama-server + service unit), so they work standalone, they don't need the pipx install above:

# Linux  -> systemd + logrotate   (macOS -> launchd LaunchDaemon)
sudo ./install.sh

# Windows (PowerShell): install.ps1; then `toki serve` yourself or wrap
# it in NSSM / a scheduled task.

Model registry

Or point binary: at any llama-server on PATH in the registry (~/.config/tokirecur/models.yaml). Override locations per-machine with TOKI_MODEL_REGISTRY (config) and TOKI_STATE_DIR (logs/pid/bin). On Windows the systemd service commands are unavailable (reported cleanly); the RAM guard uses GlobalMemoryStatusEx, and on macOS the UMA/RAM guard uses sysctl hw.memsize.

Next: Quick start, six commands to your first cached chat.