Quick start

Nothing is preinstalled, you pull exactly what you want. Clone, install, first chat:

# 1. Clone and run the installer. Cross-platform, no token needed.
#    macOS/Linux: ./install.sh (yellow progress bar). Windows: install.bat
#    The installer also fetches the llama-server binary for your machine.
git clone https://github.com/lhadzic377-netizen/TokiRecur.git
cd TokiRecur
./install.sh            # or: install.bat

# 2. Pull a model: pick a GGUF from the numbered list (no name needed,
#    the repo is registered as a lowercase slug). --use activates it:
#    boots llama-server + the toki daemon and wires opencode/openclaw/
#    hermes at it.
toki model pull bartowski/Kwaipilot_KAT-Coder-V2.5-Dev-GGUF --use

# 3. Open the launcher (chat, web UI, model switching)
toki

Prefer a direct install without the clone? pipx install "tokirecur[embeddings] @ git+https://github.com/lhadzic377-netizen/TokiRecur.git" works too, see the install docs.

No models are installed or started at install time, toki model use <name> is the explicit, documented way to bring one up (it starts llama-server + the toki daemon, stops competing big-model stacks, and wires opencode/openclaw/hermes at it). toki up --model <path> is the single-model equivalent for a raw GGUF without a registry entry. toki serve is only the API daemon (default 127.0.0.1:8787) that proxies to an already-running llama-server; it warns loudly if the backend isn't reachable instead of failing silently.

Small-RAM machines

On machines with limited RAM (8–16 GB), toki model use automatically reduces the requested context so the KV cache fits in memory alongside the model weights. It reads the GGUF header to compute KV usage per token, and the running server's real slot size after boot. The effective context, the smaller of your configured ctx_size and what the machine can actually hold, is written to the model's state dir and becomes the single source of truth: the OpenAI-compatible API advertises exactly that context, clamps max_tokens to it, and refuses to over-allocate when llama-server could not honor the requested context (which previously surfaced as confusing 400 not enough tokens errors). Failed startups no longer hang, llama-server's log tail is surfaced and the launcher returns an error. If the server still can't start, pick a smaller/quantized model or lower the model's ctx_size in the registry; the context floor is 512 tokens.

Use as an opencode backend

The daemon's /v1/chat/completions is OpenAI-compatible and tool-calling capable, so opencode can run entirely on the local KV-recycled model. Every agent turn reuses the prefix from the previous turn (system prompt, tools, earlier messages) via llama-server's KV cache, no cloud, no num_ctx fiddling, and cache hits show up in the daemon's metrics.

// ~/.config/opencode/opencode.json
{
  "provider": {
    "tokirecur": {
      "name": "Toki Recur (KV-cache, local)",
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "http://127.0.0.1:8787/v1",
        "apiKey": "demo-token"
      },
      "models": {
        "qwen3.5-9b": {
          "name": "Qwen3.5 9B (Toki Recur)",
          "contextWindow": 24000,
          "limit": { "context": 24000, "output": 8192 }
        }
      }
    }
  },
  "small_model": "tokirecur/qwen3.5-9b" // keep title-gen local too
}

• Set apiKey to whatever TOKI_AUTH_TOKEN the daemon runs with.
• The model id is a label; the actual weights are whatever llama-server has loaded. contextWindow should stay under TOKI_MAX_CONTEXT_TOKENS.
• Tool calls stream through verbatim (delta.tool_calls), finish with finish_reason: "tool_calls", and role: "tool" result messages round-trip unchanged, verified against opencode's own bash tool loop.
• Non-stream requests work too (choices[0].message.tool_calls).

Next: Command reference, the full toki CLI.