Launcher & shells

The launcher

Bare toki opens an Ollama-style full-screen launcher in a black + yellow design: a letter-spaced yellow TOKI wordmark with tagline and amber divider, then a vertically-centred column of app cards, toki Chat, OpenCode, Hermes, OpenClaw, web dashboards, each bound to the pinned model (shown as a right-aligned chip). The active row is highlighted with a yellow accent bar. A status line shows the pinned model, backend health dots, and key hints; keys 1–5 jump straight to an app (or model).

In the settings pane, Enter opens an editor for the highlighted field. Sizing and enum fields (ctx_size, max_tokens, parallel, kv, reasoning, always_on, boot) open a dropdown of presets with a custom… option that falls back to the free-text editor; paths, ports and ids stay plain text. Both editors have Save / Cancel buttons: Save writes the field to the registry (~/.config/tokirecur/models.yaml) immediately; Cancel (or Esc) discards.

The reasoning field controls thinking mode: auto/on enable it, the model's reasoning is streamed (dimmed in the TUI chat, rendered as a separate block in the web UI); off passes --reasoning off to llama-server at launch. Changes take effect on the model's next start.

Interactive shell

The shell prompt has a live status indicator, llama:✓ daemon:✓ means both are reachable. Tab completion and command history are enabled if prompt_toolkit is installed (installed by default).

up [--detach]                 start llama-server + daemon
down [-y]                     stop llama-server + daemon
status                        show stack state
doctor                        check llama-server, ROCm, GPU
bench [args]                  run the benchmark suite
run "prompt"                  one-shot completion via daemon
stats                         cache index stats
list [n]                      list cached prompts (default 20)
show <id>                     show prompt details
evict <mb>                    LRU evict to target MB
clear                         wipe cache
model <path>                  set GGUF model path
port llama <n> / daemon <n>   set ports
ctx <n> / ngpu <n>            set context size / GPU layers
config                        show current config
compare <a.json> <b.json>     compare two bench reports
help [command]                show help for a command (or all)
exit / quit / q               exit shell

TUI chat (toki tui)

The full-screen terminal chat client. Three modes, switched with Ctrl+1/2/3 (or in-chat slash commands):

toki tui                                   # chat against localhost daemon + llama-server
toki tui --auth-token <token>              # if the daemon requires auth
toki tui --daemon-url http://other:8787    # custom daemon

Features: live TTFT sparkline + running hit-rate counter in the right panel; per-backend connection health (green/yellow/red); a slot state panel showing which slots are in use and which session owns them; precision retention panel (Ctrl+P) with baseline/recycled byte-identity stats; session pinning via UUID so multi-turn chats reuse the same slot; persistent history at ~/.local/share/tokirecur/history.jsonl (XDG); graceful degradation, if the daemon is unreachable, mode auto-switches to raw.

Requires a TTY. From a non-interactive shell you'll get: error: toki tui requires an interactive terminal (TTY).

Web chat UI (toki web)

A production-ready single-page chat interface with a single-click mode toggle (Raw / Cached / Compare), streaming responses, and live per-message metrics. Default port is 7777, uncommon and unlikely to clash with other localhost services.

toki web                                      # foreground, http://127.0.0.1:7777
toki web --detach                             # background, log → /tmp/toki/web.log
toki web --port 7778 --raw-url http://...:8088  # custom port + backend

• Three inference modes via prominent toggle: Raw, Cached, Compare (side-by-side)
• Streaming responses with token-by-token display
• Live metrics per message (TTFT, total time, tokens generated, prefill tokens)
• Compare mode shows two side-by-side panels with a summary banner showing speedup and a divergence warning if outputs differ
• System prompt input persisted in the sidebar; conversation history with multi-turn support; mobile-responsive

The UI is a single static page served by a FastAPI app with WebSocket streaming (/ws/chat). No external frontend dependencies, vanilla HTML/CSS/JS.

Next: Architecture, the 5-step recycling algorithm.