Inside ALICE.
The product page covers what she does for a pilot. This one covers how — the graph, the server, the guardrails against acting on bad information, the stack, and what is not built yet.
Not a wrapper around one API call.
A topic gate sends each utterance to respond ⇄ tools, a short social reply, or an in-character decline — see app/graph.py.
Thread vs. Run
A thread (thread_id, MemorySaver) is the conversation a pilot is having with ALICE, held in memory for the session. A run is one graph.ainvoke() cycle inside it — one LangSmith trace when tracing is on. Trade-run state lives in the ledger, not in the thread.
16 tools, routed per request
The released client registers 16 tools in app/graph.py. A small decision model picks relevant tool groups for each request; uncertain or unavailable decisions give the chat model all tools. The 18 October record reported about 43% fewer schema tokens across 57 known-tool requests, with no required tool omitted. The original per-request output was not retained, so this is a historical result, not an independently reproduced current score. Schema-token estimates do not establish billing savings or task success. Source: 345269e · docs/todo.md · routing_recall.py.
One server between the pilot and everything else
The desktop client holds no API keys and no database connection. A small FastAPI server is the only thing that talks to Postgres, the model box, the TTS engine, or the prompt store — every request authenticated by a JWT from a one-time Discord login. It also holds the prompts: pilots never receive the persona text, and a Hub tag is the release gate.
"ALICE surfaces computed facts. The pilot makes the judgment calls."
Design principle, applied twice — the same rule that keeps Finalize Leg / Finalize Run manual-only.How it avoids acting on bad information.
What each safeguard does, as built in the current release.
Topic gate Built
Every utterance is classified as a request, social talk, or off-topic before the main model runs. Off-topic input gets an in-character decline; greetings get a short separate reply.
Actions only through tools Built
Only a tool call changes anything. The persona forbids claiming an action no tool performed, and a failing tool tells the model the action did not happen.
Typed arguments Built
Every tool argument is a validated Pydantic field. Spoken prices, quantities and names are converted before they reach a tool.
Manual finalize Built
Finalizing a leg or a run is never done by voice or by the model. The pilot confirms it in the overlay.
Hedged name matching Built
Ship, terminal and commodity names are resolved through aliases and known mishearings. A weak match produces a question back to the pilot instead of a guess.
Spend and rate limits Built
The server counts upstream market-data calls per day and serves cached data when the budget is spent. Hosted-model fallback is capped by a daily spend limit and a per-pilot request limit.
Every choice, with a reason.
| Layer | Choice | Why |
|---|---|---|
| LLM orchestration | LangChain | Structured output, provider-agnostic model binding |
| Conversational graph | LangGraph | Explicit guardrail/tool-calling loop as a graph, not prompt logic |
| Observability | LangSmith | Hosts server-side prompts; optional development tracing is off for deployed pilot traffic |
| Schemas | Pydantic | Structured LLM output and tool args |
| Trade-run storage | Postgres + SQLAlchemy + Alembic | Durable, queryable ledger — not a semantic-retrieval problem |
| Desktop overlay | PySide6 | LGPL licensing vs. PyQt6; real layout power vs. tkinter |
| Speech-to-text | faster-whisper | Runs locally on the pilot's machine; model size set by one env var |
| Text-to-speech | Kokoro (self-hosted); optional ElevenLabs fallback | Self-hosted speech; the implemented ElevenLabs fallback is disabled in the current deployment |
| Model serving | Ollama (gemma4) + Claude Haiku 5.5 fallback | Self-hosted inference with a hosted fallback for unavailability or capacity limits; daily spend is capped |
| Topic decision | Tev1, with chat-model fallback | Dedicated request classification and tool-group selection; the chat model handles fallback decisions |
| Server | FastAPI + Discord OAuth/JWT | Keeps provider keys and database access on the server; pilots receive a session token |
| Evals | Custom harness + Claude Haiku 5.5 judge | Real graph with repeatable fixture data; model responses and judge scores can vary |
| Packaging | PyInstaller + Inno Setup | A one-folder Windows build: a one-file executable unpacks itself on every launch and is the shape antivirus tools flag most |
| Deploy | Docker Compose on Hetzner | One box, one command; the pilot's client is never containerized |
What ships, and what remains.
Client status reflects ALICE 0.2.10. Evaluation measurements are historical.
Evals Built
Every phrasing of every case runs through the real graph against a deterministic fake world with real Stanton geography. Scored per dimension — topic gate, tool, arguments, location, price, unit, unsupported claims — with Claude Haiku 5.5 judging free text. Used to choose the serving model across ten-odd local candidates. A separate voice evaluation scores character, naturalness and restraint on a fixed judge model.
Tool routing Built
A small decision model picks the tool groups ALICE binds for each request, on by default, with a setting to turn it off. The 18 October record reports reduced schema-token estimates; see the evidence limitations in the architecture notes above. This does not measure correct tool use or completed tasks.
Security hardening Next
Prompt-injection and jailbreak defense, beyond topic filtering. Every tool argument is already Pydantic-validated — a baseline, not a solution.
Memory Partly evaluated
Within a session ALICE sees the last six pilot turns, with older tool results replaced by a placeholder. An opt-in session-recall experiment on 18 October answered 5 of 6 seeded recall questions, compared with 0 of 6 without it, across 36 follow-ups. This small synthetic test is not a production accuracy estimate; the feature is off by default. Durable memory across sessions is not built.
In-game presence layer Designed · not shipped
A separate, glanceable display for a mini tracker, timers and route cards. Not present in the 0.2.10 client. The desktop workbench — Search, Runs, Ledger and Options — is already shipped; this is an additional in-game display.