Engineering notes

Inside ALICE.

The product page covers what she does for a pilot. This one covers how — the graph, the server, the guardrails against acting on bad information, the stack, and what is not built yet.

16 tools, one agent Prompts held server-side Every tool arg Pydantic-validated
Architecture

Not a wrapper around one API call.

A topic gate sends each utterance to respond ⇄ tools, a short social reply, or an in-character decline — see app/graph.py.

User — push-to-talk voiceInput
↓ speech → text · faster-whisper, on the pilot's machine
classify_topicRequest, social or off-topic · small decision model, chat-model fallback
↓ request  ·  greeting  ·  off-topic
respond ⇄ toolsLoop
declineIn character
16 tools · UEX Corp market data · Star Citizen Wiki (reverse-engineered) · trade-run ledger · timers
↕ HTTPS + JWT · ledger server: Postgres, UEX/wiki cache, LLM & TTS gateways
↓ text → speech · Kokoro, via the TTS gateway
Reply — spokenOutput

Thread vs. Run

A thread (thread_id, MemorySaver) is the conversation a pilot is having with ALICE, held in memory for the session. A run is one graph.ainvoke() cycle inside it — one LangSmith trace when tracing is on. Trade-run state lives in the ledger, not in the thread.

16 tools, routed per request

The released client registers 16 tools in app/graph.py. A small decision model picks relevant tool groups for each request; uncertain or unavailable decisions give the chat model all tools. The 18 October record reported about 43% fewer schema tokens across 57 known-tool requests, with no required tool omitted. The original per-request output was not retained, so this is a historical result, not an independently reproduced current score. Schema-token estimates do not establish billing savings or task success. Source: 345269e · docs/todo.md · routing_recall.py.

One server between the pilot and everything else

The desktop client holds no API keys and no database connection. A small FastAPI server is the only thing that talks to Postgres, the model box, the TTS engine, or the prompt store — every request authenticated by a JWT from a one-time Discord login. It also holds the prompts: pilots never receive the persona text, and a Hub tag is the release gate.

"ALICE surfaces computed facts. The pilot makes the judgment calls."

Design principle, applied twice — the same rule that keeps Finalize Leg / Finalize Run manual-only.
Guardrails

How it avoids acting on bad information.

What each safeguard does, as built in the current release.

Topic gate Built

Every utterance is classified as a request, social talk, or off-topic before the main model runs. Off-topic input gets an in-character decline; greetings get a short separate reply.

Actions only through tools Built

Only a tool call changes anything. The persona forbids claiming an action no tool performed, and a failing tool tells the model the action did not happen.

Typed arguments Built

Every tool argument is a validated Pydantic field. Spoken prices, quantities and names are converted before they reach a tool.

Manual finalize Built

Finalizing a leg or a run is never done by voice or by the model. The pilot confirms it in the overlay.

Hedged name matching Built

Ship, terminal and commodity names are resolved through aliases and known mishearings. A weak match produces a question back to the pilot instead of a guess.

Spend and rate limits Built

The server counts upstream market-data calls per day and serves cached data when the budget is spent. Hosted-model fallback is capped by a daily spend limit and a per-pilot request limit.

Tech stack

Every choice, with a reason.

LayerChoiceWhy
LLM orchestrationLangChainStructured output, provider-agnostic model binding
Conversational graphLangGraphExplicit guardrail/tool-calling loop as a graph, not prompt logic
ObservabilityLangSmithHosts server-side prompts; optional development tracing is off for deployed pilot traffic
SchemasPydanticStructured LLM output and tool args
Trade-run storagePostgres + SQLAlchemy + AlembicDurable, queryable ledger — not a semantic-retrieval problem
Desktop overlayPySide6LGPL licensing vs. PyQt6; real layout power vs. tkinter
Speech-to-textfaster-whisperRuns locally on the pilot's machine; model size set by one env var
Text-to-speechKokoro (self-hosted); optional ElevenLabs fallbackSelf-hosted speech; the implemented ElevenLabs fallback is disabled in the current deployment
Model servingOllama (gemma4) + Claude Haiku 5.5 fallbackSelf-hosted inference with a hosted fallback for unavailability or capacity limits; daily spend is capped
Topic decisionTev1, with chat-model fallbackDedicated request classification and tool-group selection; the chat model handles fallback decisions
ServerFastAPI + Discord OAuth/JWTKeeps provider keys and database access on the server; pilots receive a session token
EvalsCustom harness + Claude Haiku 5.5 judgeReal graph with repeatable fixture data; model responses and judge scores can vary
PackagingPyInstaller + Inno SetupA one-folder Windows build: a one-file executable unpacks itself on every launch and is the shape antivirus tools flag most
DeployDocker Compose on HetznerOne box, one command; the pilot's client is never containerized
Release status · reviewed 28 October 2026

What ships, and what remains.

Client status reflects ALICE 0.2.10. Evaluation measurements are historical.

Evals Built

Every phrasing of every case runs through the real graph against a deterministic fake world with real Stanton geography. Scored per dimension — topic gate, tool, arguments, location, price, unit, unsupported claims — with Claude Haiku 5.5 judging free text. Used to choose the serving model across ten-odd local candidates. A separate voice evaluation scores character, naturalness and restraint on a fixed judge model.

Tool routing Built

A small decision model picks the tool groups ALICE binds for each request, on by default, with a setting to turn it off. The 18 October record reports reduced schema-token estimates; see the evidence limitations in the architecture notes above. This does not measure correct tool use or completed tasks.

Security hardening Next

Prompt-injection and jailbreak defense, beyond topic filtering. Every tool argument is already Pydantic-validated — a baseline, not a solution.

Memory Partly evaluated

Within a session ALICE sees the last six pilot turns, with older tool results replaced by a placeholder. An opt-in session-recall experiment on 18 October answered 5 of 6 seeded recall questions, compared with 0 of 6 without it, across 36 follow-ups. This small synthetic test is not a production accuracy estimate; the feature is off by default. Durable memory across sessions is not built.

In-game presence layer Designed · not shipped

A separate, glanceable display for a mini tracker, timers and route cards. Not present in the 0.2.10 client. The desktop workbench — Search, Runs, Ledger and Options — is already shipped; this is an additional in-game display.