Architecture
The design reference, kept current with the code. It was written stack-independent, before
the language was chosen (Rust, 2026-08-29).
docs/DECISIONS.md records why each choice was made. docs/EXPLAINER.md is the narrative
version for someone new to the ideas.
1. Goal
Section titled “1. Goal”A Discord bot that answers Magic: The Gathering rules questions. Users reference cards by full name, partial name, or nickname (“bob” → Dark Confidant). Every answer cites its sources: Comprehensive Rules (CR) sections, Scryfall rulings, and/or prior rated calls. Answers and user-submitted calls are rated 1–3 (incorrect / partially correct / correct), the two that find fault with a reason for the operator’s feedback queue. Ratings feed back into retrieval as examples and never outrank the CR; reasons never reach the model.
2. Entity-first hybrid retrieval
Section titled “2. Entity-first hybrid retrieval”Rules questions have the shape “card A + card B + rule concept C”, and the design follows from that:
- Card names are entities → extraction + lookup, not embeddings.
- CR is a numbered hierarchy → chunk at the rule level (e.g.
702.19with all lettered sub-rules andExample:lines), keep parent links, and map question categories to whole subsections (e.g.613,614). - Scryfall rulings are keyed by card → fetch directly once cards resolve.
- Three retrieval sources for the fuzzy part: category map (structured), BM25/full-text (keywords like “leaves the battlefield”), vector (semantic).
- The CR always outranks prior calls.
3. Pipeline
Section titled “3. Pipeline”Discord message (+ last N Q&A in the same thread) │ ▼[1] Entity extraction (LLM, the model's measured effort, structured output) Splits the message into card-name spans and rules concepts. Runs FIRST so fuzzy matching sees only candidate spans, not rules vocabulary. Model choice: `judge_bot::config` picks a provider per stage, from a `judge.toml`, else Anthropic direct from the environment. A provider is either Anthropic's Messages API or any OpenAI-compatible chat completions server. The Messages API is reached direct, through a proxy, or on a cloud account: Claude Platform on AWS and Bedrock (SigV4), Vertex AI (ADC). Cloud credentials come from the platform chain and are probed at startup. Every model sits behind the process's one spend-capped `Metered`, and the embedder behind a `MeteredEmbedder` on the same meter. A backend reports its `Capabilities`. If it cannot enforce the output schema server-side, the adapter appends the schema to the *user turn*. The system prompt is pinned by digest and is byte-identical on every backend. │ ▼[2] Card resolution (per span) A typed resolution order, each step tried only when the one above found nothing: alias → possessive-stripped alias ("bob's" → "bob") → exact name → printed-name table → short name before the comma ("Ragavan") → the name after the comma, a leading "the" dropped, two words or more ("Nimble Pilferer"; a one-word "ravager" goes on to fuzzy, since players mean other cards by it). When another card's name holds the same words ("Grand Arbiter Augustin IV" for Dovin, Grand Arbiter's "Grand Arbiter"), every such card is offered instead: Ambiguous, never resolved → alias as a suffix → trigram fuzzy A [[bracketed]] span has its own resolution order: exact name → printed name → alias (whole span). It is CardSpan::Exact, chosen by an exhaustive match, so no looser step can resolve it. An alias maps one spelling to one card, so [[bob]] resolves. A miss is never resolved, only offered as Ambiguous: - the cards the other naming steps (possessive, short name, the name after the comma, alias suffix) point at, under that step's matchedVia, so [[bob's]] offers Dark Confidant. Dropped as a duplicate when the extractor also named the card. - with no naming-step hit, the fuzzy neighbours. Output: Resolution = Resolved(card, matchedVia) | Ambiguous(candidates) | NotFound Duplicate references (nickname + full name) are dropped. A span is a duplicate when it is - Ambiguous from a non-fuzzy step and shares a candidate with a card Resolved from another span, or - NotFound, but its words appear as whole words in a resolved card's (face) name. Fuzzy-ambiguous spans are never dropped: trigram neighbours are not names the user could have meant. Remaining Ambiguous ⇒ "did you mean…?" buttons and stop. Never guess. Each loaded card carries its Release (D42): released, or upcoming (first printed after today) and then covered or not by the loaded CR. Covered means the CR is dated at most 21 days before the card's first printing and mentions every type, subtype and rules keyword on it. │ ▼[3] Classification (same LLM call as [1]) One required primary category plus up to 2 secondary, each with confidence, from a fixed taxonomy (~25 entries mirroring CR structure). The schema requires the primary, so an unclassified answer is an API error. Also assigns Source: CR | Commander | Tournament(MTR/IPG) | OutOfScope. Tournament/OutOfScope ⇒ reply "I don't cover that" with a pointer. Flags "nightmare" cards (Humility, Opalescence, Blood Moon, Mycosynth Lattice, Panglacial Wurm, …) → inject hand-written notes. │ ▼[4] Retrieval → Context - CR: category → curated subsection IDs (always) + tsvector BM25 + pgvector cosine. Union, dedupe, expand to full rule chunk. Order matters: the synthesis budget (25 chunks / 30 kB) renders only a prefix, and the sources return several times that. Priority: 1. the primary category's rules sharing a word with the question (ranked by text relevance, not by id) 2. BM25 3. vector 4. the primary's remaining rules 5. secondary categories (ranked) `eval recall` gates on what was retrieved (≥ 90%) and on what that prefix shows (≥ 75%). - Scryfall rulings for each resolved card (all faces) - Glossary entries for terms in the oracle text - Prior calls: vector search filtered by (cards ∩ category), labeled with rating and CR version, shown as examples AFTER the CR material. Retired calls are excluded, and so is a call about a card that was not out when it was answered (D42), for good. The refresh's retirement pass retires a call when a citation's source no longer contains its quote, or a context card's Oracle text changed. It restores the call when both hold again. Renumbered rules carry their calls with them. - Nightmare notes - Thread history (last N Q&A) │ ▼[5] Synthesis (LLM, the model's measured effort: medium on Opus 5.5, high otherwise; structured output, one tool-use round) Same per-stage provider choice as [1]. The tool round and the schema travel in each backend's wire format (strict function calling on OpenAI-compatible servers when the provider says it has it). Tool: lookup_rules(ids). The model may request additional CR sections once before answering, closing the classifier-miss gap. Output: Verdict { answer, confidence: Low|Medium|High, citations[], category } The model does not report `source`, `crVersion` or `cards`. Validation stamps them: - source from the extraction, as AnswerableSource (by type, only Cr | Commander reach this step) - crVersion from the retrieved chunks - the resolved cards (CardRef: id + name + status) from Context Every interface then shows "Cards: …" from the verdict alone, and a note above the answer naming any card not out yet that the loaded CR may not cover (D42). The material says the same under that card's heading. Each citation = typed reference + quoted span. Validation: - first, citations that quote nothing (blank, a stock word, under 4 chars) are dropped (D21). An answer with nothing else is a MalformedCitation. (a) the reference exists in Context (b) the span is a substring of that chunk. Curly and ASCII punctuation compare equal (judge_core::quote). The chunk's own text is stored for the span, so a stored quote is exact. (c) the verdict cites something and the answer is ≥ 40 chars, else JudgeError.EmptyVerdict (d) every rule number in the answer text is covered by a rule citation, else UncitedRules (D20) The answer text is an Answer newtype: a stray `\n`, `\t` or `\"` the model wrote as two characters is decoded when the verdict is made (D22). Failure ⇒ BadCitation / MalformedCitation / EmptyVerdict / UncitedRules → retry once (the notice says which), then reply with error. A misused tool round (ToolMisuse: a second lookup_rules call, or one whose ids cannot be read) produces no verdict and is retried the same way. The retry may make the one call only if the first could not be read, since nothing was fetched. A retry after UncitedRules is given the uncited rules it was not shown (`fetch_uncited`: looked up, or pinned if retrieval held them and the budget cut them), since a number reached through a glossary pointer is often not in the material. Always quotes CURRENT Oracle text (errata note if the printed text differs). │ ▼[6] Persist call (question, verdict, context ids, crVersion); rating buttons.The two model calls ([1] and [5]) and the vector search of [4] reach their providers through
one seam (docs/PROVIDERS.md).
crates/llmholds the provider-neutral request and response, the spend cap and the one-tool-round typestate.ChatRequesthas system blocks, turns, tools, an output schema and an effort.ChatResponsehas text, tool calls, a stop reason and usage.- The cap is a process total plus one adjustment.
judge_bot::budgetsets it from thespend_daysledger whenJUDGE_BUDGET_PERIODisdayormonth, which makes the cap a shared budget that survives restarts (D19). It also tellsJUDGE_ALERT_WEBHOOKwhen the cap trips, and when spending reaches aJUDGE_SPEND_WARN_ATthreshold (claimed inspend_warnings, so one process sends each).judge-cli statsand the admin dashboard read the same ledger.
- The cap is a process total plus one adjustment.
crates/anthropicandcrates/openaiare backends of it. Each owns its wire types and its schema-subset transform.- The model’s own previous turn is replayed as an opaque blob that only the backend that
produced it reads (a thinking signature, a
reasoning_content, atool_callsarray). The neutral layer never inspects it. - The pipeline’s port,
ChatModel, is sealed and implemented only byMetered, so a model that bypasses the cap is unrepresentable. The embedders’ port,WithSpace, is sealed the same way (judge_embed::MeteredEmbedder), on the same meter. - Each run also has its own
judge_core::Tally(D33). Every port that can spend takes it, andChatModel::completetakes aBillnaming the run and theStage. The two metered types add aCharge(stage, provider, the model the response named, tokens,Cost) when they settle with the meter. A response with no usage keeps its reservation on the meter and is charged at it asCost::WorstCase. - Each backend declares
Capabilities. What a server cannot enforce (an output schema, strict tools) the adapter moves into the prompt, since decoding and citation validation are client-side either way.
judge_bot::config reads a judge.toml into that: one provider per stage, secrets by
environment-variable name, and a price for every model the cap must reserve for. For
embeddings it also reads the vector Space (provider kind, model, width). The database
records that space in embedding_space, and every vector reader and writer checks it
before touching a column. Two models’ vectors are therefore never mixed, and switching is
one explicit, transactional ingest reembed. crates/core depends on none of it.
One binary, judgebot (crates/judgebot), runs the long-lived interfaces. What a
process does is a set of roles chosen at launch: --discord, --api, --web, --mcp,
--admin and --jobs, on the command line or else in JUDGE_ROLES.
Roleis exhaustive and the set is aNonEmpty, so a process that does nothing is unrepresentable.roles::planchecks every role’s requirements before the pool connects or anything binds, and reports every unmet one at once:--discordneedsDISCORD_TOKENandJUDGE_OPERATOR_DISCORD, the network rolesJUDGE_OPERATOR_EMAIL,--weba built page,--mcpanMCP_TOKEN, and--adminanADMIN_TOKENof its own (equal to neither MCP token) and a built page. The serving roles’ models are built there too, so a missing or unpriced chat model is reported before the database is touched. What it returns is the types the roles run on, so a role cannot start unchecked.--jobson its own with the schedule off is refused, since it would have nothing to do.- The roles of one process share one composition (
judge_bot::serving::Serving): one pool, oneModelsbehind oneSpendMeterand one budget ledger, oneVectors, one migration at startup. Each role keeps its own concurrency slots and limits. The HTTP listener is bound before the Discord gateway is contacted, then both run side by side, and the first to stop ends the process with a non-zero status. DISCORD_INTERACTIONS(discord::Delivery, D30) says how Discord delivers interactions.http: the Discord role takes no lease and opens no gateway; it reads the application’s id and public key over REST (retrying until they come), registers the commands, starts the symbol watcher and opensdiscord::endpoint::Interactions, which the listener serves at/discord/interactions(the network roles’ listener, or with--discordalone one onAPI_ADDRserving only it and/api/health). A request is verified (Ed25519,401otherwise), a PING answered inline, anything else dispatched on a task holding aWorkguard, and answered202once the handler’s initial response has gone through the REST callback (reply::Ack), at most 2.8 s after arrival. Until the endpoint opens the listener holds its port without listening (at most 30 s), so dials go to another container; a request that finds it closed gets503, and/api/healthfails. Both modes compare the setting with the application’s Interactions Endpoint URL at startup and report a mismatch (ERROR, alert webhook) without stopping. Both feed one dispatch (Data::dispatch, each interaction on a task of its own so a panic is answered like an error).- Over the gateway, of the processes running
--discordon one database, one holds the gateway (judge_bot::discord::gateway). The Discord role takes aGatewayLeasebefore it connects, or stands by until the holder lets go, with no bound on the wait and reconnecting if its own connection drops; its HTTP roles serve meanwhile. Having taken it, it waitsTiming::grace(15 s), unless the holder before it recorded a clean release at shutdown within 5 s (gateway_handover), checks it still holds it, and connects. The holder checks every 5 s (4 s timeout). A failed or unanswered check closes the gateway (discord::shard’s stop: a close frame with 1000, no event after it) and, if it closed within 1 s, stands by again inside the role; otherwise the role ends with an error and the process exits. The grace exceeds the check interval, the check’s timeout and the shutdown limit together (10 s), a relation checked at compile time, so an old and a new holder never answer side by side. A holder in a paused VM that loses its session is the case it cannot cover (docs/DEPLOYMENT.md§8). Every lease session sets server-side TCP keepalives (lease::SESSION_SETTINGS), so a vanished holder’s lock is freed in about 25 s, and turnsstatement_timeout,idle_session_timeoutandtransaction_timeoutoff. A loss soon after the last one pauses before standing by again (30 s doubling to 10 min) to stay inside Discord’s 1000 logins a day per token, and a failed command registration is a warning, never a connection left serving nothing. The role’sDatais oneArcacross reconnects, and the symbol watcher a connection starts is aborted with it.holdtakes the gateway as a closure handed aStop, so the hand-off is tested without Discord. The scheduler’s thread is different: beside serving roles its end is logged as an error and the process keeps answering, while a process whose only role is--jobsexits non-zero. - SIGTERM or Ctrl-C requests a graceful shutdown (
judge_bot::shutdown, D29), oneShutdownper process held byServing. The HTTP listener closes and answers the requests it accepted (axum’s graceful shutdown). The Discord role closes its gateway and releases the lease the moment the connection is closed, recorded as clean (gateway_handover) when it closed withinTiming::shutdown_limit, so the next holder connects without the grace; a standby stops waiting.discord::runtakes aWorkguard for each interaction as the gateway delivers it, held by the task that answers it, and the role returns once they are all dropped. Everything is bounded byDRAIN_LIMIT(120 s) from the request; compose’sstop_grace_periodis that plusEXIT_MARGIN(a test). With both kinds of role,mainwaits for both rather than the first, then flushes the spend ledger (bounded byFLUSH_LIMIT, 5 s) and exits 0. A gateway slow to close keeps the lease until it has, at most until the limit. A signal before the roles start, or a second one, exits at once. - Startup and the schema (D32).
migrate::upgradedecides what the pending migrations are. Within a major they only add, so they run first and the process opens (maintenance::Availability::open) beside any previous release still draining. A major upgrade’s (another major, a new major’sX.0.0-*pre-release, or a database from before 2.0) cannot run beside the previous release: the roles start at once, closed (Availability::upgrading), and answer every question with the maintenance reply (Discord: the question restated with a Retry button,render::RetryPrompt, read back from the message as a pick is;/api/judge: 503maintenancewithRetry-After;/mcp: the message as the tool’s error;/api/health: 503 through theMaintenanceprobe, after the Discord endpoint’sOpensoscripts/deploy.shcan read amaintenance:answer as “up”). Every process with a role holds the shared side ofSCHEMA_LOCKbefore it reads the schema, andSchemaHold::keepchecks it every 10 s beside the roles (retaken only while the ledger holds nothing unknown and no major holds the exclusive side, else the process exits; a database that does not answer, as whiledeploy.sh --dbreplaces it, is tried again every check). The upgrade waits for the exclusive side (lease::SchemaHold::exclusive, in 5 s attempts that give up once another process of its release has migrated), which it gets once every older process has exited, migrates, downgrades to shared, opens, and only then starts the scheduler. A process starting meanwhile queues behind it. A rollback is refused under the migration lock (migrate::rollback_refusal) unless it stays within a major. judgebot ingest <cmd>is the data command line.judgebot backup <run|list|fetch|serve>is the database backup (judge_bot::backup, D26):pg_dump -Fcover the network, gzipped, uploaded to an S3-compatible store (BACKUP_S3_*: endpoint, region, path-style or virtual-hosted) by aSigV4client of four requests, pruned pastBACKUP_KEEP_DAYSafter the upload.servechecks the bucket hourly and takes a backup when the newestjudgebot-<stamp>.dump.gzisBACKUP_EVERY_DAYSold. It writes whatscripts/backup-db.shwrites, so each reads the other’s backups.docker-compose.ymlruns the application as one long-running service,judgebot, whose command isJUDGE_ROLESor, unset,--discord --api --web --jobs:roles::COMPOSE_COMMAND, which a test holds the file to. A setAPI_INTERFACES, the retired variable that chose the oldapiservice’s interfaces, refuses the launch with theJUDGE_ROLESline that replaces it.roles::compose_rolesis the same rule in Rust, sojudge-configchecks only the roles the service would run. It has nocontainer_nameand publishes no port, soscripts/deploy.shcan run a second container of it beside the first (blue-green, D29): start the new one, wait for Docker’s healthy,docker stopthe old one, which drains. Theproxyservice (Caddy,proxy/Caddyfile) publishes127.0.0.1:8787, carries the network aliasapi(the old service name a tunnel may point at; the runbook’s isproxy:8787) and forwards tojudgebot:8787, whichever container listens, on a new connection per request with dial retries for 5 s: a draining container closes its listener, so a refused dial goes on to the other, and no kept-alive connection to it can be closed under a POST. A connection the kernel had queued for it as it closes its listener is reset, which is the only loss that remains. Its:8787site (the tunnel’s) and its admin site removeX-Judgebot-Client-IP; the host’s127.0.0.1:8787is its:8788site, which passes it on from a reverse proxy on the host (any container on the compose network can dial it too, so nothing there points at it). Both Caddyfiles drain for 70 s (grace_period), under compose’s 80 sstop_grace_period.edge(profilepublic,proxy/edge.Caddyfile, D41) is the bundled public site: Caddy on ports 80 and 443 servingJUDGE_DOMAINwith a Let’s Encrypt certificate, diallingjudgebot:8787with the same handover settings and settingX-Judgebot-Client-IPto the visitor’s address.cloudflared(profiletunnel) is the alternative route, throughproxy:8787.refreshrunsjudgebot ingeston demand, andbackup(profilebackup,.env.deployand no.env) runsjudgebot backup serve.
Three interfaces share this pipeline through the same composition root
(judge_bot::build_deps):
- Discord adapter (
crates/bot, the--discordrole):/judgeslash command, rating buttons, “did you mean…?” buttons, thread history.- No Discord library (D43):
discord::wireis Discord’s JSON as our types (only the fields used; a modal’s components kept as sent),discord::restthe REST calls (a 429 waited out within a bound, nothing else retried, no token in an error),discord::shardthe gateway connection (zero intents,READYandINTERACTION_CREATEonly; rustls 0.23 with an explicitringprovider), andendpoint::Verifierthe Ed25519 check overed25519-dalek. - Every interface hands the store an
Origin(judge_core::origin, D31): the interface, the Discord asker (id and username) and whether the call is private.CallStore::persistandrecord_failurestore it;Origin::history_scopesays which earlier calls of the thread a question reads. - Every user-facing string comes from a per-language catalog (D35): core’s
(
judge_core::locale: the source offer, the contact, the freshness lines), Discord’s (discord::catalog), the alerts’ (judge_bot::alert::catalog), the API’s (judge_api::catalog), the command lines’ (judge_agent::catalog,judgebot’scatalog), judge-config’s (judge_configure::catalogand its page’s), the web app’s (web/src/messages.ts,web/src/admin/messages.ts) and the docs site’s (site/src/i18n.ts). Each surface picks its value throughjudge_core::locale::Catalogs::new, which takes every locale’s, the test-only stub locale’s included. A request’sLocaleis the asker’s language (the Discord user’s; the browser’s; a terminal’sLC_ALL/LC_MESSAGES/LANG), elseJUDGE_LOCALE; an alert is inJUDGE_LOCALE(AlertWebhook::locale). Pick and Retry ids carry the locale their message was written in. - Every interface runs a question through
judge_bot::run::run(D33): judge, store the call or the failed call, and record the run injudge_runswith its charges. It returns the context the answer was judged from, the run’sOutcomeand cost. - A rating click (D38) reads the rater’s earlier rating of the call
(
CallStore::earlier_rating, bounded at 1.5 s from arrival, nothing deferred before it) andjudge_core::feedback::next_stepdecides: the same Correct again is “already recorded”, any other earlier rating gets a private Replace / Keep prompt, else Correct is recorded (deferred, then the judge-role lookup andCallStore::rate) and a fault asks its reason: one modal, written as JSON (discord::feedback::modal: the category menu and both texts in Label components), its values read from the interaction’s components as Discord sent them, however it was delivered (D43). A category menu (rate-cat:) an older build sent over the gateway still opens a modal of the texts in action rows, the category in its id (ids::FormId). The submission defers, looks the role up and stores the rating with its reason (Rating::faulted). Every step’s id carries the call and the score; maintenance answers each with the maintenance reply. /judge-privateisAudience::Private: acknowledged ephemerally, no history read, persisted private (with rating buttons), so it is never a prior call and never another question’s history. A call that fails is recorded infailed_callsfor any audience throughCallStore::record_failure(D27). The audience rides in the pick button’s custom-id through a card pick.- A card pick is answered from Discord alone, so any process holding the gateway
answers it, after any restart. The prompt message restates the question verbatim
above a body with no blank line in it: the lead line, the numbered choices (names cut
to 60 characters) and up to two notes (
render::PickPrompt, whoseparsemust re-render the content exactly). Each button’s custom-id (ids::Pick) carries the asker, the audience, the ambiguous span’s byte range in the question, a digest of the prompt’s content (pick::Digest, 8 bytes of SHA-256) and the card’s oracle id. - A click does not rely on Discord to check the custom-id against the message. It is
refused (as expired) unless the message’s content has the button’s digest, so a
button answers only the prompt it was made for, and unless the message’s EPHEMERAL
flag agrees with the button’s audience. A message whose flags are missing is
answered privately (
pick::audience), never recorded. The card must be on the prompt’s list. - The prompt’s age is read from the message’s last edit, else its snowflake id,
against a ten-minute TTL.
pick_claimsmakes each prompt single-use: one row per message, prompt time and digest, no text, taken after the judge slot so a “busy” leaves the buttons usable, and given back if the acknowledgement fails. Rows older than a day are deleted by the claims that follow./judge’s question option has amax_length(1300) under which the whole question fits the prompt. Longer content (UTF-16-heavy) gets the choices listed without buttons. - The click needs the message’s content. Discord’s interaction object carries “the message they were attached to” for components, ephemeral ones included as far as the docs say, and the message-content restriction exempts messages the app sends. The docs make no explicit promise for ephemeral messages, so a private prompt whose content arrives empty or altered is answered “expired”, never guessed at.
- A per-user fixed window (
discord/cooldown.rs,JUDGE_USER_LIMIT) is charged once a judge slot is held, so a “busy” is free. A card pick is not counted again. /cardand/ruleare lookups over the resolver, the retriever’slookup_rulesandPgLibrary::rulings. They call no model and touch no meter.
- No Discord library (D43):
- HTTP adapter (
crates/api+web/): anonymousPOST /api/judgebehind a per-IP fixed-window rate limit, and a SolidJS web app.- The limit’s address is
client_ipunderAPI_CLIENT_IP(config::ClientIpSource): the peer,CF-Connecting-IP, orX-Judgebot-Client-IP(ProxyHeader, read only as a single copy from a loopback or private peer), neverX-Forwarded-For. Every limiter keys onlimit::Bucket: the IPv4 address, or the IPv6 address’s /64. - Each interface is a role of its own on one listener (
API_ADDR):--apithe JSON route,--webthe web app and--mcpthe MCP transport. An interface nobody named is not mounted. GET /api/healthandGET /api/aboutare served whatever interfaces are off. The container healthcheck needs the first.- The public feed (D36):
GET /api/feed(keyset pages) andGET /api/calls/{id}, mounted beside the page under--webunlessJUDGE_FEED=false, overjudge_bot::feed, whose queries are the only public reads ofcalls(no private call, nothing older thanJUDGE_FEED_DAYS). They have their own per-address read limit. The page routes/feed,/calls/<id>and/users/<name>client-side (web/src/router.ts); an unrouted/api/path is a JSON 404. A user filter (the feed’s and the public dashboard’s) lists the calls of the Discord id behind the name’s newest public call, so a reassigned username does not list its earlier holder’s. A call the feed cannot read in this build is neither shown nor rateable (feed::rateablereads it the wayfeed::calldoes).publicly_visible(private, at, window_days), a SQL function the migrations define once and the planner inlines, is the “not private, within the window” predicate of every public read, so the feed’s partial indexes still serve them. - The public dashboard (D39):
GET /api/stats?from&to&days&interface&user&model, mounted with the feed, overjudge_bot::db::stats::public. It counts public calls and runs alone, throughpublicly_visible: questions per day by interface, outcomes (judge_core::OutcomeGroup), the rating mix (Discord and web apart), the most-asked categories, cards and askers. Cost (CostAccess) and models (ModelAccess) exist only throughstats::Disclosure, true only whenJUDGE_PUBLIC_COST/JUDGE_PUBLIC_MODELSis; amodelfilter without models is a 400. A range is 1 to 366 UTC days, in a read-only transaction with a 5 s statement timeout; a range ending before the window is a 400. The reads have their own per-address limit (30 a minute) and two slots per process. The page is/dashboard. /api/aboutis the source offer (judge_core::source): repository, built commit, licence and copyright. The AGPL requires every remote interface to offer it. The web app’s footer reads it from there. Discord gives the same in/helpand/license, and the MCP server in its initialization instructions and anabouttool.JUDGE_SOURCE_URLpoints all of them at a fork.Aboutalso carries the data’s freshness (judge_core::Freshness): the CR release loaded and the age of the last successful refresh, and whether the latest one failed. It is read fromrefresh_runsandrules(ingest::runs::freshness, within 2 s,nulland a WARN otherwise; a missing run table leaves the CR version alone). It is cached for a minute, single-flight, by aFreshnessReader:/api/abouthas one, and everyPgLibraryanother (/helpin the bot, the MCPabouttool), so a process with--mcpholds two./helplists it, the web app’s footer shows it as one line, and/api/healthignores it.- The same places name who runs the instance (
judge_core::operator). The bot takes aDiscordOperatorand the HTTP layer aNetworkOperator. The only way to make either isOperator::for_discord/for_network. So the bot cannot start withoutJUDGE_OPERATOR_DISCORD, and the network roles cannot start withoutJUDGE_OPERATOR_EMAIL, whichever of them it opens. A localjudge-clior stdiojudge-mcpholds a plainOperatorand needs neither. POST /api/calls/{id}/feedback(judge_api::rating, beside/api/judge, D38) rates a public call within the feed window as the tab’s session (Rater::Web,user_idweb:<session uuid>), with the address fromclient_ipand the user agent. The server is authoritative on replacing: an earlier rating withoutreplace: trueis a 409confirmand nothing is written. Its own limiter (rating::LIMIT), a 16 KiB body, unknown fields refused.calls_ratednever counts a web rating, because anonymous callers are not accountable identities./api/judge‘s answer carriescall(the stored call’s id). The feed’sratingscount Discord’s and the web’s apart;reasonsappear only withJUDGE_PUBLIC_FEEDBACK=true, without the raters’ identity, a dismissed queue item’s left out. Withoutreplacethe route writes withrate_if_new(insert only), so a race from one session ends in a 409, never a silent replace.- Ambiguity is returned as data and resolved statelessly. The client re-asks
with
pins: [{span, name}], which the server rewrites to[[Full Name]]with the samepin_cardused by the Discord buttons. - Follow-up history comes from a client-generated session UUID, stored as
thread id
web:<uuid>.
- The limit’s address is
- Admin interface (
--admin,judge_api::admin,web/src/admin/, D37): the operator’s view of everything stored, on a listener of its own.ADMIN_ADDR(default127.0.0.1:8789; the compose container binds0.0.0.0:8789and the proxy publishes it on the host’s loopback only, onADMIN_PORT). Reserved at startup, listening once the instance is open, drained on shutdown.roles::planrefuses anADMIN_ADDRonAPI_ADDR’s port./admin/api/*behindADMIN_TOKEN(constant-time bearer,no-store):about(JUDGE_LOCALE, the pages’ language when the browser’s has no catalog),calls(every call, private ones included, filtered by interface, asker, model, visibility and text; keyset pages),calls/{id}(origin, ratings with raters and reasons, runs with charges),failures,usage(the operator’s dashboard, D39:stats::usage, every call counted, with the spend ledger by day and month, the budget (the period and cap the ledger-keeping processes record inbudget_settingon each sync), the runs by model, interface and stage, cost per answer, latency, the dearest runs and the failed calls; the same range and filters as the public one, the asker by name or id), and the feedback queue (D38):queue(filtered by status, category, interface and rater kind; keyset pages),queue/{id}(the item with its call whole) and its two writes,POST queue/{id}/resolveand…/dismisswith{note, fixed_by}. The queries arejudge_bot::admin,judge_bot::feedbackanddb::stats. The network roles’ listener answers/adminpaths 404./adminand below serve the web app, whose admin pages are a chunk of their own. The page takes the token from the address’s#token=(judge-cli admin-url), keeps it insessionStorageand sends it only as a header.- The same reads are agent operations (
admin_calls,admin_call,admin_failures,admin_usage,queue_list,queue_show,queue_resolve,queue_dismiss), each taking ajudge_agent::admin::Adminthat onlyToolbox::from_env(a standalone toolbox) and theMCP_ADMIN_TOKENhandler hand out.
- Config editor (
crates/configure, binaryjudge-config): a page on127.0.0.1forjudge.tomland the settings in.env(D23).- The
judge.tomlform isjudge_bot::config::file_schema(): JSON Schema generated from the loader’s serde types, withx-endpointsfromEndpointKey::on, the table the loader’s misplaced-key check reads. env::VARSlists every variable.env.examplecarries as a secret or a setting. A value reaches the page only through aSetting. A secret, a variable the list does not know, or a hidden setting is reported as set or not, and is changed write-only (DotEnv::replace): the page sends a new value and every reply is scrubbed of it. Help text is.env.example’s comments.- Each draft goes through
check::run: thejudge.tomlloader,discord::Config::from_vars,ApiConfig::from_varsandcheck, the migration flag. Errors carryConfigError::location(), the key or variable to fix. - Writes go through
toml_editand a line-preserving.envwriter, so only changed lines move. They are refused when a file changed since it was read.
- The
- Agent adapter (
crates/agent): the judge as a tool surface for other agents.- Over MCP,
judge-mcpserves a local client on stdio. For a remote one,judgebot --mcpmounts the same handler at/mcpbehindMCP_TOKEN. The flag without a token is refused at startup. The token without the flag serves nothing and warns. - The operator’s reads are operations too (D37):
admin_calls,admin_call,admin_failures,admin_usage(judge-cli calls,call,failures,stats), and the feedback queue’s (D38):queue_list,queue_show,queue_resolve,queue_dismiss(judge-cli queue,queue-show,queue-resolve,queue-dismiss). Each takes anadmin::Admin, whichjudge_agentconstructs only beside a standalone toolbox (Toolbox::from_envreturnsLocal) and inmcp::admin_http_service; an instance toolbox (Client::instance) gets none. What a connection may call is its handler’s type:JudgeMcp<Public>(whatMCP_TOKENreaches) has no admin tools,JudgeMcp<Privileged>(stdiojudge-mcp, and/mcpwithMCP_ADMIN_TOKEN) adds them, and each admin call is logged at INFO with its tool’s name.judge_api::mcp::routertakes the public service by that type; the bearer scheme is matched case-insensitively. judge-clihas one subcommand per operation with JSON out, for a shell agent (the repo’s.claude/skills/judgeskill).- It offers the pipeline two ways:
- The
judgetool runs it as above, with the built-in model calls. It is spend-capped, shares the web route’s concurrency semaphore, and is offered only when a model is configured (ANTHROPIC_API_KEYor ajudge.toml). - A session runs it in pull mode, where the calling agent is the model.
- The
judge_bot::sessionis that state machine.beginreturns the extraction prompt (steps 1 + 3 as text, plus the JSON Schema).- The agent’s
ExtractionJSON drives steps 2 + 4 and yields the synthesis prompt: the same system prompt, withHarness-specific wording for the onelookup_rulesround and the output format, plus the rendered material. - The agent’s
VerdictJSON goes through the sameVerdict::validateagainst the session’s ownContext, with the same one retry and the same rejection notice.
- On the API path, typestates carry the invariants (one tool round, one
retry, only a validated verdict is persisted). Here they are a
Stageenum, because the state lives in Postgres between calls (agent_sessions, one jsonb document, optimistic version). The session id is a server-minted handle passed as a tool argument, which is what the 2026-07-28 MCP revision adopted in place of protocol sessions. The HTTP transport is served statelessly for every protocol version. - Sessions are unauthenticated at the tool level. Their thread ids are
therefore a type (
AgentThread, alwaysagent:<uuid>) that cannot name a Discord or web thread, and their inputs are bounded (question, spans, concepts, lookup ids, answer length). A persisted call is keyed by session (calls.session_id, unique), so persisting twice cannot file two calls. It is excluded from the prior-call query: nothing can rate it, and it is history for its own thread only. - Over HTTP,
judgeruns are also capped per window (MCP_JUDGE_LIMIT) so a leaked token cannot take the public web app’s slots and spend cap with it. - Read-only lookups (resolve a card, rules by id or search, rulings, notes,
glossary) are the rest of the surface (
PgLibrary).
- Over MCP,
The Discord and web interfaces draw Magic’s card symbols ({W}, {2/U}, {T}) as
pictures, from one set of names:
- Discord substitutes application emoji (
<:mana_w:…>). The bot owns them, not a server, so they work in every guild and cost no emoji slots.ingest emojiuploads them: Scryfall’s SVG → a 128 px PNG via resvg, scaled to fit and centred (nine of the 84 symbols are not square).- The emoji name is defined once, in
judge_core::symbol, with no I/O. It is in core because the uploader and the renderer are separate programs that must agree on it exactly. A test pins the mapping as total and injective over everything Scryfall publishes, so no symbol can silently overwrite another’s emoji. - A tag is ~28 characters where
{W}is three, and Discord counts the tag.mana::Renderedkeeps text as segments and lets only plain text be cut, so a half-written tag is unrepresentable rather than tested against. - An application with no emoji uploaded gets an empty table and the literal
{W}, unchanged. - The table sits behind a swappable
mana::SharedSymbols; a render takes a snapshot.discord::symbols::watch, a task spawned onReadyso connecting never waits on it, reads the run record’s mark, lists the emoji, then every ten minutes readsrefresh_runsfor a run finished since the mark whoseemojistep may have uploaded (runs::emoji_since: anything but skipped oruploaded: 0). It lists them again after such a run, after a failed listing, while the table is empty, and hourly regardless, which catches uploads and deletions the record does not show. So a refresh by any process (a--jobsrole, a cron’d or manualjudgebot ingest) or an upload by hand reaches a running bot without a restart.
- Web renders Scryfall’s SVGs inline from their CDN (
web/src/symbols.tsis the generated table,Symbols.tsxthe component).- A symbol the table does not know, or an image that fails to load, falls back to the literal text.
split/lookupmirror the Rust scanner, so both surfaces accept the same spellings ({W/U},{w/u},{U/W},{WU}) and leave the same text alone.- Seven symbols (
{E} {P} {PW} {CHAOS} {TK} {L} {D}) are flat black with no disc, invisible on the dark palette. They carry aflatflag and are inverted in dark mode. The coloured ones must not be. - If a Content-Security-Policy is ever added to the HTTP adapter,
img-srcmust allowhttps://svgs.scryfall.io.
4. Data
Section titled “4. Data”| Source | Refresh | Storage |
|---|---|---|
Scryfall bulk oracle-cards.json |
daily (the scheduled refresh judgebot --jobs runs, judge_bot::jobs, every JUDGE_REFRESH_HOURS — DEPLOYMENT.md §7) |
cards (oracle_id, name, layout, type_line, first_printed — the earliest printing’s date from default-cards, D42 —, …) + card_faces (oracle_id, face_idx, name, oracle_text, mana_cost, …) |
Scryfall bulk default-cards.json (names only) |
daily | printed_names (printed_name, oracle_id) — old names, errata’d names |
Scryfall bulk rulings.json |
daily (bulk-loaded, keyed by oracle_id) | rulings (oracle_id, key, published_at, text) — key = content hash (judge_core::ruling_key), so a reindexed ruling keeps its identity |
| Comprehensive Rules txt | on CR release — detected daily from the .txt link on Wizards’ rules page vs max(cr_version) |
rules (id, parent_id, subsection, heading, body, examples, embedding, cr_version) |
| CR Glossary | same | glossary (term, text, embedding) |
Scryfall /symbology (84 card symbols) |
daily (idempotent, uploads only missing symbols) | not stored: uploaded as Discord application emoji (ingest emoji) and hard-coded for the web app (web/src/symbols.ts) |
| Nicknames | hand-curated YAML, compiled in; the refresh reloads a built-in copy an upgrade changed | card_aliases (alias, oracle_id) |
| Nightmare notes | hand-written markdown, likewise | card_notes (oracle_id, note) |
| Curated list sources | written by each alias or note load, in its transaction | curated_lists (list, source builtin/file, digest, loaded_at): the refresh’s lists step reloads only a builtin list whose digest is not the binary’s (judge_bot::ingest::lists) |
| Categories → subsections | YAML (single source of truth; the enum is generated from it) | categories |
| Calls | continuous; retired_at/retired_reason recomputed on each refresh from citation validity; deleted with their ratings past JUDGE_CALL_RETENTION_DAYS by the refresh’s prune step (D34) |
calls (id, thread_id, interface, asker_id, asker_name, private, locale, question, answer, category, citations jsonb, source, cr_version, retired_at, retired_reason, embedding, unreleased_cards — the cards its reply noted as unreleased, D42; a private call has no embedding) |
| Failed calls | on each operator failure; pruned to 30 days / 500 rows on write | failed_calls (thread_id, interface, asker_id, asker_name, private, question, error, first_rejection, attempts) |
| Judge runs | one row per built-in pipeline run, any outcome (D33) | judge_runs (started_at, finished_at, interface, private, outcome, retried, tool_round, call_id, failed_call_id) + judge_run_charges (run_id, seq, stage, provider, model, tokens, micro_usd, cost_estimated) |
| Ratings | continuous; a web rating’s address and user agent cleared past FEEDBACK_IP_DAYS by every rating write and the refresh’s sweep step (D38) |
ratings (call_id, user_id, score, is_judge, ts, rater_kind discord/web, category, what_wrong, right_answer, web_name, ip, user_agent) |
| Feedback queue | one item per rating given a reason; reopened by a replaced reason | feedback_queue (id, call_id + user_id → ratings, status open/resolved/dismissed, note, fixed_by, opened_at, resolved_at) |
| Refresh runs | one row per refresh, scheduled or judgebot ingest refresh |
refresh_runs (started_at, finished_at, trigger, process, cr_before, cr_after, steps jsonb, ok) |
The loaders in the table (cards, rulings, the CR, symbols, nicknames, notes) and the
embedding step are judge_bot::ingest (crates/bot/src/ingest/), beside the other
Postgres adapters. judgebot ingest is the command line over them.
The database has three advisory lock keys, listed together in judge_bot::lease with a
compile-time check that they differ:
| Key | Scope | Held by |
|---|---|---|
REFRESH_LOCK |
session | a data-writing run (RefreshLease) |
GATEWAY_LOCK |
session | the process connected to the Discord gateway (GatewayLease) |
CALLS_REWRITE_LOCK |
transaction, shared or exclusive; session for a migration | a persist that writes a vector and each embed batch (shared); the CR load, the retirement pass, a space switch (exclusive); a migration, for its whole run |
The two session locks are a Lease<K>, typed by what they guard (K is Refresh or
Gateway, a sealed LeaseKey carrying the key, the application_name label and, in its
own impl, the wait), so one lease cannot stand in for the other.
Every loader that writes the database takes a RefreshLease, a session-level advisory
lock (REFRESH_LOCK) held on a connection of its own, so two runs never overlap, in one
process or several, and a step without it does not compile. Steps take it as
&mut RefreshLease, so two cannot run at once under one lease either. Waiting for it is
bounded (an hour), and a run checks before each step that its session still holds it. Postgres drops the lock with
the session, so a crashed run leaves nothing to clean up. It is separate from
CALLS_REWRITE_LOCK, the short transaction-scoped lock the CR load, the retirement pass
and vector writes take inside a run (and a migration for its whole run). A run takes the
lease first and the calls lock inside it, never the other way round.
Every ingest::refresh, whatever started it, checks before each step that it still
holds the lease and that the migration ledger matches its binary (migrate::skew), and
stops at ingest::RUN_TIMEOUT (3 h), abandoning the step in progress. A schema change
skips the remaining steps and the run is RunOutcome::Stopped, stored with ok null:
neither a success nor a failure, and not an attempt that backs the schedule off. A
single-step judgebot ingest command makes the same
check once (ingest::ensure_writable). runs::history counts an unfinished row older
than runs::ABANDONED_AFTER as a failed run, so a process that dies mid-run still
lengthens the failure streak.
A scheduled run also stops at its process’s shutdown (D40). ingest::refresh takes a
Caller: Caller::Schedule(&Shutdown) from the scheduler, Caller::Manual from
judgebot ingest refresh. Once the shutdown is requested the step in progress is cut
off at its next await point (Skip::Interrupted), the rest are skipped
(Skip::Shutdown), and the run is Stopped, whatever its earlier steps did. Every step
is safe to cut off: each writes in transactions or idempotent statements, and downloads
are renamed into place whole (the table in D40).
emoji_since counts an interrupted emoji step as one that may have uploaded.
The schedule is judge_bot::jobs, which a process with --jobs starts after migrating. It runs
on an OS thread of its own, with a current-thread runtime and a three-connection pool, so
a run’s file I/O, CR parse and queries never take a worker or a pooled connection from
the request path. The database is shared, though: while the CR load or the retirement
pass holds the exclusive side of CALLS_REWRITE_LOCK, a PgCallStore::persist that
writes a vector waits on the shared side, and both interfaces persist before they reply.
A cron run has the same effect.
Every ten minutes or so the scheduler reads the migration ledger (a schema ahead of or
behind the binary pauses it), then refresh_runs; a database with no rules waits for
init, which records itself as a manual run, so the first scheduled refresh comes one
interval after it. The pure jobs::due decides on the database’s clock: due when the last success
is older than JUDGE_REFRESH_HOURS and the last attempt older than jobs::backoff (1 h,
doubling per failure in a row, capped at the interval). When due it tries the lease
without waiting, re-reads the record under it and runs ingest::refresh with
Trigger::Schedule. That trigger also caps embed at embed::UNATTENDED_CEILING rows
(Skip::EmbedCeiling), so an unattended run cannot pay for a mass re-embed. The webhook
(judge_bot::alert, shared with the spend cap) is sent the first failure of a streak,
the recovery, a new ceiling skip and a panicking check. D24 records why the schedule is
in the process and not in cron.
jobs::start takes the process’s Shutdown. On the request the scheduler starts no
check, every wait outside a step is dropped (ingest::unless_stopped), the run in
progress stops as above, its row is closed and the lease released (those writes share
one deadline, ingest::STOP_WRITES after the request: ingest::until_stopped), a run
that finished before the request still posts its alerts, and the thread ends
(Ended::Shutdown),
shutting its runtime down so every connection it opened closes. Its connections carry
client_connection_check_interval (jobs::CLIENT_CHECK_INTERVAL, 2 s), so a statement
the cut step left running on the server ends within seconds of that, and with it the
transaction and any CALLS_REWRITE_LOCK it held. main waits for the thread after the
roles drain, at most until the drain’s limit (Scheduler::stopped), then writes the
spend ledger and exits. jobs::STOP_LIMIT (everything after the request) is a
const under DRAIN_LIMIT.
The record is also what users and the operator see of it. About carries the CR release
loaded and the age of the last successful run (see the HTTP adapter above), and
judge-cli stats lists the last five runs (runs::recent) with their outcome and failed
steps.
Database: Postgres 16 + pgvector + pg_trgm. Scale: ~30k cards, ~2k rule chunks, <10k calls.
Embeddings: Voyage AI by default (voyage-3.5 or voyage-4 family; voyage-3 is
superseded). It was chosen over local models for the zero-config setup, because local
models are not worth it on WSL2. Embedder is an interface, so this can change:
judge_embed also has an OpenAI-compatible /v1/embeddings adapter for local or hosted
models, chosen by [models.embed] in judge.toml.
Every embedder carries its Space (provider kind, model, width). The one-row
embedding_space table records the space the stored vectors belong to, and
ingest embed refuses to write into another. On a mismatch the adapters turn vector search
off (error log, never mixed). The space is re-checked on every use, and anything that
writes a vector holds it under a shared advisory lock. ingest reembed --yes switches the
database in one transaction after probing the new embedder (docs/PROVIDERS.md §4.3).
Rating aggregation: Bayesian-smoothed mean (prior 2.0, weight 3) over Discord ratings
only (calls_rated reads rater_kind = 'discord'). A rating with
is_judge = true (operator-assigned role) dominates crowd votes. It is used for
labeling and ordering among prior calls, and for exclusion (< 1.5, with ≥ 5
votes unless a judge rated it). It never ranks prior calls above CR chunks. A rating’s reason is read by the
operator’s queue alone.
5. Domain model
Section titled “5. Domain model”The model is language-neutral.
CardId = Scryfall oracle_idRuleId = "702.19" | "702.19b"Face { name, oracleText, manaCost, typeLine }Card { id, name, layout, faces: NonEmpty[Face] }Category closed set, ~25 values, sourced from categories.yamlSource CR | Commander | Tournament | OutOfScopeConfidence Low | Medium | HighResolution Resolved(card, matchedVia) | Ambiguous(query, candidates) | NotFound(query)Quote non-blank text; compared to its source with typographic punctuation foldedCitation Rule(id, quote) | ScryfallRuling(card, rulingKey, quote) | OracleText(card, quote) | PriorCall(id, quote)Context { cards, rules, rulings, glossary, prior, notes, history }Verdict<S> { answer, confidence, citations, category, source, crVersion, cards } S = Unvalidated | Validated; only Verdict<Validated> can be stored or shownRejection BadCitation(citation) | Malformed(what) | Empty(why) | Uncited(rule ids) | Tool(SecondRound(ids) | Unreadable(what)) | Oversized { chars }RejectedAttempt { answer, rejection } — quoted back to the retry as a blockquoteJudgeError AmbiguousCards(...) | CardsNotFound(...) | OutOfScope(source) | BadCitation | MalformedCitation | EmptyVerdict | UncitedRules | ToolMisuse | LlmRefused | Upstream(err)
Ports: Extractor, Resolver, Retriever, Synthesizer, Embedder, CallStorejudge : Question -> IO[Either[JudgeError, Verdict]]6. Build order
Section titled “6. Build order”The build was eval-first: retrieval was measured before any synthesis existed.
- A gold set of adversarially verified questions with expected rule ids
(
eval/gold.yaml, 24 questions today, one of them a human correction and two about unreleased cards). - Ingest.
- Extraction and resolution, tested on the gold set’s card mentions.
- Retrieval, behind a gate of ≥ 90% of gold rule ids present in the Context.
- Synthesis with citation validation, scored against the gold answers.
- The Discord adapter with rating buttons.
- The prior-call query, which needs rated data to exist.
judge-eval recall still runs that gate for free on every retrieval change.
7. Non-goals
Section titled “7. Non-goals”- Tournament policy (MTR/IPG). The bot declines those questions.
- Accounts on the web app. Its visitors are anonymous, so a rating from it would never count toward retrieval (D10).
- Retraining of any kind.
- Automatic detection of “nightmare” cards (the notes are curated by hand).
- Multi-server tenancy. The bot is meant to be run by each community for itself
(
docs/DECISIONS.mdD16), so one process has one spend cap, one judge role and one token.
MTG Judgebot is unofficial Fan Content permitted under the Fan Content Policy. Not approved or endorsed by Wizards of the Coast. Portions of the materials used are property of Wizards of the Coast. ©Wizards of the Coast LLC.
The Comprehensive Rules come from Wizards of the Coast. Card data, rulings and card symbols come from Scryfall, which is not affiliated with this project. Rule links go to the Yawgatog mirror. License and attribution.