Changelog
Notable changes an operator or user would notice. The format follows
Keep a Changelog and the versions follow
Semantic Versioning. A major version may change
configuration, the HTTP and MCP interfaces or the schema in a way that needs an
operator’s attention. Its entry says what to do. Migrations apply automatically
unless JUDGE_AUTO_MIGRATE=false.
Your data carries forward. Stored calls, ratings, sessions, the spend ledger and the
loaded cards and rules survive every upgrade, major versions included. Schema changes ship
as forward migrations, and a released migration is never edited. Embeddings survive too,
unless a release’s notes ask for a reembed. That pays the embedder again and never
touches calls. A release that needs more than docker compose pull && docker compose up -d
(a re-embed, a new required variable) says so under its own heading. Downgrading across a
migration is not supported. Restore the backup taken before the upgrade instead.
2.0.0 is a major release. Its migrations cannot run beside 1.x, its compose file runs one
judgebot service in place of bot and api, and some JSON shapes change. Read
Upgrading before deploying it.
-
Unreleased cards (D42). An answer about a previewed card that is not out yet opens with a note naming it when the loaded Comprehensive Rules may not cover it: on Discord, the web app (and the feed, as the note stood when the answer was given) and the agent toolbox’s
note. The note is left off during prerelease week once the set’s rules update is loaded and mentions every type and keyword on the card. The model is told the same in its material. An answer about a card that was not out yet is never shown as an example, even after the release. The cards step records each card’s first printing date, so the first refresh after the upgrade fills it in. The HTTP API’scardscarry astatus(released,previewed,unreleased). The gold set gains two questions about unreleased cards, on invented cards (fixture_cards) that never go stale. -
Public feed. The web app lists answered questions from every interface at
/feed, newest first, with where each was asked and the Discord asker’s username (anonymous otherwise), filterable by interface and searchable (q: the question, the answer or the asker’s username);/calls/<id>is one answer’s page and/users/<name>one user’s questions. Only questions asked private stay off it. New endpointsGET /api/feedandGET /api/calls/{id}, with their own read limit per address (120 a minute); an entry’sratingsare{"discord": {...}, "web": {...}, "judge"}.JUDGE_FEED=falseturns it off;JUDGE_FEED_DAYShides older calls without deleting them./helpsays public questions may be listed. Under--web, an unknown/api/path is now a JSON 404 rather than the page. -
Dashboard. The web app’s
/dashboardpage (a tab beside the feed) shows what the instance has been asked and how it went: questions per day by interface as a chart (with a table of the same figures), how they went (answered, a “which card?” question, card not found, out of scope, declined, failed), the rating mix (the web’s on request), and the most-asked topics, cards and askers, over 7, 30 or 90 days or a year, by interface, and per Discord user (?user=, linked from their page). It counts only questions not asked private, withinJUDGE_FEED_DAYS, and is served wherever the feed is. New endpointGET /api/stats?from&to&days&interface&user&model(30 reads a minute per address), and/api/aboutgainsstats.JUDGE_PUBLIC_COST=trueadds what the answers cost,JUDGE_PUBLIC_MODELS=truewhich models answered them and a model filter; both are off by default. -
Ratings with reasons. On Discord, Partially correct and Incorrect now ask why: a form with the kind of problem (wrong conclusion, missed part of the question, a citation doesn’t support it, wrong card, misread the question, shouldn’t have refused, outdated, other) and, optionally, what’s wrong and the right answer, up to 1000 characters each, in one form however interactions arrive. Correct is recorded at once, as before. Rating an answer you already rated asks first (Replace or Keep), and pressing Correct again says it is already recorded. Rating buttons on answers posted before the upgrade get the new flow. The reasons go to the operator, never the model.
-
Rating on the web. The chat, the feed and call pages have the same three buttons, and Partially correct and Incorrect open the same form under the answer (with an optional name). Rating again asks before replacing. A web rating is anonymous (the tab’s session) and never counts toward the examples. The feed’s counts are Discord’s; “Include N from the web” adds the web’s. New endpoint
POST /api/calls/{id}/feedback(beside/api/judge, 30 ratings per address per ten minutes), and/api/judge‘s answer carries the stored call’s id (call).JUDGE_PUBLIC_FEEDBACK=trueshows raters’ reasons on the feed and call pages, without who gave them (raters are told so); a dismissed queue item’s reason is taken off. -
A new web app design. The page reads as a conversation with the judge: questions and answers as messages, the question box docked at the bottom, citations as one section listing each one’s label that opens to every quote, and replies that are not answers as titled notices. The header links to the documentation and has a theme switch (Auto, Dark, Light). The feed has a Compact view (one line per question, the default) and an Expanded one. Figtree is bundled; nothing is loaded from another site.
-
Citation kinds in the HTTP API. Every citation in a reply or a feed entry carries
kind(rule,ruling,oracle_text,prior_call), so a client can tell them apart without reading the label, which may be translated. -
Translation groundwork. Every message the Discord bot, the web app, the admin pages, the HTTP API, the alert webhook,
judge-config, the command lines’ usage and the docs site’s header and footer show now comes from a per-language catalog, and each request picks its language: the Discord user’s; the browser’s on the web (sent as the new optionallocalefield ofPOST /api/judge, elseAccept-Language) and injudge-config; the terminal’s (LC_ALL,LC_MESSAGES,LANG) forjudge-cli,judgebotandjudge-config.JUDGE_LOCALEsets the fallback, and the language of the alert webhook’s messages. Only English ships, so nothing reads differently yet.GET /api/aboutgainslocaleandlocales. A “did you mean?” or Retry button in another language will carry it in its id; English ones keep their shape. The answers, the rules material, JSON output, the agent tools’ text and diagnostics (logs, the command lines’ errors, health checks) stay English. -
Admin interface. A new role,
--admin, serves the operator’s pages on a port of its own (ADMIN_ADDR, default127.0.0.1:8789) behind a newADMIN_TOKEN: every stored call, private ones included, filterable by interface, asker, model, visibility and text; one call with its asker, its ratings and their raters, and its run’s models, tokens and cost; the failed calls; the feedback queue; and the Dashboard below.judge-cli admin-urlprints the page’s address with the token. In compose theproxyservice publishes it on the host’s127.0.0.1:8789only (ADMIN_PORTmoves the host port); from another machine, forward the port over SSH. The public listener answers/adminpaths 404, and the admin listener servesGET /admin/api/about. Off unlessJUDGE_ROLESnames it. -
The admin Dashboard. The public dashboard’s report over every call, private ones included and counted apart, by range, interface, asker and model, with the failed calls, spend per day and month, the budget period, the cap and what is left, the runs by model, interface and stage, cost per answer, latency, the dearest runs and the runs the cap refused.
judge-cli statsand the MCP tooladmin_usagereturn the same report and take the same range and filters (--from,--to,--interface,--asker,--model). The budget is the instance’s: every process that keeps the spend ledger records its period and cap beside it, sojudge-cli statsno longer reports its own. A report’s queries are cut off past 5 seconds on the public page and a minute for the operator. -
Admin reads for agents.
judge-cli callsandcall <id>, and the MCP toolsadmin_calls,admin_call,admin_failuresandadmin_usage, on stdiojudge-mcp(and so over SSH) and on/mcpfor a newMCP_ADMIN_TOKENonly, neverMCP_TOKEN. Every admin call over/mcpis logged with its tool’s name. -
The feedback queue. Every rating given a reason is an item for the operator, beside the call’s question, answer, citations and cards: the admin page’s Feedback tab,
judge-cli queue,queue-show,queue-resolveandqueue-dismiss, and the admin MCP toolsqueue_list,queue_show,queue_resolveandqueue_dismiss, filterable by status, kind of problem, interface and kind of rater. An item is resolved (with a note and what fixed it) or dismissed (with a note); a rater who replaces their reason reopens it.judge-cli statscounts the queue by kind of problem (feedback) and web ratings apart from Discord’s. Thejudgeskill walks an agent through it. -
FEEDBACK_IP_DAYS. How long a rating given on the web keeps the address and user agent it came from (default 90 days, 1 to 3650). Every rating written clears older ones, and so does a new refresh step,sweep, betweenpruneandretire. -
Every call records where it came from. A stored call names its interface (Discord, web, MCP,
judge-clior an agent session) and, on Discord, the asker’s user id and username at asking time.judge-cli statscounts questions by each interface and how many were private. -
Private calls are stored. A
/judge-privateanswer is now kept, marked private, and has rating buttons. Only the operator sees it: it is never read as a later question’s history, never used as an example, never embedded and never on the feed. The MCPjudgetool andbegin_sessiontakeprivate, andjudge-cli judgeandbegintake--private./forgetnow also removes the user’s id and username from their questions, and a private question’s text. -
Failed calls are kept for troubleshooting. A call that fails for a reason the operator has to diagnose is stored in
failed_callswith the question, the error and the answer text of each synthesis attempt (each cut to a fixed length; the newest 500 rows, 30 days), with its interface and asker. A question asked with/judge-privateis stored too, flagged.judge-cli failuresprints{"failures": [...]}, each row with itsid./forgetanonymizes a user’s rows (id, question, error and text removed) as well as deleting their ratings. -
Every judge run is recorded with its cost. A
/judge, a web question, an MCPjudgeand ajudge-cli judgeeach leave ajudge_runsrow, whatever the outcome (answered, ambiguous, not found, out of scope, refused, at the spend cap, failed), private ones included, with onejudge_run_chargesrow per model call and embedding: the stage, the model the response named, its tokens and its cost. Each run’s cost is its own, not a difference of the process’s running total, so concurrent questions no longer bleed into each other’susd=log figure. A response that reports no usage is recorded at the worst case the spend cap kept for it, flagged as an estimate. No text or user id is stored there. -
Call retention.
JUDGE_CALL_RETENTION_DAYS(unset or0: keep every call, as before) deletes answered calls older than that many days, with their ratings, at each refresh (a newprunestep) or withjudgebot ingest prune. Each run’s usage record (judge_runs) and the daily spend ledger stay.judge-cli statscounts questions and ratings from the stored calls, so its counts for days past the retention drop to what is left. A deleted call stops being an example for later answers; a newer call that cited it keeps its citation. A rating button on a deleted answer says it is no longer stored. -
Spend warnings before the cap.
JUDGE_SPEND_WARN_ATlists percentages ofJUDGE_MAX_USD(such as50,80). Each is logged and posted toJUDGE_ALERT_WEBHOOKonce per period when spending reaches it. In adayormonthperiod one process sends each warning, however many share the budget. Unset by default. -
Discord interactions over HTTP. With
DISCORD_INTERACTIONS=http, Discord POSTs slash commands, button clicks and forms to/discord/interactions(the application’s Interactions Endpoint URL, a public HTTPS address such as the tunnel’s) instead of sending them over a gateway connection. The bot then opens no gateway connection, and Discord keeps being answered while a deploy switches containers.gatewaystays the default. The bot reports a setting that disagrees with the Developer Portal in its log and toJUDGE_ALERT_WEBHOOK. Steps:docs/DEPLOYMENT.md, “Discord over HTTP”. -
Deploys without downtime.
scripts/deploy.shstarts the newjudgebotcontainer beside the running one, waits until Docker reports it healthy, then stops the old one. The web app and the API stay up throughout. Discord is quiet for the new container’s login, a few seconds: the old one records that it closed its gateway, so the new one connects without the 15-second grace a takeover after a crash waits (tablegateway_handover). A new container that is not healthy within five minutes is removed and the old one keeps serving.--buildbuilds from the checkout instead of pulling. -
scripts/deploy.shdeploys a major release too. It asks the new image what its migrations are (judgebot ingest upgrade, a new command that printsupgrade: in-place,majorormajor-from-1.x). For a major it takes a backup when one is configured, starts the new container beside the old one, stops the old one as soon as the new one listens, and waits for the migration. -
Major upgrades answer with a maintenance reply. A release whose migrations cannot run beside the previous release (a new major) starts its roles at once and answers every question with a maintenance message: on Discord the question with a Retry button, on the web a Try again button, over the API a
maintenancereply (503,Retry-After), over/mcpthe message as the error. It migrates once every process of the previous release has stopped (a lock everyjudgebotwith a role holds while it runs), then answers as usual./api/healthfails meanwhile. Releases within a major still migrate first and run beside the previous release. -
judgebotrefuses to start against a newer major’s schema. A rollback within a major still starts and runs, since those migrations only add. Across a major, or between two builds of a new major’s pre-release, it exits naming both builds, because the schema can no longer hold what it reads and writes: run the newer build, or restore the backup taken before the upgrade. A 1.x image has no such check. A pre-release tag needs that exact version in the tree. -
judge-cliandjudge-mcpcheck the instance’s release. Each migration records the release and commit that ran it (tableinstance_release). Against a newer release of the same major the tools print a note and keep working. Against another major, another build of a new major’s pre-release (such as2.0.0-dev), or a database missing one of their migrations, every operation fails and says what to update, naming the instance’s commit. Between releasesmaincarries a-devversion. Migrations within a major only add; dropping, renaming and rewriting wait for the next major (CONTRIBUTING). -
judgebotshuts down gracefully. On SIGTERM (docker stop) or Ctrl-C it stops taking new work, releases the Discord gateway at once, stops a scheduled refresh in progress, finishes the answers in flight within two minutes, writes the spend ledger and exits 0. A second signal exits at once. Before,docker stopwaited ten seconds and killed it, and a question in progress got no answer. The compose service’sstop_grace_periodis 130 seconds. -
The long-running process refreshes the data itself (the
--jobsrole, which the composejudgebotservice runs). EveryJUDGE_REFRESH_HOURS(default 24, 1 to 720,0= off) one of them runs the steps ofjudgebot ingest refresh: cards and rulings, a new CR release, the built-in curated lists, retention, retirement, embeddings, emoji. No host cron is needed. The schedule is kept in the database, so the processes, a restart and a cron run agree on it, and the refresh lease means a run happens once however many there are. A failed run is retried after an hour, then less often while it keeps failing. The run has its own thread, runtime and database connections, so its downloads and parsing take nothing from answering. While a CR load or the retirement pass holds the calls lock, saving an answer still waits for it, as under a cron run. A process whose image does not match the schema pauses its schedule with one warning instead of writing, and a database with no rules loaded waits forinit. A badJUDGE_REFRESH_HOURSstops every binary that loads the configuration, asJUDGE_BUDGET_PERIODdoes. -
JUDGE_ALERT_WEBHOOKhears the scheduled refresh: the first failure of a streak (saying when it timed out), the recovery after one, a run whose process died, an embedding step the spend guard skipped, and a crashed check. Each is posted once, not on every retry. -
A scheduled refresh never pays for a mass re-embed. With more than 800 rows waiting for a vector it skips
embed, logs the count and alerts.judgebot ingest embed(orscripts/refresh-data.sh embed) does it when that spend is expected. A manual or cron’djudgebot ingest refreshhas no ceiling. -
Each refresh is recorded in a new
refresh_runstable: start and finish, what started it and where, the CR version before and after, each step’s outcome and whether all succeeded. A run that loads a new CR logsCR <old> → <new>.judge-cli statslists the last five: when each started, what started it, its outcome (ok,failed,stopped,runningorabandoned), the CR version before and after, and the steps that failed.judgebot ingest initis recorded as one of the steps it shares with a refresh (it now runs the retirement pass too), so the schedule counts a first load as fresh data, and the steps it did not reach after a failure are recorded as skipped. -
Every interface says how fresh the data is.
GET /api/about, the MCPabouttool andjudge-cli aboutcarryfreshness: the Comprehensive Rules release loaded, the seconds since the last successful refresh, and whether the latest one failed. It isnullwhen the database does not answer, and the rest is served regardless. The web footer shows it as one line and/helpas a Data. list./api/healthignores it. -
The refresh keeps the built-in alias and note lists current. Each load of a curated list records its source in a new
curated_liststable:builtinforinitand forjudgebot ingest aliases/noteswith no file,filewhen you name one, with a digest of the YAML. A new refresh step,lists(afterrules), reloads a built-in list when the binary’s copy differs, so a release that improvesdata/aliases.yamlordata/notes.yamlno longer needs a manual reload. A list loaded from your own file is never touched, by the refresh or byinit, which now keeps it with a warning: loading a file opts that list out of built-in updates until you load it again with no file. A built-in list loaded before the cards were, or naming a card added since, fills in by itself. The step’s outcome is inrefresh_runs.stepsas each list’scurrent,reloaded,adopted,kept(with a reason) orfailed. -
The bot picks up new card-symbol emoji without a restart. When a refresh run, by any process, uploads emoji, the bot lists them again within ten minutes. A bot with none checks every ten minutes, so the first
judgebot ingest emojiafter starting it is seen too, and every bot lists them hourly regardless. A listing that failed at startup is retried the same way, and connecting no longer waits for it. -
One process answers Discord, however many run. A
judgebot --discordtakes a gateway lease in the database before it connects. A second one on the same database (a replica, a new container started before the old one stopped) logsstanding by: another instance holds the Discord gateway, naming the holder, serves its HTTP roles meanwhile, and connects 15 seconds after the holder exits, or about 25 seconds more when the holder vanished without closing its connection (power loss, a partition). The holder checks every 5 seconds; when its lease is lost (a database restart) it disconnects and stands by again instead of restarting the process, so the two never answer the same question. Losses in quick succession pause before reconnecting, up to ten minutes, to keep within Discord’s daily login limit.DATABASE_URLmust be a direct connection, not a pooler in transaction mode. -
The weekly backup can run with no host cron. The new
backupcompose service, off by default behind thebackupprofile (COMPOSE_PROFILES=tunnel,backup), dumps the database, uploads it to an S3-compatible bucket and prunes old copies, the same backupscripts/backup-db.shtakes, with the same.env.deploysettings, object names and format. It takes one whenever the newest backup in the bucket isBACKUP_EVERY_DAYSold (new, default 7), checking hourly, so a restart does not reset the schedule and a cron’d script run counts. A failed attempt is retried after an hour, then less often, andJUDGE_ALERT_WEBHOOK(in.env.deploy) hears the first failure of a streak, a failure at a different step, and the recovery. It reaches Postgres over the compose network with the image’s ownpg_dump16 (new in the image, about 6 MB unpacked), so it needs no Docker socket, and it reads.env.deployand nothing of.envbutJUDGE_LOCALE. The upload carries the dump’s SHA-256, which the store checks. Its pruning deletes only backup-named objects directly under the prefix, and never the two newest, however old, so a lapse longer thanBACKUP_KEEP_DAYScannot leave a single restore point (the script’s pruning has no such floor). -
judgebot backup run | list | fetch <name> [file] | serve | check. The service’s commands by hand:docker compose run --rm backup runtakes one now,listprints the objects, anddocker compose run --rm --no-deps -T backup fetch <name> > <name>downloads one for the restore drill. Each works on the script’s backups too.checkreads the settings and findspg_dump, sending nothing. Settings that are not usable are posted toJUDGE_ALERT_WEBHOOKby name, andservethen stays up refusing rather than restarting with the same environment.scripts/deploy.shrunscheckwith the new image before it changes anything when thebackupprofile is on, and brings thebackupservice onto the new image once the deploy has succeeded. -
judge-eval grade. The eval’s answers are graded by a model (the synthesis model): whether each answer’s ruling follows from its quotes, and whether it agrees with the gold reference (D28).judge-eval grade <run.json>grades a stored run (resumable;--forceregrades),answer --gradegrades a new one. A graded run costs about $0.40. -
An Academy Manufactor note and gold question. The built-in notes cover Manufactor replacing the whole token creation (an effect making a modified Food makes a plain Food, Clue and Treasure), and the gold set has a 22nd question on it. The refresh’s
listsstep loads the note by itself. -
A nickname whose card name several cards share is left unresolved (and logged), as notes and the card resolver already did, instead of pointing at whichever card the database returned first. A card’s own name still beats the same name on another card’s face. None of the built-in aliases is affected.
-
Public HTTPS without Cloudflare. The compose file has a
publicprofile:COMPOSE_PROFILES=publicwithJUDGE_DOMAIN=judge.example.comstartsedge, a Caddy on ports 80 and 443 that serveshttps://<JUDGE_DOMAIN>with a Let’s Encrypt certificate it gets and renews itself, in front ofjudgebot:8787with the same handover as the proxy, so deploys keep serving. Without the profile nothing changes: the stack answers on127.0.0.1:8787only. A Cloudflare Tunnel is now the alternative rather than the runbook’s setup, anddocs/DEPLOYMENT.md§3 lists the routes (local only, public HTTPS, your own reverse proxy, the tunnel).API_CLIENT_IPgainsproxy: the rate limit countsX-Judgebot-Client-IP, whichedgesets to the visitor’s address and the proxy’s other sites remove from callers, read only as one copy from a loopback or private peer. A reverse proxy of your own on the host can set it too.judge-configchecksJUDGE_DOMAINunderpublic(the rule theedgecontainer applies) and suggestsAPI_CLIENT_IP=proxywith it andcloudflarewith the tunnel, andscripts/deploy.shchecks and startsedgewhen the profile is on. Every rate limit now counts an IPv6 visitor by its /64 rather than the full address. Caddy finishes the requests in flight for up to 70 s whenproxyoredgestops. -
scripts/deploy.sh --dbupdates the database to the imagedocker-compose.ymlpins: a Postgres minor or a pgvector release, with a few seconds’ outage. It refuses another Postgres major or an older pgvector before changing anything, takes a backup when one is configured, and updates the extensions afterwards. A code deploy refuses while thedbcontainer runs another image than the compose file pins. A new Postgres major’s dump-and-restore runbook is indocs/DEPLOYMENT.md§8, Database image. -
scripts/deploy.sh --proxyreplaces theproxyandedge(Caddy) containers when their service changed (a new Caddy release, an edited service), at a moment you choose: only one container can hold a host port, so new connections wait while one is replaced. A plain deploy refuses until it has run. A Caddyfile edit still needs neither. -
The tunnel’s connector updates with no downtime. A deploy starts the new
cloudflaredbeside the old one and stops the old one once the new one has registered with Cloudflare. It no longer has a fixed container name (judgebot-tunnel);docker compose psanddocker compose logs cloudflaredfind it. -
Full redeploy in one command.
scripts/deploy.sh --db --proxy --pull(or--build) runs--db,--proxyand the deploy in that order, each waiting for the one before it, and stops at the first failure.--fullis the same command (--full --buildbuilds).--pullis a new name for the plain deploy. -
Extensions follow the database image. Every migrating start (and
judgebot ingest migrate) updatesvectorandpg_trgmto the version the database image carries. A new pgvector image used to leave the extension at the version that created it.
Changed
Section titled “Changed”-
No Discord library. The bot speaks Discord’s API itself (its own wire types, REST client and gateway connection, D43) instead of through serenity 0.12, whose rustls 0.22 carried security advisories that will not be fixed on its line. Nothing to configure: commands, replies and buttons are the same, a
Botprefix onDISCORD_TOKENis still accepted, and the containers’RUST_LOGno longer namesserenity. A rating’s reason is one form over the gateway too. -
/judge-privatereplaces/judge private: True. Asking privately is its own command, next to/judgein Discord’s command list, instead of a true/false option people had to find and set./cardand/rulelose theirprivateoption and always post in the channel (in a DM with the bot they are already private). A/judgesent from an older registration withprivate: Trueis still answered privately. -
Measured efforts per stage. A
judge.tomlstage that names noeffortnow runs at the level its model was measured at on the gold set, for extraction as well as synthesis. The default setup is Claude Opus 5.5 at medium on both stages: extraction at low now and then wrote part of a card’s name alone and asked “did you mean?”, which medium never did, for about 3% more a question. Sonnet 5.5’s synthesis default moves from high to low, which agreed with the reference as often for a quarter less, and Haiku 5.5 extracts at high. An extraction at high effort or above gets a 4000-token output ceiling by default. Setefforton a stage to keep the previous behaviour. -
Backups go to any S3-compatible store. The backup’s settings were Cloudflare R2’s and are now
BACKUP_S3_ENDPOINT,BACKUP_S3_BUCKET,BACKUP_S3_ACCESS_KEY_IDandBACKUP_S3_SECRET_ACCESS_KEY, with two new optional ones:BACKUP_S3_REGION(the region requests are signed for, defaultauto, R2’s) andBACKUP_S3_ADDRESSING(path, the default, orvirtual-hosted, which AWS recommends).docs/DEPLOYMENT.md§6 gives the values for Cloudflare R2, AWS S3, Backblaze B2 and MinIO. The object names are unchanged, so a bucket’s backups keep listing, fetching and setting the schedule. TheR2_*names are refused, naming their replacements (Upgrading). -
One binary,
judgebot, whose roles are launch options.bot,apiandingestare now one program.judgebot --discord --api --web --mcp --admin --jobsruns any non-empty set of the bot, the JSON route, the web app, the MCP transport, the admin interface and the scheduled refresh in one process; with no flags it readsJUDGE_ROLES(the same flags). Each role’s requirements are checked before anything connects or binds, and every unmet one is named at once:--discordneedsDISCORD_TOKENandJUDGE_OPERATOR_DISCORD, the network rolesJUDGE_OPERATOR_EMAIL,--weba built page,--mcpanMCP_TOKEN,--adminanADMIN_TOKEN, and any serving role a chat model that builds. The HTTP listener is bound before the Discord gateway is contacted, and the first serving role to stop ends the process with a non-zero status. The data command line isjudgebot ingest <command>, with the same commands and arguments as before.judgebot --helpandjudgebot ingest --helplist them. -
judge-bot,judge-apiandjudge-ingestare gone. The image carriesjudgebotalone, and it is the image’s entrypoint: with no arguments it runs the roles inJUDGE_ROLES. -
One compose service,
judgebot, in place ofbotandapi. It runsJUDGE_ROLESwhen that is set in.env, else--discord --api --web --jobs: what the two services ran together.JUDGE_ROLES='--api --web --jobs'runs it without Discord. Its healthcheck probes/api/healthwhen the process listens (an HTTP role, or Discord over HTTP) and passes otherwise.refreshrunsjudgebot ingest, with the same arguments as before, andjudgebotmounts thejudgebot-ingest-cachevolumerefreshalready used. -
A new
proxyservice publishes127.0.0.1:8787(Caddy,proxy/Caddyfile), forwarding tojudgeboton a new connection per request, and127.0.0.1:8789for the admin interface. Thejudgebotcontainer publishes no port and has no fixedcontainer_name, so that a deploy can run two of them. The address and a host reverse proxy in front of127.0.0.1:8787are unchanged. The network nameapinow belongs to the proxy, so a tunnel whose service ishttp://api:8787goes through it. Edits toproxy/Caddyfileload without a restart. -
API_INTERFACESis retired.JUDGE_ROLESreplaces it. A.envthat still sets it is refused at startup, and the error names theJUDGE_ROLESline that replaces it, rather than the process starting without the interfaces it chose.judge-configreports the same error, and checks only the roles the service would run, so a deployment without Discord needs noDISCORD_TOKENthere. -
One spend meter per process. The roles of one
judgebotprocess bill to one meter and one ledger. WithJUDGE_BUDGET_PERIOD=process, a process running both the bot and the HTTP interfaces has oneJUDGE_MAX_USDbetween them, where the separatebotandapiprocesses had one each.dayandmonthbudgets were already shared. Each role keeps its ownJUDGE_CONCURRENCYslots and the API its rate limits. -
The scheduled refresh runs only under
--jobs. A process whose only role is--jobsexits non-zero if the scheduler’s thread ends, and refuses to start withJUDGE_REFRESH_HOURS=0. Beside a serving role, a scheduler thread that ends is logged as an error and the process keeps answering. -
judge-cli statshas a new shape. It is the admin Dashboard’s report (andadmin_usage’s, and/admin/api/usage’s).by_daylists every day of the range, oldest first, with the calls per interface (discord,web,mcp,cli,agent_session, andagentfor calls stored before this release) andprivate; the ledger’s spend and model calls per day moved tospend_by_day(withspend_by_monthbeside it).ratingsis{"discord": {"correct", "partial", "incorrect"}, "web": {...}}and counts the ratings of the calls in the range, where it was an array of the ratings given in it.days,questions,usd,retired_callsandlowest_ratedkeep their meaning; everything else (run_stats,budget,failed_calls,feedback,refresh_runs, the dashboard’s counts) is new. -
Embeddings count toward the spend cap. Every embedding request (a question’s query, a stored call’s vector,
judgebot ingest embed/reembed/init/refreshand the scheduled refresh’s embed step) now reserves its worst case againstJUDGE_MAX_USDbefore it is sent and settles to the provider’s reported usage, on the same meter as the chat calls, so one cap covers both. Voyage models are priced from Voyage’s list (voyage-3.5$0.06,voyage-3.5-lite$0.02,voyage-3-large$0.18 per million tokens; an unknown Voyage model at the dearest, $0.18), from the first token: the free allowance is not subtracted.[models.embed.pricing] input = …overrides the list. At the cap a question is still answered, without vector search, and its call is stored without a vector for the nextembedto fill; aningestembedding run stops naming the cap and keeps what it embedded.judgebot ingestnow writes the spend ledger, so a re-embed counts towardJUDGE_BUDGET_PERIODand shows injudge-cli stats, combined with the chat spend (the ledger’s call count stays chat calls only), and so does a--jobs-only process.reembed’s dry run prints the cost at the configured price and what is left under the cap, in place of the generic estimate, andreembed --yesrefuses to clear the vectors when the re-embed’s worst case does not fit. A refused embedding does not trigger the “refusing questions” webhook. -
A chat response with no usage counts its worst case against the spend cap. An OpenAI-compatible server that leaves
usageout of a paid model’s response, fills it with zeros, or sends a 2xx body that cannot be read was counted at $0. It now keeps the reservation taken before the request, as an embedding with no usage already did. A free model is unaffected. -
A web or MCP question whose caller goes away stops at its next model call. Closing the tab (or cancelling the MCP request) mid-answer used to drop the run outright: the request in flight was still billed, but nothing recorded it. Now that request finishes and is charged, no further model call is sent, and the run is recorded as
abandoned. An answer already written is still stored. -
Discord answers a command it cannot run. A slash command this version does not have (Discord can still list a previous release’s for a moment after a deploy) gets a private “I don’t recognise that command” instead of no response, and
/judgein a direct message a private “works in a server only”. Replies ping nobody unless they mention someone on purpose. -
“Did you mean…?” buttons survive a restart and a gateway hand-off. A click is answered from the message the buttons are on, by whichever process holds the gateway, for ten minutes after the prompt was shown. Nothing about the question is kept in memory. The message restates the question exactly as asked, then lists the choices as before (a card name past 60 characters is shortened in the list, not on its button). A new
pick_claimstable makes each prompt answer one click (a second gets “A card has already been picked”). It stores the message id, when the prompt was shown and a short digest of the message, never the question, so a/judge-privatequestion still leaves no text in the database. Claims older than a day are deleted as new ones are taken. -
A
/judgequestion is at most 1300 characters. Discord enforces the limit in its command box, so the whole question fits in a “did you mean…?” message. A question that still does not fit (emoji and some other characters can count twice toward the message limit) gets the choices as a list, without buttons, and asks you to ask again with the card written as[[Full Card Name]]. -
POST /api/judgeanswers an unreadable request in JSON. A body that is not JSON, has the wrong shape, is too large or is sent without a JSONContent-Typekeeps its 4xx status (400, 422, 413, 415) but now gets{"kind":"error","message":…}instead of a plain-text body, so a client can parse every reply the same way. -
Data refreshes take a lock in the database. Every
judgebot ingestcommand that writes data (refresh,initafter its migration,cards,rules,aliases,notes,embed,reembed,retire,prune) first takes the refresh lease, a Postgres advisory lock. It waits up to an hour for a run that holds it, logging who holds it, then fails. The schedule, a cron run, a manual step and a workstation’sjudgebot ingesttherefore take turns instead of overlapping. Postgres drops the lock with the session, so a killed run leaves nothing behind.scripts/refresh-data.shno longer takes its.refresh.lockdirectory. -
A refresh stops writing when it should. Before each step
judgebot ingest refreshchecks that the migration ledger matches its binary, and every single-step command (cards,rules,embed, …) checks once before it starts. A run betweendocker compose pullandup -d(schema behind) or on an old image (schema ahead) writes nothing, namesjudgebot ingest migrateor the newer image, and exits non-zero. It is recorded as stopped, neither a success nor a failure. A run stops after three hours, abandoning the step in progress, so a hung download cannot hold the lease. -
A deploy stops the scheduled refresh in progress. A container asked to stop (every
scripts/deploy.sh,docker stop) cuts the step in progress off, records the run as stopped, releases the refresh lock and exits within the drain. The new container’s first check runs the refresh again. A stopped run, by a shutdown or a schema change, does not delay the next one. Inrefresh_runs.stepsthe step cut off is"reason": "interrupted"and the ones after it"reason": "shutdown". The schedule’s database connections setclient_connection_check_interval(2 s, Postgres 14 on). -
Downloads time out. A step’s connection to Scryfall or Wizards gives up after 30 seconds without connecting or two minutes without data, so a stalled connection fails that step instead of hanging the run.
-
A truncated CR download is never loaded. The CR file is cached through a
.partfile and a rename, and a text that ends before its Credits section is refused (and its cached copy removed) instead of being loaded, which would have deleted every rule it did not reach. -
A refresh with no embedder configured logs
refresh step skippedforembed. It used to logrefresh step ok.emojiwith noDISCORD_TOKENwas already logged as skipped. -
Building from source needs Rust 1.99 (
rust-toolchain.toml, theDockerfile). -
The
dbimage is pinned topgvector/pgvector:0.8.7-pg16-bookwormand its digest, instead of the floatingpg16, so the database changes only when the compose file does. Runscripts/deploy.sh --dbonce after upgrading to move onto it. -
Caddy 2.11.9 for
proxyandedge, and cloudflared 2026.11.0 for the tunnel. -
scripts/deploy.shis the one way to apply a change: a release, an edited.env,.env.deployorjudge.toml(no moredocker compose restart judgebot), a profile added toCOMPOSE_PROFILES, and the first start, where it starts the database too. It brings thebackupservice and the tunnel’s connector up to date with the rest. The documentation no longer teaches a baredocker compose up -dorpull && up -d --remove-orphans, which recreate the database and the proxies without the checks. Downtime is always its own flag (--db,--proxy), and a plain deploy refuses while the database’s or a proxy’s service differs from its container. -
refreshandbackupno longer start the database. Adocker compose runof either used to start or recreatedb; nowdbmust be running (scripts/deploy.shstarts it), andscripts/refresh-data.shsays so when it is not. -
A running
judgebotwaits out a database outage. It used to exit when the database was gone for more than about 30 s (its schema hold could not be taken again), and restart until the database answered. A database that comes back without this release’s migrations (another data volume, an older backup) now ends it.
- Newly spoiled cards load. Scryfall lists a fresh preview with an empty
gameslist until it knows where the card will be played, and the card step read that as digital-only and dropped it, so the newest spoilers could not be asked about however often the data refreshed. An empty list now counts as paper unless Scryfall marks the printing digital. - A card’s epithet names it. The words after the comma in a legendary card’s name now resolve on their own when they are two words or more: “Nimble Pilferer” is Ragavan. When another card’s name holds the same words, the bot asks: “Grand Arbiter” offers Dovin and Grand Arbiter Augustin IV, “mind sculptor” Jace and Way of the Mind Sculptor. Either way a question that names the card in full as well no longer stops on a “did you mean?”. Before, such a span went to the fuzzy search, which never counts as the same card. A one-word epithet (“ravager”) still asks, since players often mean another card by it.
scripts/refresh-data.shworks on Compose v2.20 (Synology’s Container Manager), which has nodocker compose run --pull. It checks that the pulled image is present instead, and still never builds on the host.- A retry after a rejected answer now tells the model to send the whole answer. A retry
that came back with an empty
answerended the call as “empty verdict”. - The retry after an answer named a rule without citing it now gets that rule. A rule
reached through a pointer (“see rule 111.10” in a glossary entry) was often not in the
material, so the retry had nothing to quote and could not call
lookup_rules. - A judge’s Incorrect now drops an answer from the examples the model is shown, at once. It used to need five ratings in all, like the crowd’s, so a wrong answer only a judge had rated stayed in.
Upgrading
Section titled “Upgrading”This release cannot run beside 1.x, and 1.x cannot run on its schema. Plan a few minutes of maintenance, take a backup, and update the checkout and the image together.
Before the deploy
- Rename the backup’s settings in
.env.deploy. They no longer assume Cloudflare R2. Keep each value:R2_ENDPOINTbecomesBACKUP_S3_ENDPOINT,R2_BUCKETBACKUP_S3_BUCKET,R2_ACCESS_KEY_IDBACKUP_S3_ACCESS_KEY_IDandR2_SECRET_ACCESS_KEYBACKUP_S3_SECRET_ACCESS_KEY;sed -i 's/^R2_/BACKUP_S3_/' .env.deploydoes all four. R2 needs nothing else:BACKUP_S3_REGIONdefaults to itsauto. Rename the lines rather than filling in the new example’s blankBACKUP_S3_*=lines as well: the last line of a name wins, a blank one too (the script andscripts/deploy.shrefuse a name set twice). A setR2_*stops thebackupservice,scripts/backup-db.shandscripts/deploy.sh, each naming the lines, so rename them right aftergit pull, before a cron’d script next runs and before deploying. The deploy then recreates thebackupservice with them. - Take a backup. It is the only way back to 1.x (see Rolling back below).
scripts/deploy.shtakes one itself whenBACKUP_S3_BUCKETis set in.env.deploy; otherwisescripts/backup-db.sh, ordocker compose run --rm backup runonce the new compose file and image are in place. - Update the compose file with the image. This image cannot run the previous
release’s
docker-compose.yml: itsbotservice startsjudgebotwith no roles, and itsapiandrefreshservices name binaries the image no longer has. Rungit pullbefore or in the same step asdocker compose pull, never the image alone, and pull the image straight after: a cron’dscripts/refresh-data.shin between would run the new compose file’sjudgebot ingeston the old image, which has nojudgebot. AJUDGE_IMAGE_TAGpinned to a 1.x release (1.2,1.2.0, an earliersha-tag) moves to this release in the same step. - Set
JUDGE_ROLESif the defaults are not what you ran. A.envwith neitherJUDGE_ROLESnorAPI_INTERFACESruns--discord --api --web --jobs, what the two services did.API_INTERFACESis refused at startup:--discord --jobsplus its value is the line to write, soAPI_INTERFACES='--api --mcp'becomesJUDGE_ROLES='--discord --api --mcp --jobs'. A deployment that ran onlyapi, with noDISCORD_TOKEN, setsJUDGE_ROLES='--api --web --jobs'(add--mcpif it served/mcp), orjudgebotrestarts forever withDISCORD_TOKEN is not set. The startup errors name these lines too.judgebot ingestandjudgebot backupread no roles, soscripts/refresh-data.shkeeps running meanwhile. - Decide what the feed shows. The feed is on by default and lists every call not
asked private, including those asked before this release, with the asker’s username
where one was stored (earlier calls have none); the public dashboard lists the
Discord users who asked most. To keep them unlisted, set
JUDGE_FEED=false(which takes the dashboard off too) or a shortJUDGE_FEED_DAYSin.envbefore upgrading. - Check port 8789 on the host. The new
proxyservice publishes127.0.0.1:8787and127.0.0.1:8789(the admin interface, whether or not--adminruns). If something on the host already listens on 8789, setADMIN_PORTin.envto a free port first, or the proxy cannot start and takes 8787 down with it. - The first deploy recreates the
proxyservice once. Its host port now maps to a new site, and it drains for up to 70 s when stopped. While it is recreated,127.0.0.1:8787and the tunnel are down from the moment it stops listening until the answers it was serving finish (up to 70 s) and the new one starts; Discord over the gateway is not affected. - A Cloudflare Tunnel setup needs no change.
proxy:8787(and theapialias) behaves as before,API_CLIENT_IP=cloudflarestill readsCF-Connecting-IP, and127.0.0.1:8787still answers. Thepublicprofile is opt-in and claims ports 80 and 443 only when it is on. A reverse proxy of your own on the host that setsCF-Connecting-IPwithAPI_CLIENT_IP=cloudflarekeeps working; settingX-Judgebot-Client-IPwithAPI_CLIENT_IP=proxyis now the documented way. - Leave
/discord/interactionsunchallenged if you turn on Discord over HTTP. Discord’s requests come from its servers, not a browser, so they cannot pass a challenge: a Managed Challenge rule or a Cloudflare Access policy that covers that path, or Bot Fight Mode at all (it covers the whole zone and cannot skip a path), fails Discord’s URL check and every interaction.DISCORD_INTERACTIONSstaysgatewayunless you set it.
New variables, all optional, in .env unless noted (.env.example documents each,
and scripts/config.sh edits them):
- Roles and listeners:
JUDGE_ROLES;ADMIN_TOKEN(needed by--admin, at least 24 characters),ADMIN_ADDR(outside compose; compose sets0.0.0.0:8789in the container),ADMIN_PORT(the host port compose publishes it on, default 8789);MCP_ADMIN_TOKEN(the admin tools over/mcp).MCP_TOKEN,MCP_ADMIN_TOKENandADMIN_TOKENmust all differ: a process that reads two equal ones refuses to start. - Discord:
DISCORD_INTERACTIONS(gateway, the default, orhttp). - Exposure:
JUDGE_DOMAIN(the host name thepublicprofile serves;edgerefuses to start without it), andproxy, a new value ofAPI_CLIENT_IP. - The data:
JUDGE_REFRESH_HOURS(default 24;0in a development.env),JUDGE_CALL_RETENTION_DAYS(default: keep),FEEDBACK_IP_DAYS(default 90). - What the public sees:
JUDGE_FEED(default on),JUDGE_FEED_DAYS(default: no window),JUDGE_PUBLIC_FEEDBACK,JUDGE_PUBLIC_COST,JUDGE_PUBLIC_MODELS(each default false). - Spend and language:
JUDGE_SPEND_WARN_AT(default: no warnings),JUDGE_LOCALE(defaulten). A badJUDGE_LOCALEstopsjudgebot, thejudgebot ingestcommands that write data (they keep the spend ledger, whose alerts it sets the language of) and thebackupservice, which reads it from.envthrough Compose. .env.deploy:BACKUP_EVERY_DAYS(thebackupservice’s interval, default 7),JUDGE_ALERT_WEBHOOK(thebackupservice’s alerts; it reads only that file),BACKUP_S3_REGION(defaultauto) andBACKUP_S3_ADDRESSING(defaultpath).- Retired:
API_INTERFACES(above);R2_ENDPOINT,R2_BUCKET,R2_ACCESS_KEY_IDandR2_SECRET_ACCESS_KEY, renamedBACKUP_S3_*(above).
A bad value of any of these stops every binary that reads it, at startup.
The deploy
-
Move the database onto the pinned image first, at a quiet moment:
git pull, move aJUDGE_IMAGE_TAGpinned to a 1.x release to this one, thenscripts/deploy.sh --db. This release pins thedbimage (pgvector/pgvector:0.8.7-pg16-bookwormand its digest) in place of the floatingpg16, andscripts/deploy.shrefuses until the container runs it. The database is offline for a few seconds while it is replaced; a backup is taken first when one is configured. -
Deploy with
scripts/deploy.sh. It sees a major upgrade from 1.x (the new image's migrations: major-from-1.x), takes the backup when one is configured, removes the 1.xbotandapicontainers, and startsjudgebotalone. Discord gets a maintenance reply while it migrates; the web app, the API on127.0.0.1:8787and the tunnel are down until it has, when the deploy starts the rest. The 1.x containers have no SIGTERM handler, so Docker stops them after 10 seconds, and an answer still in progress then is cut off. Later deploys within 2.x keep serving throughout (blue-green), and a later major takes the same command. -
Not with a bare
docker compose up -d. The compose file’sbotandapiservices are now onejudgebotservice, andup -dalone leaves the oldjudgebot-botandjudgebot-apicontainers running: one holds port 8787, or the old bot answers Discord beside the new one, every question twice.scripts/deploy.shremoves them. Afterwards,docker ps -a --filter name=judgebot- --format '{{.Names}} {{.Status}}'listsjudgebot-dbandjudgebot-proxyand nojudgebot-botorjudgebot-api; if either is there,docker rm -f judgebot-bot judgebot-apiremoves it. -
With
JUDGE_AUTO_MIGRATE=false, migrate by hand before deploying.scripts/deploy.shrefuses a major upgrade with the setting off, before it changes anything, because the new container would never migrate it; its message names these steps:Terminal window docker stop judgebot-bot judgebot-api # the 1.x containersgit pull && docker compose pulldocker compose run --rm refresh migrate # from the new imagescripts/deploy.sh # now an in-place deployUntil the migrations are applied, the feed, web rating, the dashboard and the admin dashboard read a SQL function they add (
publicly_visible) and answer500. -
Rebuild
judge-cliandjudge-mcpwith the image. A 1.x tool has no release check and fails on this release’s schema with SQL errors.
After the deploy
-
Check the tunnel’s service in Cloudflare: the public hostname of your tunnel, under Zero Trust → Networks → Tunnels.
http://api:8787, what the runbook said, needs no change: the name now belongs to theproxy. A service naming the 1.x container (http://judgebot-api:8787) stops resolving: change it tohttp://proxy:8787. -
Nothing changes in the Discord Developer Portal. The commands are the same six and re-register at startup; the rating forms need no permission or intent. Over the gateway a fault’s rating shows a menu before the form, over HTTP one form (D38): to get the one-step form, switch to
DISCORD_INTERACTIONS=http(docs/DEPLOYMENT.md, “Discord over HTTP”). -
Use the service name with Compose. The
judgebotcontainer has no fixed name (Compose names it after the checkout’s directory, such asmtg-judgebot-judgebot-1), sodocker compose logs judgebot,scripts/deploy.shafter ajudge.tomledit, anddocker compose run --rm --entrypoint judge-cli judgebot …forjudge-cli, instead ofdocker logs judgebot-bot.scripts/refresh-data.sh, its cron entry anddocker compose run --rm refresh <command>keep working. -
A compose file or script of your own that runs
judge-bot,judge-apiorjudge-ingestmoves tojudgebot:judge-botisjudgebot --discord --jobs,judge-api --api --webisjudgebot --api --web --jobs, andjudge-ingest <command>isjudgebot ingest <command>. -
Running from source, the binary is
judgebot.cargo run -p judge-botbecomescargo run -p judgebot -- --discord --jobs,cargo run -p judge-api -- --api --webbecomescargo run -p judgebot -- --api --web(add--jobsfor the schedule), andcargo run -p judge-ingest -- <command>becomescargo run -p judgebot -- ingest <command>.target/release/holdsjudgebotin place ofbot,apiandingest. -
The schedule is on by default. An instance with no recorded refresh runs one within minutes of starting, so one that never had cron catches up that way. A cron’d
scripts/refresh-data.shkeeps working: it takes the same lease and writes the same record, so the schedule counts its run and never overlaps it. Remove the cron entry whenever convenient, or setJUDGE_REFRESH_HOURS=0to keep cron in charge. A development.envpointingcargo runat a local database should setJUDGE_REFRESH_HOURS=0, or that database is refreshed for real. -
The first refresh settles the curated lists. The alias and note lists loaded so far have no record of their source. For each, the
listsstep compares the table with what a built-in copy loads. When they match, the list is recorded as built-in (adopted) and kept current from then on. When they differ (you loaded your own file, or edited the table), it is left as it is, and every run logs a warning until you choose:judgebot ingest aliases <file>records your list as yours, andjudgebot ingest aliaseswith no file takes the built-in copy. Likewise fornotes. On a deploy host, mount the file:docker compose run --rm -v ./data:/data:ro refresh aliases /data/aliases.yaml. -
To move the backup off cron, add
backuptoCOMPOSE_PROFILESin.env, addJUDGE_ALERT_WEBHOOKto.env.deployif you want its alerts, and runscripts/deploy.sh. Then remove the cron entry forscripts/backup-db.sh, or keep it: the service counts the script’s backups, so the two can run side by side, and the script keeps working unchanged. ABACKUP_KEEP_LOCALthe script used is a host path, which the container does not see: mount a directory for the service (docs/DEPLOYMENT.md§6) or leave it to the script. -
An
openai-kind[models.embed]needs a price. An embedding model on akind = "openai"provider now needs[models.embed.pricing]or the provider markedpricing = "free"; otherwisejudge.tomlfails to load withmodels.embed: no price for <provider>/<model>. For OpenAI’s owntext-embedding-3-small, add under[models.embed]:[models.embed.pricing]input = 0.02 # USD per million tokensA local server (Ollama, vLLM) takes
pricing = "free"on its[providers.<name>]table instead. Voyage needs nothing. Areembedthat costs more than what is left underJUDGE_MAX_USDnow stops at the cap: run it asdocker compose run --rm -e JUDGE_MAX_USD=20 refresh reembed --yes. -
A
.refresh.lockdirectory left in the repository root by a killed run is no longer read and can be removed.
What changes for clients and stored data
- Discord members ask privately with
/judge-private./judge,/cardand/ruleno longer have aprivateoption. The bot registers the new commands when it starts, so there is nothing to do; tell your server if they used the option. - More is stored about each question. A Discord call stores the asker’s user id and
username, and a
/judge-privateanswer is stored (for the operator only)./helpsays so. Earlier calls are labelled by interface from their thread id, have no asker and stay public. An MCP orjudge-clicall from before the upgrade is labelledagent, because the two could not be told apart. - Earlier ratings keep their scores and have no reasons. A Partially correct or
Incorrect rating given before this release opens no feedback-queue item; it still
counts toward the examples and in
judge-cli stats’ lowest-rated list. - “Did you mean…?” buttons sent before the upgrade answer “That choice has expired”. They held their question in the old process’s memory, as a restart always lost it. Ask the question again.
- HTTP clients:
POST /api/judgehas a new reply,{"kind": "maintenance", "message", "retry_after_secs"}(status 503 withRetry-After), during a major upgrade; an answer gainscalland each citationkind; an unreadable request gets a JSONerrorbody.GET /api/aboutgainsfreshness,locale,locales,feed,ratings,public_feedbackandstats. judge-cliand MCP consumers:judge-cli statshas the new shape above (a script readingby_day[].usdreadsspend_by_daynow, andratingsis an object).judge-cli failures(new) prints{"failures": [...]}. The MCP tools are a superset of 1.x’s:judgeandbegin_sessiontakeprivate, and the admin and queue tools are listed only toMCP_ADMIN_TOKENand stdiojudge-mcp. During a major upgrade every/mcpoperation fails with the maintenance message.
Rolling back
-
Back to 1.x means restoring the backup. A 1.x image has no release check: on this release’s schema it starts and fails on the queries the new schema broke. Stop
judgebotfirst (left running, it would migrate the restored database again), then restore into an empty database, sincepg_restore --cleancannot drop the 2.x tables that reference 1.x ones, and then start the previous compose file and image together. An image from before this release has nojudgebotbinary, so the new compose file cannot start it.Terminal window docker compose stop judgebotdocker compose run --rm --no-deps -T backup fetch judgebot-<stamp>.dump.gz \> judgebot-<stamp>.dump.gzdocker compose exec -T db dropdb -U judgebot --force judgebotdocker compose exec -T db createdb -U judgebot judgebotgunzip -c judgebot-<stamp>.dump.gz \| docker compose exec -T db pg_restore -U judgebot -d judgebotgit checkout v1.2.0 -- docker-compose.yml # and JUDGE_IMAGE_TAG=1.2.0 in .envdocker compose pull && docker compose up -d --remove-orphansscripts/backup-db.sh fetch <name>downloads a backup without thebackupservice; a localpg_dump -Fcfile restores the same way withoutgunzip.--remove-orphansremoves thejudgebotandproxycontainers, which would otherwise hold port 8787 or answer Discord beside the restoredbot. -
Within 2.x, a rollback starts and runs: migrations within a major only add. A 2.x image refuses to start on a later major’s schema, naming both builds.
1.2.0 - 2026-10-08
Section titled “1.2.0 - 2026-10-08”-
Claude Haiku 5.5 is measured and priced. The built-in price table knows
claude-haiku-5-5and both of its rate cards ($0.10 input, $0.50 output per million tokens, and five times that for every token of a request whose prompt passes 100K), so ajudge.tomlnaming it needs nopricingtable. With noeffort, it synthesizes atmedium. On the gold set it cost five cents for the run: none of its answers contradicted the reference, 14 of the 18 in-scope questions were answered in full, 2 in part, and 2 got “did you mean?” (eval/published/v1-haiku-5-5.json). Sonnet 5.5 stays the recommended budget option. The README’s results and the Model choice page compare all three. -
judge-config, a config editor on localhost. It editsjudge.tomland.envfrom forms generated from the loader’s own types. The help text comes from the files’ documentation. Each draft runs through the loaders the binaries run at startup, and the panel says whether the database settings, the models, the Discord bot and the HTTP API would start, naming the key to fix if not. Only changed lines are written, after a diff. Secrets are write-only: never shown, and replaced from a masked input.scripts/config.shruns it from the image (aconfigcompose service that needs no.env, so it is the first setup step), andcargo run -p judge-configurefrom source. The setup guides now use it, with hand-editing.envas the alternative. See the Config editor page.
Changed
Section titled “Changed”- The eval binary is
judge-eval, the name the documentation already used. It was built aseval. - Log wording. The resolver’s
card resolved/card ambiguouslines name the matching step asstep=(wasrung=), and the embedding-space lines sayvector search off/on(wasvector legs). A log query filtering on the old text needs updating. .env.examplegains commentedANTHROPIC_BASE_URL,VOYAGE_MODELandVOYAGE_DIMENSIONSlines, and each comment block now sits directly above the variables it describes. No variable’s meaning or default changed. ItsAPI_INTERFACESexample is quoted: uncommented as it was (--api --webbare), everycargo runbinary refused the.envat startup. Compose was unaffected.- The image builds the web page and the docs on Node 26 (was 24). CI and the
contributor setup moved with it. The site holds TypeScript at 6 until
@astrojs/checkaccepts 7. - Dependencies are current. Cargo, the web page and the docs site take the pending
Dependabot updates, and
yoke-derivemoves past a yanked release thatcargo denyrefused. - Contributor tooling. The pre-push hook runs only the groups the pre-commit hook
leaves out (
sqlx,testandlint), andscripts/clean-target.shtrims atarget/directory that has grown large. Neither affects a deployment.
- A local image build no longer copies
.envinto a layer.COPY . .took the operator’s.env, and a.env.deployif present, into the builder stage. The build context now leaves out both and keeps only.env.example, whichjudge-configcompiles in. Images published from CI were not affected, since a clone has no.env.
Upgrading
Section titled “Upgrading”Nothing is required: docker compose pull && docker compose up -d is the whole upgrade,
and no migration is involved.
judge-configneeds a 1.2 image.scripts/config.shruns the editor from the image, so a deployment withJUDGE_IMAGE_TAGpinned to1.1or1.1.0has to move to1.2(orlatest) first.- A log query on
rung=orvector legsneeds the new wording (step=,vector search off/on), as described above. - A copy of
.env.examplekept as a.envwithAPI_INTERFACESuncommented bare (--api --web) should quote it, as the file now does.
1.1.0 - 2026-09-29
Section titled “1.1.0 - 2026-09-29”Changed
Section titled “Changed”- The default model is Claude Opus 5.5 (
claude-opus-5-5) on both stages, and the built-in price table knows its rates ($4 input, $20 output per million tokens). On the gold set, with the fixes below, it answered all 18 in-scope questions, all agreeing with the reference, for about 30% less per run than Opus 5. The README’s results and the Sample answers page come from that run (eval/published/v1-opus-5-5.json). The price table lists current models only, so it no longer knowsclaude-opus-5, and an unknown Anthropic model now prices at Opus 5.5’s rate. Ajudge.tomlthat still namesclaude-opus-5should give it a[models.<stage>.pricing]table ($5 input, $25 output), or the cap under-counts it. - Claude Sonnet 5.5 is the measured budget option. The built-in price table knows
claude-sonnet-5-5($2 input, $10 output, $0.20 cache reads per million tokens), so ajudge.tomlnaming it needs nopricingtable and the spend cap no longer prices it as Opus 5. On the gold set, at high effort on this release’s code, every answer it gave agreed with the reference, for about 60% of the default’s cost per answer. It answered 17 of the 18 in-scope questions and asked “did you mean?” on the other (eval/published/v1-sonnet-5-5.json). The README’s results and the Model choice page compare it with the default. - Server-side refusal fallbacks are sent on the direct API only. Anthropic documents
the
fallbacksbeta for the Claude API only, so Claude Platform on AWS no longer sends it, as the proxy, Bedrock and Vertex doors already did not. A provider table’s newrefusal_fallbacks = true | falseoverrides the door’s default either way. - The extraction and synthesis prompts were reworded. Both were written for earlier
models. The extraction prompt now lives in
crates/bot/src/prompts/extract_system.mdbeside the synthesis one. Its card-span rules are one list, it says what makes a shortened name clear, and a shortened name is replaced by a full name added for a group nickname too, which Sonnet 5.5 did not do (“tower” beside “Urza’s Tower”). The synthesis prompt keeps every rule, with the capitals gone, the face label named as not citable, and the retry instructions left to each rejection’s own notice. judge-eval answerrecords the retry. Each row says why the first attempt was rejected when the retry ran, and how many stub citations were dropped, and the table totals both by kind. Citations are stored with their whole quote.rescoreandshowstill read older run files, and say those did not record it.- Synthesis effort follows the model. With no
efforton[models.synth], Claude Opus 5.5 now synthesizes atmedium(it washigh), Claude Sonnet 5.5 stays athigh, and any other model runs athigh. On the gold set, Opus at medium answered all 18 in-scope questions in agreement with the reference with no wrong asides, as at high, for slightly less; Sonnet at medium misdescribed a card on one question and made three wrong asides (one run each). Ajudge.tomlthat setseffortis unaffected, and so is Bedrock’santhropic.claude-opus-5-5, which is not in the table. An answer cut off atmax_tokensis still retried at medium, and now at low when it was already medium.judge-eval answerrecords the synthesis effort in the run file. The published Opus run, the README’s results and the Sample answers page are the medium run. - The gold set separates decisive rule ids from supporting ones.
eval/gold.yamllists what a correct answer must cite (decisive_rule_ids) apart from background it may leave out (supporting_rule_ids).judge-eval answerandrescorescore recall on the decisive ids and report supporting ids that were cited;recallstill gates on both. The oldexpected_rule_idskey, like any unknown key, is now an error.
- Citations written one field over. With structured output the model writes keys in
the schema’s order, which was alphabetical: a citation’s
idbefore itskind, a ruling’s quote before its key. The schema now asks forkind, then the reference, thenquote, as the prompt does. The retry after an unreadable citation now says a value was in the wrong field rather than assuming a placeholder. Together with the next fix, and one gold run before and after, Sonnet 5 went from 6 answered questions to 13. - “Did you mean?” for a shorthand the question had already made clear. The extractor sent “tower” beside “Urza’s Tower”. It now sends the full name alone when the message makes the card clear, and still leaves an unclear one (“Teferi’s” with no hint which Teferi) for the user to pick.
- A nickname in brackets asked “did you mean?”.
[[bob]]offered Dark Confidant as a choice instead of answering. A bracketed span that is exactly an alias now resolves to that alias’s card: the alias table names one card for that spelling, so it is not a guess. Looser matches in brackets ([[bob's]], a near spelling) are still offered. - A second
lookup_rulescall ended the question. Asking for the tool again after its one call, or sending well-formed JSON whose ids are not rule ids, failed as an upstream error with no retry. Both are now a rejection that gets the one retry, with a notice saying what the call allows. Two of Sonnet 5’s five unanswered gold questions failed this way. The rerun of an answer cut off at the token limit can no longer fetch rules a second time. - A card’s type line cited as Oracle text failed twice. The material prints a face’s
name, mana cost and type line on its
[oracle …]label line, and only the Oracle text under it is citable. A citation quoting the label got the retry notice for a mistyped quote, so the model sent the same quote again and the question went unanswered (Sonnet 5.5, Valki // Tibalt). The notice now names the part of the label that was quoted and says to drop the citation, and the Cards heading says the label is not citable. - Examples were printed under the wrong rule. A rule-level excerpt printed all of a
rule’s examples, and its sub-rules’, after its last sub-rule, without their
Example:label. An example of 903.3 therefore read as part of 903.3e and was cited as 903.3e; 174 of the CR’s 277 examples were out of place. Each example now sits labelled directly under the line it belongs to, in the model’s material, thelookup_rulesresult and/rule. A quote of rule text filed under a neighbouring rule still gets a retry notice naming the rule that holds it. An existing database is corrected by the next CR release, or at once by reloading the current one (below). - Backslashes in answers. Sonnet 5.5 sometimes wrote line breaks as a literal
\n(and quotes as\"), which Discord and the web page showed as written. The answer’s stray escapes are now decoded when the verdict is made (D22). Calls stored before this keep them.
Upgrading
Section titled “Upgrading”- A
judge.tomlnamingclaude-opus-5needs a[models.<stage>.pricing]table ($5 input, $25 output), or the spend cap prices it as Opus 5.5 and under-counts it by about a fifth. The built-in table lists current models only. - Claude Platform on AWS no longer sends refusal fallbacks. Set
refusal_fallbacks = trueon that provider to keep sending them. - Optional: reload the current CR to move its examples under their rules now rather
than at the next CR release.
rules latestskips a version already loaded, so name the file:docker compose run --rm refresh rules "<the .txt link on Wizards' rules page>", thendocker compose run --rm refresh embed. About 150 rules are re-embedded (well under a cent). Stored calls keep their citations; the nightly retirement pass checks them against the new text as it does after any CR load.
1.0.0 - 2026-09-20
Section titled “1.0.0 - 2026-09-20”The first release. docs/ARCHITECTURE.md describes everything below as it stands, and
docs/DECISIONS.md records why.
- The judge pipeline. A Magic: The Gathering rules question goes through extraction and classification, card resolution, retrieval and synthesis. Every answer cites the Comprehensive Rules, Scryfall rulings, Oracle text or a rated prior call. Before the answer is shown or stored, every citation’s quote is checked verbatim against its source, and every rule number in the answer’s text must be one of those citations. A failed check gets one retry that is told what was rejected.
- Card resolution that never guesses. Aliases, printed names, short names and fuzzy
matches resolve in a fixed order,
[[bracketed]]names are taken exactly, and an ambiguous name becomes a “did you mean?” choice. - Retrieval over three legs: the question’s rules category, full-text search and vector search (optional, on when an embedder is configured), plus rulings, glossary entries, notes on notoriously difficult cards and rated prior calls.
- The Discord bot.
/judge(guild-only) with rating buttons and “did you mean?” buttons,/cardand/rulelookups that call no model,/help,/license, and/forget, which deletes the caller’s ratings (the only per-user data kept)./judge private: Trueanswers the asker alone and stores nothing. Each member getsJUDGE_USER_LIMITquestions per window (default 6 per 10 minutes). An Incorrect rating says where to report a wrong ruling. Ratings shape which prior calls are retrieved, and a rating from a member with the judge role overrides the crowd’s. Mana and card symbols render as application emoji. judge-api, with one flag per front door:--api(POST /api/judge),--web(the SolidJS page) and--mcp(the MCP transport at/mcp, which also needsMCP_TOKEN).GET /api/healthandGET /api/aboutare served whatever is switched off. The anonymous API is rate limited per client address (API_CLIENT_IPispeerorcloudflare), ahead of a concurrency limit and the spend cap. The page keeps a session’s history and offers “did you mean?” choices./mcpaccepts only the hosts inMCP_ALLOWED_HOSTSand limitsjudgeruns per window (MCP_JUDGE_LIMIT,MCP_JUDGE_WINDOW_SECS), which bounds what a leaked token can spend.- Agent sessions.
judge-cliand thejudge-mcpserver let an outside agent do the model’s work step by step, under the same citation validation and limits as the built-in pipeline, alongside card, rule, ruling and glossary lookups. - A spend cap on every model call (
JUDGE_MAX_USD), which reserves the worst-case cost before a request is sent.JUDGE_BUDGET_PERIOD=day|monthmakes it one budget for the period, shared bybotandapiand kept across restarts.JUDGE_ALERT_WEBHOOKis told when the cap trips and when the nightly refresh or the backup fails.judge-cli statsshows questions, spend and ratings per day. - Providers as configuration. With only
.env, the judge runs on Anthropic’s API with Voyage embeddings if keyed. Ajudge.tomlchooses a provider and model per stage: Anthropic direct, through a proxy, Claude Platform on AWS, Bedrock or Vertex, any OpenAI-compatible chat completions endpoint, and Voyage or OpenAI-compatible embeddings.judge.example.tomldocuments every knob. - Embedding-space tracking. The database records which embedder’s vectors it holds.
A mismatch turns the vector legs off instead of mixing spaces, and
judge-ingest reembedswitches space. - A one-command first load.
docker compose run --rm refresh initcreates the schema and loads the cards, the rules, the alias and note lists (built into the binary) and the embeddings, with no Rust toolchain on the host. - Data ingest and nightly refresh (
judge-ingest): Scryfall cards and rulings, the Comprehensive Rules (a new release is detected from Wizards’ rules page), aliases, notes, embeddings and Discord emoji. A renumbered rule keeps the calls that cite it. A call is retired when its citations or its cards’ Oracle text stop holding, and restored when they hold again. - The source offer and operator contact on every remote interface (AGPL-3.0-or-later
§13): the repository, the commit the binary was built from, the licence and who runs
the instance.
JUDGE_SOURCE_URLpoints the offer at a fork. The bot requiresJUDGE_OPERATOR_DISCORDandjudge-apirequiresJUDGE_OPERATOR_EMAIL. - Deployment. A
docker composesetup (Postgres with pgvector,bot,api, an optional Cloudflare Tunnel or any reverse proxy, arefreshjob) with a database-backed healthcheck, migrations applied at startup and a backup script for Cloudflare R2. The published image is a manifest list forlinux/amd64andlinux/arm64. A GitHub releasevX.Y.Ztags the image built for that commit asX.Y.Z,X.YandX, andJUDGE_IMAGE_TAGaccepts those alongsidesha-<short>. - An evaluation harness (
judge-eval): a retrieval gate that needs no API key and a 21-question gold set for live runs. Two graded runs are published undereval/published/with the results in the README: on the default configuration 17 of 18 in-scope questions answered, all 17 agreeing with the reference, and the eighteenth asked which card was meant. - The documentation site, organised around running your own judgebot, from the
canonical files in
docs/.
MTG Judgebot is unofficial Fan Content permitted under the Fan Content Policy. Not approved or endorsed by Wizards of the Coast. Portions of the materials used are property of Wizards of the Coast. ©Wizards of the Coast LLC.
The Comprehensive Rules come from Wizards of the Coast. Card data, rulings and card symbols come from Scryfall, which is not affiliated with this project. Rule links go to the Yawgatog mirror. License and attribution.