Skip to content

Explainer 2: Data and retrieval

Part two of the explainer. Part one covered the pipeline end to end.

Everything lives in one Postgres 16 database with two extensions: pgvector (vector columns and indexes) and pg_trgm (trigram similarity). Migrations are in crates/bot/migrations/.

Table Source Refreshed Notes
cards, card_faces Scryfall bulk oracle_cards daily one row per Oracle identity, and faces hold the Oracle text
printed_names Scryfall bulk default_cards daily every name ever printed, for old or errata’d names
rulings Scryfall bulk rulings daily keyed by a hash of the content, so a re-import is the same ruling
rules, glossary the CR .txt from Wizards on release, detected daily see chunking below
card_aliases data/aliases.yaml when the built-in copy changes, detected daily nicknames
card_notes data/notes.yaml when the built-in copy changes, detected daily nightmare cards
curated_lists each alias or note load with the list where each list came from (built-in or a file), and its digest
categories data/categories.yaml, via the compiled enum with each CR load category → CR subsections
calls, ratings every interface continuous answers and their votes
embedding_space ingest embed on switch one row: which embedding model the vectors came from
agent_sessions the agent surface continuous state for the step-by-step agent mode

Scale is small: ~30k cards, ~2k rule chunks, well under 10k calls, so nothing here needs a dedicated vector database.

A text file is useless to search until it is cut into pieces (“chunks”). Where to cut matters more than any other choice in a RAG system, because a piece is what gets found, what gets shown, and what gets cited.

The CR is a numbered hierarchy: section 702 (Keyword Abilities) → rule 702.19 (Trample) → sub-rules 702.19a, 702.19b. The parser emits rows at two granularities:

  • Rule-level rows (702.19): the body is the rule’s own sentence plus every lettered sub-rule, each Example: paragraph directly under the line it belongs to. These are the search unit. They get embeddings and appear in retrieval results. A rule plus its sub-rules is usually one coherent idea of a few hundred words, the right size for a model to read in one piece.
  • Leaf rows (702.19b): one line each, with parent_id = 702.19. These are the citation unit. A model that quotes sub-rule b should cite 702.19b, not the rule above it.

Scoring and lookups treat a leaf and its parent as covering each other. No rows exist for three-digit sections. Asking for 702 expands to every rule in it.


The bot combines three search techniques because each fails differently.

This source is structured and always on. The classifier put the question in triggered_abilities. The YAML says that category maps to CR sections 603 and 113.3. Every rule in those sections goes into the context. Rules that share a word with the question come first, ranked by the same relevance score as source B. The secondary categories’ rules are added the same way, after the other sources.

It costs nothing and never misses when the classifier is right. It gives the model the surrounding rules it needs even when the “obvious” rule alone is not enough. It fails when the classifier is wrong or when the answer lives in a section nobody would file the question under.

This source matches keywords, using Postgres’s built-in full-text engine. Each rule row has a generated tsvector column: the text split into words, lower-cased, reduced to word stems (“triggers” → “trigger”), with common words like “the” removed. The question is reduced the same way, and a row matches if it contains any of those stems. Rows are ranked with ts_rank_cd, a relevance score in the same family as BM25. Such scores count the matching words, weight rare words higher, and discount long documents. Matches on concept phrases from the extraction weigh double those on the raw question. The top 12 rows are taken.

This finds rules that share vocabulary with the question: “leaves the battlefield”, “in response”, “upkeep”. It is exact, cheap, and needs no external service. It fails when the user and the CR use different words for the same idea (“dies” versus “is put into a graveyard from the battlefield”). Common words also distract it.

This source matches meaning.

Embeddings. An embedding model is a neural network that turns a piece of text into a list of numbers, a vector, typically 512 to 3072 floats long. This bot uses Voyage AI’s voyage-3.5 at 1024 dimensions by default. The model is trained so that texts with similar meaning land near each other in that space, whatever words they use. “Dies in response to its trigger” and “an ability that triggers on leaving the battlefield resolves even if the source is gone” should be neighbours even though they share almost no words.

Similarity. Two vectors are compared by cosine similarity: the cosine of the angle between them, 1.0 for identical direction, 0 for unrelated. pgvector exposes this as the <=> operator (cosine distance, 1 minus similarity, so smaller is closer). The source is essentially:

SELECT ... FROM rules
WHERE parent_id IS NULL AND embedding IS NOT NULL
ORDER BY embedding <=> $question_vector
LIMIT 12

(The real query also restricts id to the NNN.N rule pattern.)

Indexing. Comparing the question against 2,000 rule vectors by brute force would be fine at this scale. pgvector also provides an HNSW index (Hierarchical Navigable Small World), a graph structure that finds approximate nearest neighbours quickly. It is approximate, so it can occasionally miss the true nearest row. That is acceptable here because the other two sources cover for it. The index is partial: it covers rule-level rows only (WHERE parent_id IS NULL). Every row it returns is therefore one the query wants, and no filter applied afterwards can shrink the result below the limit.

Query versus document. Voyage’s API takes an input_type of document or query. The model embeds a short question differently from a long passage so that the two match up better. The ingest job embeds rules as documents. The retriever embeds the user’s question as a query.

Cost and storage. Embedding is paid per token, once per rule, at ingest time. A new CR release re-embeds only the rules whose text changed. The loader nulls those embeddings and the next refresh’s embed step fills them. Each question costs one small embedding call.

The vector search has failures of its own. It matches by meaning, so it returns rules that are about the same theme without being the one that decides the question. It cannot tell 702.19 from 702.20 if their wording is similar. Its operational problems are listed in §7.

The sources are merged in priority order, and a rule found twice is kept once:

  1. the primary category’s rules that share a word with the question,
  2. full-text hits,
  3. vector hits,
  4. the rest of the primary category,
  5. each secondary category.

The synthesis budget shows the model only the front of this list, so the order decides what it reads. The primary category’s matching rules always survive, and the secondary categories are trimmed first. The retrieval gate in the eval suite (judge-eval recall) requires that at least 90% of the gold set’s expected rule ids appear in the context, and at least 75% in the part the budget shows.

Whatever all three miss is left to the one lookup_rules tool round in synthesis. Having read the material, the model can ask for rules by number once.

The prior-call query is the feedback loop. Earlier answers are stored with their own embedding (of the question). The prior-call query picks calls in the same categories, about at least one of the same cards, not retired and not down-voted. It orders them by vector distance to the new question. The rating is a Bayesian-smoothed mean of the Discord ratings (an anonymous web rating is never counted): (2.0 × 3 + Σ scores) / (3 + n). That is “pretend there were three votes of 2.0 before anyone voted”. One 3 does not make a call look perfect, and one 1 does not bury it. The latest rating from someone with the Judge role overrides the crowd. Calls scoring under 1.5 are excluded: with five or more votes on the crowd’s score, at once on a judge’s.

Prior calls are rendered after all CR material, as examples of how a question was answered, and the prompt says they never outrank it.


The model does not write free text for the program to parse. It fills a JSON schema derived from the Verdict struct. The lookup_rules tool round is bounded to one by a typestate:

  • Synth<Fresh> can send and become Synth<ToolRequested>.
  • Synth<ToolRequested> can fulfil the request and become Synth<Final>.
  • Synth<Final> has no method that requests tools.

A second tool round is therefore code that does not compile.

The model can still ask for a second round, or send ids that are not rule ids. Neither is fetched. Each is a typed rejection (ToolMisuse) and gets the one retry, with a notice saying what the call allows. The retry may make the call itself only when the first one could not be read, because then nothing was fetched.

The validator (crates/core/src/verdict.rs) checks each citation against the context:

  • The reference must exist: the rule id was shown or fetched, the ruling key was rendered under that card, the prior-call id was in the list, the Oracle face exists.
  • The quote must be a contiguous substring of that source’s text.

The most common rejection in practice was punctuation. The CR is typeset with curly apostrophes (doesn’t) and em dashes. Models reliably retype them as ASCII (doesn't) even when told not to. Rejecting a correct citation over one character wastes the only retry. So the comparison (crates/core/src/quote.rs) maps each character to one plain equivalent: curly quotes to straight, every dash to a hyphen, a non-breaking space to a space. The leniency applies only while matching. What gets stored is the source’s span, not the model’s string, so the database is byte-exact and later strict checks stay strict. Case, word order and line breaks must still match, so a paraphrase is still rejected.

A verdict with no citations, or an answer under 40 characters, is rejected as empty. Two fields the model could misreport, source and cr_version, are not in the model’s schema. The program stamps them from the extraction and the retrieved chunks.

Verdict<Unvalidated> is what JSON decodes into. Verdict<Validated> is the only type CallStore::persist and the Discord renderer accept, and the only way to make one is validate(). Serde’s Deserialize is implemented for the unvalidated state only, so decoding straight into the validated one does not compile, and deleting the check is a type error.

There is one retry, with a “Previous attempt rejected” notice showing the failing citation and why. The retry starts with the tool disabled, so the model cannot spend another round. Any rules the first attempt fetched are rendered regardless of budget. The first rejection is logged at INFO so that when the second attempt also fails, the operator can read both.