Every claim checked against the official voting booklet. Each Yes
or No comes with the booklet line behind it.
GeaNaVid reads the Swiss Federal Council's voting booklet and a
claim in German, French or Italian. It answers
Gea, Na or Vid,
Romansh for yes, no and empty: the booklet supports the claim,
contradicts it, or does not say. Every Gea and every Na points to the page
and passage it rests on.
Voters hear the claim. The answer is on page 4 of 40.
Before every federal vote, each household receives the booklet:
the proposal, the legal text, the arguments of the committee and
the position of the Federal Council and Parliament, in German,
French and Italian. During the campaign, claims about it travel
fast. Checking one means finding the right paragraph in 40 to 90
pages, often in another language than the claim.
Claim · German
„Gemäss dem Abstimmungstext ist die Bevölkerung seit Einführung
der Personenfreizügigkeit im Jahr 2002 um rund 500'000 Personen
gewachsen.“
Booklet of 14 June 2026 · page 4 · In Kürze
„Seit Einführung der Personenfreizügigkeit im Jahr 2002 ist die
Bevölkerung um rund
1,7 Millionen Personen gewachsen.“
Na · contradiction (2)
Approach
Narrow first, then let Apertus read.
Sending the whole booklet to the model costs about 20,000 input
tokens per claim. GeaNaVid gives Apertus the few passages of the
right proposal instead, about 1,500 to 2,200 input tokens per claim.
Read the booklet
The PDF is split into passages with page, proposal, section and
author, including the note that a double page was written by the
committee. No language model, no tokens.
Find the proposal and the passages
The vote title narrows the booklet to one proposal. A claim in
another language is translated by Apertus into the booklet
language; BM25 then picks five passages and their neighbours.
Ask Apertus
One call to Apertus v1.5 with numbered passages. The answer is
strict JSON: first the passages that decide, then the label.
With 8B, unsure verdicts are re-checked claim part by claim part,
and a neutral verdict Apertus is not fully sure of is asked once
more whether one detail is contradicted.
Quote the deciding sentence
The evidence is the booklet sentence that covers the claim best,
verbatim and with its PDF page, chosen without another model
call. A claim that names its source ("the summary says", "the
committee thinks") is answered from that section.
Results · 14 October 2026
OST v1.1, test split
OST's benchmark MNLIoverSwissVotingBooklets v1.1, split by voting
date: 493 test pairs from 7 voting dates, 21 booklets, two thirds
cross-lingual. Macro-F1
over entailment, neutral and contradiction; input tokens per claim
from the API. Both models are shown: switch between Apertus 8B and
70B, or compare them side by side.
Results with Apertus 70B
Macro-F1, test split (493 pairs)
0.965[0.944–0.983]
Whole booklet, same model, test split (493)
0.889 at 20,242 tokens
Macro-F1, 546 dev claims outside the tuning subset
0.971
Macro-F1, Beginner task, test split (493)
0.980
OST evidence score, 324 gold entailments / contradictions of the test split
96.3 %
Input tokens per claim, test split (493)
1,461
Time per case, mean / p95 (cold, 50 dev cases)
1,274 / 2,197 ms
OST's Docker run with OST's evaluate.py, task A / B macro-F1 (493 + 493 cases)
0.965 / 0.976 B 0.974 in a second run
OST's Docker run, task A (493 cases): evidence found; input tokens; time mean / p95
312 of 324 · 1,460 · 1,059 / 2,344 ms
Results with Apertus 8B
Macro-F1, test split (493 pairs)
0.959[0.933–0.980]
Whole booklet, same model, test split (493)
0.720 at 20,242 tokens
Macro-F1, 546 dev claims outside the tuning subset
0.971
Macro-F1, Beginner task, test split (493)
0.977
OST evidence score, 324 gold entailments / contradictions of the test split
96.9 %
Input tokens per claim, test split (493)
2,159
Time per case, mean / p95 (cold, 50 dev cases)
366 / 545 ms
OST's Docker run with OST's evaluate.py, task A / B macro-F1 (493 + 493 cases)
0.961 / 0.977
OST's Docker run, task A (493 cases): evidence found; input tokens; time mean / p95
314 of 324 · 2,163 · 469 / 1,443 ms
Apertus 8B and 70B side by side
Measure
8B
70B
Macro-F1, test split (493 pairs)
0.959[0.933–0.980]
0.965[0.944–0.983]
Whole booklet, same model, test split (493)
0.720 at 20,242 tokens
0.889 at 20,242 tokens
Macro-F1, 546 dev claims outside the tuning subset
0.971
0.971
Macro-F1, Beginner task, test split (493)
0.977
0.980
OST evidence score, 324 gold entailments / contradictions of the test split
96.9 %
96.3 %
Input tokens per claim, test split (493)
2,159
1,461
Time per case, mean / p95 (cold, 50 dev cases)
366 / 545 ms
1,274 / 2,197 ms
OST's Docker run with OST's evaluate.py, task A / B macro-F1 (493 + 493 cases)
0.961 / 0.977
0.965 / 0.976 B 0.974 in a second run
OST's Docker run, task A (493 cases): evidence found; input tokens; time mean / p95
314 of 324 · 2,163 · 469 / 1,443 ms
312 of 324 · 1,460 · 1,059 / 2,344 ms
Less context, better verdicts: 8B with the
selected passages (0.959) beats 70B with the whole booklet
(0.889), at 7 to 11 % of the tokens (test split, 493 pairs).
Translation is the largest step: cross-lingual
claims (330 of the 493 test pairs) rise from 0.70 to 0.94 with 8B
and from 0.74 to 0.94 with 70B (ablation step; the full defaults reach
0.943 and 0.958) once Apertus translates them into the
booklet language.
Evidence as OST scores it: one of the first five
quoted sentences lies in OST's gold passage for 97 % (8B) and 96 %
(70B) of the 324 gold entailment and contradiction cases of the test
split, by the rule of OST's own evaluation script.
Claims outside the tuning subset: 546 development claims
that only confirmed options chosen on the tuning subset give 0.971
(8B) and 0.971 (70B). The test split was scored 26 times, for 24
configurations.
Time as OST measures it: cold run on 50
development cases, mean / p95 per claim 366 / 545 ms (8B) and
1,274 / 2,197 ms (70B); 48 ms of it is our own code. Re-checking only
unsure neutral answers cut the 8B p95 by 44 % at the same F1.
New booklets: on a synthetic hold-out from three
voting dates outside OST v1.1 (89 claims, labels checked blindly by four
open-weight models) the lead over the whole booklet holds: 0.874 (second evaluation; first 0.845) vs
0.665 (8B), 0.932 vs 0.872 (70B).
Older booklets: OST v1.1 covers 2020–2026. On two
synthetic stress sets from 1998–2018 (26 voting dates, 802 claims;
claims by an open model, labels agreed by two other open models, no
human check) task A first reached only 0.79 (8B) and 0.82 (70B),
because older layouts name no proposal in their page headers.
Reading the proposal structure from the booklet's own outline raised
it to 0.84 and 0.89. One held-out half first got worse (a booklet
with a single proposal); the fix was found there, so the report
shows both numbers. OST results are unchanged.
OST v1.1 repeats 122 of the 493 test pairs, so the 95 % intervals
resample copies together: 0.933–0.980 (8B), 0.944–0.983 (70B);
without the copies the defaults score 0.957 (8B) and 0.962 (70B).
8B and 70B do not differ significantly. Variants were built on
development data, but test scores were seen 26 times, for 24
configurations and decided some of them, so the 546
development claims and the synthetic hold-out are the honest
checks. The rounds are in docs/ITERATIONS.md.
Open
open OST's weighting of F1, evidence,
tokens and time
open Token counts of OST's proxy
Video
Two minutes with GeaNaVid
Pitch
Nine slides, one idea
The pitch deck runs in the browser. Arrow keys move between
slides; a PDF version is linked inside.
Architecture, use of Apertus, data, evaluation and limitations,
with sources. State of 14 October 2026 with the results on OST
v1.1; open points are marked as such.
Questions, ideas or feedback on GeaNaVid? Write to Team Apertool and we will answer by e-mail.
Your name, e-mail address and message are forwarded by e-mail to Team
Apertool, solely to answer you. The form does not store them and does not
pass them on. This site uses no analytics and sets no cookies.