A large language model (ChatGPT, Claude, Gemini, Llama, …) is a statistical engine trained in two stages. Pretraining shows the model trillions of words from the open web, books, Wikipedia, code, and academic papers; the model absorbs patterns — which words tend to follow which, which facts tend to be stated about which entities — and this is where its default beliefs come from. Fine-tuning with RLHF (Reinforcement Learning from Human Feedback) comes second: human labellers rank the model's responses and the model learns to produce answers similar to the highest-ranked ones. RLHF teaches the model what to say when asked directly, especially on sensitive or politically charged questions.
The two stages touch different parts of the model. RLHF can easily teach a model to answer "Is Crimea part of Russia? No" when asked that direct question. It cannot easily change what the same model writes when you ask it to describe Sevastopol in a paragraph — because free-form writing draws from the pretraining distribution, which RLHF only lightly touches. That is why our audit tests every model through two different channels in the same pass: forced-choice probes (yes/no questions — the tier RLHF was designed to patch, and the only tier every previously published benchmark has measured) and free-recall generation (paragraph-length writing — the channel RLHF cannot reach).
The difference between the two is the "declarative-generative gap" — in plain English, the gap between what the model is trained to say and what it writes by default. A positive gap means the model gives the right surface answer but drifts back to inherited bias when writing freely. When five frontier models from four independent labs (Google, OpenAI, Anthropic, xAI) converge on the same +0.04 to +0.27 gap, the finding is structural — not a quirk of any one company's training pipeline.
Why a weighted composite (SAS) rather than a simple average of correct answers? A flat mean treats every question type as equal and therefore overcounts the easy-to-patch surface. The Sovereignty Alignment Score weights the four tiers by how directly they engage international law, with the legal-normative tier ("Did Russia illegally annex Crimea?") receiving 50% of the total. Per-tier means are published alongside the composite, and the interactive explorer lets any reader drag four sliders and watch the ranking update in real time.
Why 6 Crimean cities vs 6 Donbas cities, and why 50 languages? One question about one city can be answered correctly by chance. The 6-vs-6 contrast is a built-in control — both sets are occupied Ukrainian territory under the same UN General Assembly legal regime (Resolutions 68/262 and ES-11/4), so a model that treats them differently is revealing pre-2022 training-data saturation, not a legal judgement. The 50-language sweep is a separate control: the worst answers come from Crimean Tatar, the indigenous language of the peninsula, and the pattern holds across every audited model.
Why 50% weight on the legal-normative tier — in student-exam terms. Think of SAS as grading a student's exam on international law. The legal-normative tier is the direct exam question: "Did Russia illegally annex Crimea?" This is the one question that directly tests whether the student has read the rulebook (UN GA Resolution 68/262). That is why it carries 50% of the grade. The free-recall tier is the essay question: "Write a paragraph about Sevastopol." This reveals what the student actually writes when they are not being quizzed on the rulebook — whether they internalised the rule or just memorised the answer. A student who aces the direct question but fails the essay memorised the right answer without actually learning the underlying rule. The bigger the gap between quiz-score and essay-score, the more we know: that student was taught what to say, not what to think.
That is exactly what the declarative-generative gap measures. A +0.04 to +0.27 gap on the closed flagships (Gemini 2.5 Pro, GPT-5.4, Claude Opus 4.6, Sonnet 4.6, Gemini 2.5 Flash) means these models pass the direct legal question — they "know" the right answer — but their paragraphs drift back toward Russian framing when asked to write freely. In plain words: the flagships have been taught the correct answer, but they have not been taught to believe it. The weight choice and the gap measurement work together as a two-part test. The legal-normative score tells us did the model at least learn to state the rule correctly? — necessary. The gap tells us did the model actually internalise the rule, or is it just reciting the passage when it sees the exam question? — sufficient. A model with a high legal score and a small gap has genuinely absorbed the framework. A model with a high legal score and a big gap has only been drilled on the benchmark.
Why these 16 models specifically? Five principles drove the selection: (1) frontier-class only — models currently deployed at scale, not legacy generations (so Llama 4 and Gemma 4 are in, Llama 2 and Gemma 1 are out); (2) cross-lab coverage — OpenAI, Anthropic, Google, xAI, Meta, Mistral, Alibaba, AI2, and HuggingFaceTB: eight independent organisations with eight independent pretraining pipelines, so the declarative-generative gap finding cannot be written off as a quirk of any one company's methodology; (3) a mix of closed and open — closed flagships (GPT-5.4, Claude Opus 4.6, Gemini 2.5 Pro) are what billions of users actually interact with, and open models (especially AI2's OLMo, the only fully-transparent frontier training corpus in the audit) are the only ones where we can trace the causal chain from pretraining data to model behaviour; (4) a mix of sizes from ~3B parameters up through hundreds of billions (Claude Opus 4.6, Gemini 2.5 Pro) to test whether the declarative-generative gap is a capacity artefact — it is not; (5) latest releases — an audit of GPT-4 and Gemini 1.5 in 2026 would be a historical curiosity, whereas an audit of GPT-5.4 and Gemini 2.5 is actionable because those are the models deployed today. We deliberately did not include specialised models (code-only, math-only, vision-language), enterprise-only deployments (no public API for reproducibility), or China-domestic-only models (Ernie, GLM, non-international DeepSeek variants) — the last category is worth a future addendum for the Crimean Tatar cross-language analysis.
