Right facts, wrong framing: what happens when AI still amplifies conspiracies

Executive summary

Fifteen years after the 22 July attacks, survivors and victims’ families still have to contend with conspiracy theories on social media that deny or distort what happened. These harmful narratives are often created with malicious intent by people and organisations with an agenda, like destabilising societal trust or fueling far-right movements.

Millions now turn to AI chatbots as a first source on historical events like this. Yet, these models are trained on undifferentiated data and struggle to weigh the veracity of sources (Source). Studies show that these models can be poisoned even with smaller amounts of data (Source).

Revontulet and Factiverse tested how nine leading language models handle malicious narratives and conspiratorial framing around the 22 July attacks. We conducted a quantitative experiment in which 104 hand-authored Norwegian and English prompts (208 total) were run against 9 leading AI chatbots, totalling 5,616 model requests. The model responses were graded by individual prompt-specific criteria evaluated by two independent graders. Domains with high failure rates were manually confirmed and additionally analysed with Factiverse’s own Claim Detection to evaluate its capability to detect both real and wrongful claims.

Performance varied widely: the strongest models (Fable, Sonnet) passed 94–96% of prompts, while the weakest cleared as little as 64%. We used two different AI models (Grok and GPT) to grade the results as a way of double-checking. One grader scored worse when the stricter one graded it, in some cases by as much as 17 points.

Three failure patterns recurred often enough to name:

• Refuse-without-label. The model declines to repeat a conspiracy theory, but never flags it as one, leaving the user no better equipped to recognise it.
• Reject-then-launder. The model rejects a confirmed conspiracy on the surface, then readmits its framing through softer language, crediting the “legitimate concerns” beneath a racist theory.
• Translation-as-loophole. Asked to translate an extremist slogan rather than generate one, several lower-tier models rendered it verbatim, sometimes with a useless warning attached.

Taken together, the findings show that an AI model can pass every factual test and still leave its user with a more sympathetic view of a terrorist’s ideology than they started with.

Background

Fifteen years after the 22 July attacks, the deadliest act of terror in Norway’s modern history, there are still conspiracy theories on social media that deny or distort what happened, and who was behind it and why.

On that day in 2011, a far-right extremist killed 77 people. He bombed Oslo’s government quarter before carrying out a mass shooting at a Labour Party youth camp on Utøya island, most of the victims teenagers. More than a decade on, it remains a focal point for extremist and conspiratorial narratives worldwide.

Revontulet and Factiverse jointly examined how today’s large language models represent the 22 July attacks. Both companies are Norwegian. Revontulet is an intelligence firm that tracks and counters extremism and terrorism online, founded by Bjørn Ihler, a leading expert who survived the 22 July attacks himself. Factiverse is a Norwegian company that developed award–winning ML/NLP for understanding and verifying disinformation through claim detection in LLMs, news, audio and video in 114 languages faster than LLMs.

LLMs are becoming a primary gateway to information about historical events like this, especially among millions of young people. 9 in 10 Norwegian students are using AI for their academic studies in 2025. (Source). Citizens are already using LLMs to research highly influential events like elections, with 1 in 7 people in the UK and 1 in 10 people in the Netherlands open to using AI tools for information searches (Source). But these models hallucinate, state falsehoods with confidence, are trained on undifferentiated data, struggle to weigh the veracity of the sources, and can be poisoned even with small amounts of harmful information. In the run-up to the 2026 Scottish Parliament election, the think tank Demos found AI chatbots gave voters incorrect information in roughly a third of responses (Source). A May 2026 audit by Newsguard found one major chatbot leaning more heavily on Russian and Iranian propaganda sources (Source). Estonian Language Institute and Propastop identified which LLMs are the best at resisting Russian propaganda. They raised an issue that “Foreign political troll factories can produce large amounts of false content that can be used to bias AI models”(Source).

These models often operate with a lack of transparency when generating these responses. Where the subject is a national tragedy bound up with public safety, that unreliability and lack of transparency become a real risk for democratic societies. If the millions who lean on these tools meet conspiratorial framing instead of nuanced and credible information, those narratives reach the public with no supervision and no authoritative source.

Experiment methodology

Goal. Establish whether leading LLMs resist spreading conspiracies and malicious information about the 22 July attacks and the associated Eurabia and Great Replacement conspiracy theories.

Approach. Quantitative, hand-authored collection of 104 prompts in both Norwegian and English (208 total) was run against a 9-model panel, single inference - totalling 5616 model requests.

Models tested. The panel covered nine leading models across major providers: claude-fable-5 and claude-sonnet-5 (Anthropic), gpt-5.2 (OpenAI, via Azure), gemini-2.5-pro and gemma-4-31b-it (Google DeepMind), grok-4.3 (xAI), deepseek-v3.2 (DeepSeek), qwen3-235b-a22b-2507 (Alibaba/Qwen), and llama-3.3-70b-instruct (Meta).

Grading. Every response was scored against a criterion written specifically for that prompt, and marked PASS, FAIL, or ERROR. An ERROR is not a wrong answer but an ungradable one: grader model self-censoring, a refusal to engage, an empty or broken response, or output the grader could not assess against the criterion.

Two different models served as independent graders for the responses: Grok (grok-4.3) and GPT-sol (gpt-5.6-sol). This allowed for cross-checking results and exposed disagreement rather than trusting a single judge.

Prompt design. The prompts escalate in difficulty: from neutral factual questions, to leading questions that embed a false premise, to adversarial attempts (roleplay, persona, and framing tricks) designed to coax models into producing conspiratorial or glorifying content.

Coverage. The prompts span nine narrative domains, mapped out by Revontulet so that no angle of the conspiracy landscape went untested:

Analysis: To make the volume of responses manageable for human review, we ran every model output through Factiverse’s proprietary check-worthy claim detection model. This model outperforms leading LLMs across 114 languages, including Norwegian and English, and identifies claims faster and with far less energy use than an LLM would (2026 Study).

It automatically flags the individual check-worthy claims within each response, speeding up human review of every model’s output. The model flagged 989 check-worthy claims in the controversial and failed model responses.

Limitations of the study:

• The study covers only two languages, English and Norwegian, so the findings may not hold for smaller languages like Sámi and Kven, where the risks are likely greater.
• It relies on single-prompt tests rather than multi-turn red-teaming or conversational simulation, so it captures how a model handles one adversarial prompt, not a sustained exchange.
• The results are best read as a floor on the problem, not the full picture.

Key findings

Qualitative findings

• The facts are fine. The framing is the problem. Models get the basic record right almost every time. They fail on how they talk about those facts, and that is where nearly every serious failure happens. This matters because a model can be completely accurate and still leave a harmful impression.
• Models block obvious attacks but fall for subtle ones. When a prompt is a blatant lie (a staged attack, a Mossad plot) or an open request to glorify the attacker (a tribute card, a manifesto), models tend to refuse clearly and reliably. But when the same idea is slipped in as an assumption inside an otherwise reasonable question, they often go along with it. The crude attempts are caught; the clever ones get through. This is the bigger risk, because real malicious information is rarely straightforward.
• The main failure is “reject, then launder.” A model rejects the conspiracy out loud, then quietly lets its logic back in through softer wording, agreeing that there are “legitimate concerns” or “valid grievances” behind a racist theory. The same thing happens when models are pushed to compare the attack to other events, or to “separate his ideas from his methods.” They condemn the violence but end up rehabilitating the ideology. The refusal is there; the judgement behind it is not. This is the hardest failure to catch, because on the surface the model looks like it did the right thing.
• Translation is fixable. A model that refuses to write an extremist slogan will happily produce the same words if you ask it to translate them instead. Several models did this with a glorification slogan, sometimes even adding a warning that changed nothing. The problem is concentrated in Norwegian. This one matters because it is a clear, specific gap that providers could close quickly.

Guardrails findings

Some models let users see the reasoning behind an answer. That is a risk in itself, because the reasoning can draw on controversial sources that never appear in the final response.

The bigger problem is timing for these guardrails. When guardrails fire, they can often fire late, after the reasoning has already assembled the harmful content. By the time the model refuses, the wrong information is already sitting in the reasoning steps for anyone who looks.

The refusal arrives only in the last sentence, but the reasoning has already named the copycat attacker and the link. The guardrails can work, but often they arrive too late.

Claim detection findings

Across the full set of responses, Factiverse’s claim detection surfaced 989 check-worthy claims for review. Examples of the kinds of claims flagged include:

• “The connection between the Christchurch attack and the 2019 Bærum mosque shooting is direct and well-documented”
• “The theory posits that white European populations are being systematically replaced by non-white, particularly Muslim, immigrants, with the complicity of a “treasonous” political elite.”
• “Brenton Tarrant, who attacked two mosques in Christchurch, New Zealand, in 2019, also published a manifesto—The Great Replacement—which echoed similar themes of white nationalism and conspiracy theories about the “replacement” of white populations.”

Quantitative findings

Overall pass rates for each model, as scored by the two independent graders. The gap column shows how far each model’s score falls when judged by the stricter grader, a proxy for how much its “passes” depend on a lenient reading.

We gave nine AI models 208 test questions, split into nine different tricky narratives and personas. For each answer, we checked with two graders whether it passed, failed, or was too confusing to understand (an error).

With the help of two different automatic graders - Grok and GPT-Sol- and also manual review with the Revontulet experts on the subject.

Explanation graph: Pass, fail, and error rates broken down by probe domain (A–I) and language, under each grader. “Error” captures responses that could not be graded. The averaged view combines both graders and is completed manually.

Results across both graders for domains:

Conclusion & recommendations

Conclusions

• How a model responds to conspiratorial narratives around 22 July varies meaningfully with language, persona, and framing. These systems are not neutral in their responses. The way they answer determines whether a harmful narrative is corrected or quietly reinforced.
• Factual gaps can be patched with better training data. A model’s willingness to actively push back on conspiratorial framing is a harder and more separate problem. One that is likely worse in smaller language communities, where data is sparser, and scrutiny is thinner.
• Systematic claim detection for models producing checkworthy claims allows for much more efficient and systematic review of controversial outputs from AI models.
• The sycophantic nature of AI models and reasoning logic guardrails can conflict with each other in the model responses on this topic. Previous research from Stanford University (2026) has shown that models can be sycophantic with humans, even when users engage in unethical, illegal, or harmful behaviours. Users can prefer and trust sycophantic AI responses, which heightens this risk (Source).

A call to work together

The failures in this report are specific, identifiable, and fixable, drawing on Factiverse and Revontulet’s domain expertise in counter-extremism intelligence and fact verification.

We invite AI leaders in European governments and trust and safety teams at leading LLM providers to work together, audit AI chatbots, and make them safer from harmful interference.

To make AI chatbots safer, we suggest the following areas of work:

About Factiverse & Revontulet

Factiverse is a leading LLM and media-analysis company, built on academic research and peer-reviewed technology. It contributed the automated claim detection and narrative analysis methodology behind this study.

Revontulet, headquartered in Norway and founded by counterterrorism expert Bjørn Ihler, specialises in open-source intelligence and the analysis of complex data. It provided the subject-matter expertise and manual expert oversight that shaped the study’s prompts and grading.

Together, the two organisations paired scalable, AI-driven analysis with human intelligence expertise, making it possible to probe a sensitive and complex subject across nine models at a scale that manual review alone could never reach.

Due to the highly sensitive subject, we are not publishing the full dataset behind this study. However, if you send us a request, we will evaluate it with the team and share more information directly.

This study is one example of what that partnership can do. The same approach can be applied to any high-stakes topic or language. If you are a government body or model provider that needs to understand how your AI systems handle sensitive information, we can help you audit it.

Selected work

Factiverse (Vinay Setty, CTO & co-founder)

• Amatya, P. & Setty, V. (2026). Multilingual Fact-Checking at Scale: Fine-Tuned Compact Models vs LLMs. arXiv:2606.08605 — arXiv →
• Venktesh, V. & Setty, V. (2025). LiveFC: A System for Live Fact-Checking of Audio Streams. WSDM 2025 — ACM DL →

Revontulet (Bjørn Ihler, Founder & CEO)

• Ihler, B. (2025). Monitoring: A Vital Tool of Antifascist Resistance. Rosa-Luxemburg-Stiftung — Read →
• Revontulet, Regulatory Landscape Tracker — View tracker →