Full paper
Towards Robust Numerical Claim Verification
By Peter Røysland Aarnes and Vinay Setty
The problem
Automated fact-checking tools frequently rely on Large Language Models (LLMs) to verify relevant information claims. However, LLMs often struggle with accuracy when it comes to numerical claims. Even a slight change to statistics, percentages, or dates can easily trick them. For example, if a true claim contains "3.5%" and is altered to "six percent," standard models often fail to catch the error. This happens due to the fact that such models rely on matching general words rather than strictly checking numbers against source evidence.
The solution
To solve this vulnerability, in this study, researchers, including Factiverse CTO Vinay Setty, introduced adversarial fine-tuning using parameter-efficient techniques like QLoRA. By deliberately training smaller open-source AI models on statements with tweaked numerical values, the models learned to accurately detect when a number change flips a claim from true to false. The fine-tuned compact models achieved up to 98.7% accuracy on modified numerical claims.
The results
It is noteworthy that with the use of a fraction of the computing power these small models surpassed commercial frontier models—including GPT-5.4 Pro (74.0%) and Gemini 2.5 Flash (73.9%).
.avif)
