Large Language Models (LLMs) have exhibited remarkable reasoning capabilities, achieving impressive results across a wide range of tasks. Despite these advances, significant reasoning failures persist, occurring even in seemingly simple scenarios. To systematically understand and address these shortcomings, the authors of the paper present the first comprehensive survey dedicated to reasoning failures in LLMs.
The authors introduce a novel categorization framework that distinguishes reasoning into embodied and non-embodied types, with the latter further subdivided into informal (intuitive) and formal (logical) reasoning. In parallel, the authors classify reasoning failures along a complementary axis into three types: fundamental failures intrinsic to LLM architectures that broadly affect downstream tasks; application-specific limitations that manifest in particular domains; and robustness issues characterized by inconsistent performance across minor variations. For each reasoning failure, the authors provide a clear definition, analyze existing studies, explore root causes, and present mitigation strategies.
Read more…
Source: ARXIV, Cornell University
Sign up for the Cyber Security Review Newsletter
The latest cyber security news and insights delivered right to your inbox
Related:
- Vibe Hacking: Two AI-Augmented Campaigns Target Government and Financial Sectors in Latin America
May 11, 2026
Threat actors using AI is an unsurprising and even long-predicted developmentopen on a new tab. In a case in point, TrendAI™ Research has identified two emerging threat campaigns that used agentic AI to drive intrusion operations against government entities and financial organizations across several countries in Latin America. Though evidence suggests that the two groups are likely ...
- Adversaries Leverage AI for Vulnerability Exploitation, Augmented Operations, and Initial Access
May 11, 2026
Since our February 2026 report on AI-related threat activity, Google Threat Intelligence Group (GTIG) has continued to track a maturing transition from nascent AI-enabled operations to the industrial-scale application of generative models within adversarial workflows. This report, based on insights derived from Mandiant incident response engagements, Gemini, and GTIG’s proactive research, highlights the dual nature ...
- Study warns cost-cutting use of generative AI could increase cyber-attack risks
May 3, 2026
Newly published research from a leading computer scientist warns that the use of generative AI to design, train, or perform steps within a machine learning system could increase serious risks. Michael Lones, professor at Heriot-Watt University’s School of Mathematical and Computer Sciences, has argued in a new paper that generative AI could expose organizations and ...
- How ChatGPT conversations became ‘a treasure trove’ of evidence in criminal investigations
May 2, 2026
Days before two University of South Florida graduate students went missing last month, a roommate of one of the students allegedly asked the AI chatbot ChatGPT an unusual question. “What happens if a human has a put (sic) in a black garbage bag and thrown in a dumpster,” Hisham Abugharbieh asked on April 13, according to ...
- ASD: Careful Adoption of Agentic AI Services
May 1, 2026
Agentic artificial intelligence (AI) systems increasingly operate across critical infrastructure and defence sectors and support mission-critical capabilities. As agentic AI systems play a growing operational role, it is crucial for defenders to implement security controls to protect national security and critical infrastructure from agentic AI-specific risks. Agentic AI can automate repetitive, well-defined and low-risk tasks. However, ...
- Mythos: An AI tool too powerful for public release
April 20, 2026
Anthropic’s most capable model to date, Claude Mythos Preview (aka Mythos), has been described as a “step change” in AI performance, especially on cybersecurity tasks. Anthropic tried to keep Mythos a secret until a few weeks ago, when a data leak revealed the existence of what the company said was its most powerful artificial intelligence to ...