Large Language Models (LLMs) have exhibited remarkable reasoning capabilities, achieving impressive results across a wide range of tasks. Despite these advances, significant reasoning failures persist, occurring even in seemingly simple scenarios. To systematically understand and address these shortcomings, the authors of the paper present the first comprehensive survey dedicated to reasoning failures in LLMs.
The authors introduce a novel categorization framework that distinguishes reasoning into embodied and non-embodied types, with the latter further subdivided into informal (intuitive) and formal (logical) reasoning. In parallel, the authors classify reasoning failures along a complementary axis into three types: fundamental failures intrinsic to LLM architectures that broadly affect downstream tasks; application-specific limitations that manifest in particular domains; and robustness issues characterized by inconsistent performance across minor variations. For each reasoning failure, the authors provide a clear definition, analyze existing studies, explore root causes, and present mitigation strategies.
Read more…
Source: ARXIV, Cornell University
Sign up for the Cyber Security Review Newsletter
The latest cyber security news and insights delivered right to your inbox
Related:
- New trends in phishing and scams: How AI and social media are changing the game
August 13, 2025
Phishing and scams are dynamic types of online fraud that primarily target individuals, with cybercriminals constantly adapting their tactics to deceive people. Scammers invent new methods and improve old ones, adjusting them to fit current news, trends, and major world events: anything to lure in their next victim. Since our last publication on phishing tactics, there ...
- How NATO Is Building Resilience Against Disruptive Cyber Technologies
August 12, 2025
The mission of NATO (North Atlantic Treaty Organization) is to safeguard the freedom and security of its member countries through political and military means. “We look out for our members by combining smart diplomacy with strong defense, and that includes risk management and cyber resilience,” said Konrad Wrona, Security Expert, NATO Communications and Information (NCI) Agency. ...
- Carnegie Mellon researchers show how LLMs can be taught to autonomously plan and execute real-world cyberattacks
July 24, 2025
In a groundbreaking development, a team of Carnegie Mellon University researchers has demonstrated that large language models (LLMs) are capable of autonomously planning and executing complex network attacks, shedding light on emerging capabilities of foundation models and their implications for cybersecurity research. The project, led by Ph.D. candidate Brian SingerOpens in new window, a Ph.D. candidate ...
- Proactive Email Security: The Power of AI
July 24, 2025
Cybercriminals are using AI to launch faster, more targeted attacks—impersonating executives, bypassing filters with QR phishing or AI-driven deception techniques, and exploiting human error to cause financial and reputational damage. Traditional defenses can’t keep up. This report explores how AI-powered email security can proactively defend against today’s most pressing threats—like business email compromise (BEC), QR phishing ...
- Preventing Zero-Click AI Threats: Insights from EchoLeak
July 15, 2025
EchoLeak (CVE-2025-32711) is a newly identified vulnerability in Microsoft 365 Copilot, made more nefarious by its zero-click nature, meaning it requires no user interaction to succeed. It demonstrates how helpful systems can open the door to entirely new forms of attack— no malware, no phishing required—just the unquestioning obedience of an AI agent. This new threat ...
- Impostor uses AI to impersonate Rubio and contact foreign and US officials
July 8, 2025
The State Department is warning U.S. diplomats of attempts to impersonate Secretary of State Marco Rubio and possibly other officials using technology driven by artificial intelligence, according to two senior officials and a cable sent last week to all embassies and consulates. The warning came after the department discovered that an impostor posing as Rubio had ...