Large Language Models (LLMs) have exhibited remarkable reasoning capabilities, achieving impressive results across a wide range of tasks. Despite these advances, significant reasoning failures persist, occurring even in seemingly simple scenarios. To systematically understand and address these shortcomings, the authors of the paper present the first comprehensive survey dedicated to reasoning failures in LLMs.
The authors introduce a novel categorization framework that distinguishes reasoning into embodied and non-embodied types, with the latter further subdivided into informal (intuitive) and formal (logical) reasoning. In parallel, the authors classify reasoning failures along a complementary axis into three types: fundamental failures intrinsic to LLM architectures that broadly affect downstream tasks; application-specific limitations that manifest in particular domains; and robustness issues characterized by inconsistent performance across minor variations. For each reasoning failure, the authors provide a clear definition, analyze existing studies, explore root causes, and present mitigation strategies.
Read more…
Source: ARXIV, Cornell University
Sign up for the Cyber Security Review Newsletter
The latest cyber security news and insights delivered right to your inbox
Related:
- Jailbroken AIs are helping cybercriminals to hone their craft
June 26, 2025
Cybercriminals are bypassing the guardrails that are supposed to keep AI models from carrying out criminal activities, according to researchers. We’ve seen the misuse of AI models by cybercriminals growing rapidly over the past several years, shaping a new era of digital threats. Early on, attackers focused on jailbreaking public AI chatbots, which meant they used ...
- UK Police use controversial AI tool that looks at people’s sex lives and beliefs
June 16, 2025
British police forces have signed contracts with a controversial US tech giant to buy AI-powered software that uses data about an individual’s race, sex life, health and political beliefs, it can be revealed. An internal police memo obtained by The i Paper and Liberty Investigates confirms an intention to “nationally” apply the “Nectar” intelligence system, currently ...
- Toxic trend: Another malware threat targets DeepSeek
June 11, 2025
DeepSeek-R1 is one of the most popular LLMs right now. Users of all experience levels look for chatbot websites on search engines, and threat actors have started abusing the popularity of LLMs. kaspersky previously reported attacks with malware being spread under the guise of DeepSeek to attract victims. The malicious domains spread through X posts and ...
- 5 Things Security Leaders Need to Know About Agentic AI
June 9, 2025
From writing assistance to intelligent summarization, generative AI has already transformed the way businesses work. But we’re now entering a new phase where AI doesn’t just generate content, but takes independent action on our behalf. This next evolution is called ‘agentic AI’, and it’s moving fast. Amazon recently announced a dedicated R&D group focused on agentic ...
- Ransomware hiding in fake AI, business tools
June 5, 2025
Artificial intelligence (AI) and small business tools are being abused as smokescreens to hit unsuspecting victims with ransomware. In the masquerade campaigns discovered by Cisco Talos, cybercriminals hid malware behind software and install packages that mimicked the websites or names of the lead monetization service Nova Leads, the enormously popular Chat GPT, and an AI-empowered video ...
- Pakistan among least affected by web threats
May 25, 2025
At its annual Cyber Security Weekend for the Middle East, Turkiye and Africa (META) region, the Kaspersky Global Research and Analysis Team presented cybersecurity trends, including ransomware, advanced persistent threats (APTs), supply chain attacks, mobile threats, artificial intelligence and IoT developments. The first quarter of 2025 showed that Turkiye and Kenya had the highest number of ...