Large Language Models (LLMs) have exhibited remarkable reasoning capabilities, achieving impressive results across a wide range of tasks. Despite these advances, significant reasoning failures persist, occurring even in seemingly simple scenarios. To systematically understand and address these shortcomings, the authors of the paper present the first comprehensive survey dedicated to reasoning failures in LLMs.
The authors introduce a novel categorization framework that distinguishes reasoning into embodied and non-embodied types, with the latter further subdivided into informal (intuitive) and formal (logical) reasoning. In parallel, the authors classify reasoning failures along a complementary axis into three types: fundamental failures intrinsic to LLM architectures that broadly affect downstream tasks; application-specific limitations that manifest in particular domains; and robustness issues characterized by inconsistent performance across minor variations. For each reasoning failure, the authors provide a clear definition, analyze existing studies, explore root causes, and present mitigation strategies.
Read more…
Source: ARXIV, Cornell University
Sign up for the Cyber Security Review Newsletter
The latest cyber security news and insights delivered right to your inbox
Related:
- Threat Predictions for 2024: Chained AI and CaaS Operations Give Attackers More “Easy” Buttons Than Ever
November 9, 2023
With the growth of Cybercrime-as-a-Service (CaaS) operations and the advent of generative AI, threat actors have more “easy” buttons at their fingertips to assist with carrying out attacks than ever before. By relying on the growing capabilities in their respective toolboxes, adversaries will increase the sophistication of their activities. They’ll launch more targeted and stealthier hacks ...
- OpenAI Blames ChatGPT’s Intermittent Outages On ‘Abnormal Traffic’ That Suggests Potential Cyber Attack
November 9, 2023
ChatGPT continued to face intermittent outages late Wednesday, which the platform’s maker OpenAI blamed on a potential cyberattack, hours after the AI chatbot platform recovered from a wide outage that the company initially attributed to a surge in interest for its new features. Early on Thursday, OpenAI’s service status page displayed a notification saying both ChatGPT ...
- Tech firms to allow vetting of AI tools
November 3, 2023
The most advanced technology companies will allow governments to vet their artificial intelligence tools for the first time, Rishi Sunak has announced, as Elon Musk warned the technology could eventually replace all human jobs. Companies including Meta, Google DeepMind and OpenAI have agreed to allow regulators to test their latest AI products before releasing them ...
- Increasing transparency in AI security
October 26, 2023
New AI innovations and applications are reaching consumers and businesses on an almost-daily basis. Building AI securely is a paramount concern, and we believe that Google’s Secure AI Framework (SAIF) can help chart a path for creating AI applications that users can trust. Today, we’re highlighting two new ways to make information about AI supply ...
- Hackers trying to corrupt AI, raising level of ransomware threat
October 17, 2023
Cyber criminals are actively trying to corrupt generative artificial intelligence (AI), which may then put the ability to create ransomware in the hands of individuals. The looming threat is what keeps Mr Willis Lim, the director of the National Cyber Threat Analysis Centre at the Cyber Security Agency of Singapore (CSA), up at night. Generative ...
- China’s cyber security association sets up special committee to bolster AI research
October 15, 2023
China has set up a professional committee focusing on governance of artificial intelligence (AI) security in a bid to build a sustained foundation for the sound development of the emerging industry, according to the country’s cyber security association. On Thursday, an inaugural meeting was held in Beijing for the AI security governance committee under the ...