Large Language Models (LLMs) have exhibited remarkable reasoning capabilities, achieving impressive results across a wide range of tasks. Despite these advances, significant reasoning failures persist, occurring even in seemingly simple scenarios. To systematically understand and address these shortcomings, the authors of the paper present the first comprehensive survey dedicated to reasoning failures in LLMs.
The authors introduce a novel categorization framework that distinguishes reasoning into embodied and non-embodied types, with the latter further subdivided into informal (intuitive) and formal (logical) reasoning. In parallel, the authors classify reasoning failures along a complementary axis into three types: fundamental failures intrinsic to LLM architectures that broadly affect downstream tasks; application-specific limitations that manifest in particular domains; and robustness issues characterized by inconsistent performance across minor variations. For each reasoning failure, the authors provide a clear definition, analyze existing studies, explore root causes, and present mitigation strategies.
Read more…
Source: ARXIV, Cornell University
Sign up for the Cyber Security Review Newsletter
The latest cyber security news and insights delivered right to your inbox
Related:
- Indirect prompt injection in the real world: how people manipulate neural networks
August 12, 2024
Large language models (LLMs) – the neural network algorithms that underpin ChatGPT and other popular chatbots – are becoming ever more powerful and inexpensive. Systems built on instruction-executing LLMs may be vulnerable to prompt injection attacks. A prompt is a text description of a task that the system is to perform, for example: “You are a ...
- Elon Musk’s X accused of AI data grab in ‘blatant breach of law’
July 29, 2024
Privacy organisation Open Rights Group has branded a move by social network X (formerly Twitter) to use user data to train its Grok AI as a “blatant breach of GDPR”. The GDPR (General Data Protection Regulation) is an EU law protecting people’s data, and is implemented in the UK as part of the Data Protection Act ...
- CrowdStrike Took Down Australia And Half The World Now Facing Massive Compensation Claims
July 19, 2024
The reputation of a Company that describes themselves as one of the world’s best cyber security Companies is in tatters tonight, with the US business facing the potential of being sued by hundreds of business including major retailers in Australia and insurance Companies looking to claw back payouts for lost income, airline delays and customers ...
- AI, cyber-attacks and amateur experiments threaten to upend global biosecurity, WHO warns
July 13, 2024
Artificial intelligence, cyber-attacks and genetic engineering could pose potentially catastrophic biosecurity threats to countries around the world, the WHO has warned. Rapid technological advances in the past decade have “redefined the biological threat landscape” and heightened risks of manipulation, the updated guidance from the WHO’s Technical Advisory Group on Biosafety said. The report advised that member ...
- NATO releases revised AI strategy
July 10, 2024
On Wednesday (10 July 2024), NATO released its revised artificial intelligence (AI) strategy, which aims to accelerate the use of AI technologies within NATO in a safe and responsible way. AI or Artificial intelligence concept. It builds on one published in 2021 and takes account of recent advances in AI technologies, such as generative AI, and ...
- OpenAI breach is a reminder that AI companies are treasure troves for hackers
July 5, 2024
There’s no need to worry that your secret ChatGPT conversations were obtained in a recently reported breach of OpenAI’s systems. The hack itself, while troubling, appears to have been superficial — but it’s reminder that AI companies have in short order made themselves into one of the juiciest targets out there for hackers. The New York ...