Large Language Models (LLMs) have exhibited remarkable reasoning capabilities, achieving impressive results across a wide range of tasks. Despite these advances, significant reasoning failures persist, occurring even in seemingly simple scenarios. To systematically understand and address these shortcomings, the authors of the paper present the first comprehensive survey dedicated to reasoning failures in LLMs.
The authors introduce a novel categorization framework that distinguishes reasoning into embodied and non-embodied types, with the latter further subdivided into informal (intuitive) and formal (logical) reasoning. In parallel, the authors classify reasoning failures along a complementary axis into three types: fundamental failures intrinsic to LLM architectures that broadly affect downstream tasks; application-specific limitations that manifest in particular domains; and robustness issues characterized by inconsistent performance across minor variations. For each reasoning failure, the authors provide a clear definition, analyze existing studies, explore root causes, and present mitigation strategies.
Read more…
Source: ARXIV, Cornell University
Sign up for the Cyber Security Review Newsletter
The latest cyber security news and insights delivered right to your inbox
Related:
- Link Trap: GenAI Prompt Injection Attack
December 17, 2024
With the rise of generative AI, new security vulnerabilities are emerging. One such vulnerability is prompt injection, a method that malicious actors can exploit to manipulate AI systems. Typically, the impact of prompt injection attacks is closely tied to the permissions granted to the AI. However, the attack discussed in this article differs from commonly known ...
- What is high bandwidth memory and why is the US trying to block China’s access to it?
December 8, 2024
The US government has imposed fresh export controls on the sale of high tech memory chips used in artificial intelligence (AI) applications to China. The rules apply to US-made high bandwidth memory (HBM) technology as well as foreign-produced ones. High bandwidth memory (HBM) are basically a stack of memory chips, small components that store data. They ...
- AI chatbot provider exposes 346,000 customer files, including ID documents, resumes, and medical records
December 3, 2024
Researchers have discovered a huge Google Cloud Storage bucket, found freely accessible on the internet and containing a treasure trove of personal information. AI startup WotNot provides companies with the ability to create their own customized chatbot. The company reportedly has 3,000 customers including some household family names. But the way its solution is set up ...
- INTERPOL campaign warns against cyber and financial crimes
December 3, 2024
INTERPOL has launched a campaign to raise awareness on the growing threat of cyber and financial crimes against vulnerable individuals and organizations. The Think Twice campaign, which includes a series of short videos, focuses on five rising online threats: ransomware attacks, malware attacks, phishing, generative AI scams, and romance baiting. These sophisticated scams have seen a ...
- Singapore cyber defenders fight simulated attacks on AI-enabled systems in 4-day exercise
November 15, 2024
More technology is moving onto the cloud – meaning its data is hosted on remote servers rather than on personal devices – and integrating artificial intelligence (AI), which opens it up to new kinds of malicious attacks. To improve Singapore’s ability to counter these emerging threats, soldiers from the Singapore Armed Forces (SAF) and civilians from ...
- ModeLeak: Privilege Escalation to LLM Model Exfiltration in Vertex AI
November 12, 2024
In the race to gain a competitive edge, organizations are increasingly training artificial intelligence (AI) models on sensitive data. But what if a seemingly harmless AI model became a gateway for attackers? A malicious actor could upload a poisoned model to a public repository, and without realizing it, your team could deploy it in your environment. ...