Large Language Models (LLMs) have exhibited remarkable reasoning capabilities, achieving impressive results across a wide range of tasks. Despite these advances, significant reasoning failures persist, occurring even in seemingly simple scenarios. To systematically understand and address these shortcomings, the authors of the paper present the first comprehensive survey dedicated to reasoning failures in LLMs.
The authors introduce a novel categorization framework that distinguishes reasoning into embodied and non-embodied types, with the latter further subdivided into informal (intuitive) and formal (logical) reasoning. In parallel, the authors classify reasoning failures along a complementary axis into three types: fundamental failures intrinsic to LLM architectures that broadly affect downstream tasks; application-specific limitations that manifest in particular domains; and robustness issues characterized by inconsistent performance across minor variations. For each reasoning failure, the authors provide a clear definition, analyze existing studies, explore root causes, and present mitigation strategies.
Read more…
Source: ARXIV, Cornell University
Sign up for the Cyber Security Review Newsletter
The latest cyber security news and insights delivered right to your inbox
Related:
- The US government is building an AI sandbox to tackle cybercrime
January 10, 2023
A joint effort between the Science and Technology Directorate (S&T) – housed within the Department of Homeland Security (DHS) – and the Cybersecurity and Infrastructure Security Agency (CISA), an AI sandbox will be designed for researchers to collaborate and test analytical approaches and techniques in combating cyber threats. CISA’s Advanced Analytics Platform for Machine Learning (CAP-M) ...
- AI cyber attacks are a ‘critical threat’. This is how NATO is countering them
December 26, 2022
Artificial intelligence (AI) is playing a massive role in cyber attacks and is proving both a “double-edged sword” and a “huge challenge,” according to NATO. “Artificial intelligence allows defenders to scan networks more automatically, and fend off attacks rather than doing it manually. But the other way around, of course, it’s the same game,” David van ...
- DARPA’s CASTLE to Fortify Computer Networks
October 24, 2022
An ever-expanding cyber-attack surface, infrequent computer vulnerability scans, and burdensome security procedures create a seemingly lopsided battle when it comes to defending critical computing assets. Couple those factors with costly cybersecurity assessments that often lack actionable feedback, and the odds may appear to favor bad actors. DARPA intends to change that dynamic through a new program ...
- Phishing works so well crims won’t bother with deepfakes, says Sophos chap
October 17, 2022
Panic over the risk of deepfake scams is completely overblown, according to a senior security adviser for UK-based infosec company Sophos. “The thing with deepfakes is that we aren’t seeing a lot of it,” Sophos researcher John Shier told El Reg last week. Shier said current deepfakes – AI generated videos that mimic humans – aren’t the ...
- UK privacy watchdog fines Clearview AI £7.5m and orders UK data to be deleted
May 24, 2022
The Information Commissioner’s Office (ICO) has fined controversial facial recognition company Clearview AI £7.5 million ($9.4 million) for breaching UK data protection laws and has issued an enforcement notice ordering the company to stop obtaining and using data of UK residents, and to delete the data from its systems. In its finding, the ICO detailed how ...
- Artificial Intelligence: How to make Machine Learning Cyber Secure?
December 14, 2021
Machine learning (ML) is currently the most developed and the most promising subfield of artificial intelligence for industrial and government infrastructures. By providing new opportunities to solve decision-making problems intelligently and automatically, artificial intelligence (AI) is applied in almost all sectors of our economy. While the benefits of AI are significant and undeniable, the development of ...