Generative AI systems are made up of multiple components that interact to provide a rich user experience between the human and the AI model(s).
As part of a responsible AI approach, AI models are protected by layers of defense mechanisms to prevent the production of harmful content or being used to carry out instructions that go against the intended purpose of the AI integrated application. This blog will provide an understanding of what AI jailbreaks are, why generative AI is susceptible to them, and how you can mitigate the risks and harms.
Read more…
Source: Microsoft
Related:
- Carnegie Mellon researchers show how LLMs can be taught to autonomously plan and execute real-world cyberattacks
July 24, 2025
In a groundbreaking development, a team of Carnegie Mellon University researchers has demonstrated that large language models (LLMs) are capable of autonomously planning and executing complex network attacks, shedding light on emerging capabilities of foundation models and their implications for cybersecurity research. The project, led by Ph.D. candidate Brian SingerOpens in new window, a Ph.D. candidate ...
- Proactive Email Security: The Power of AI
July 24, 2025
Cybercriminals are using AI to launch faster, more targeted attacks—impersonating executives, bypassing filters with QR phishing or AI-driven deception techniques, and exploiting human error to cause financial and reputational damage. Traditional defenses can’t keep up. This report explores how AI-powered email security can proactively defend against today’s most pressing threats—like business email compromise (BEC), QR phishing ...
- Preventing Zero-Click AI Threats: Insights from EchoLeak
July 15, 2025
EchoLeak (CVE-2025-32711) is a newly identified vulnerability in Microsoft 365 Copilot, made more nefarious by its zero-click nature, meaning it requires no user interaction to succeed. It demonstrates how helpful systems can open the door to entirely new forms of attack— no malware, no phishing required—just the unquestioning obedience of an AI agent. This new threat ...
- Impostor uses AI to impersonate Rubio and contact foreign and US officials
July 8, 2025
The State Department is warning U.S. diplomats of attempts to impersonate Secretary of State Marco Rubio and possibly other officials using technology driven by artificial intelligence, according to two senior officials and a cable sent last week to all embassies and consulates. The warning came after the department discovered that an impostor posing as Rubio had ...
- AI Goes on Offense: How LLMs Are Redefining the Cybercrime Landscape
June 26, 2025
In their last blog, Rapid7 explored the broader rise of AI-enabled threats across ransomware, phishing, and nation-state operations. Now, they’re narrowing in on a specific piece of that evolution: how cybercriminals are using large language models to scale and automate their tactics. AI in cybersecurity is no longer experimental. It’s embedded in workflows, transforming everything from ...
- Jailbroken AIs are helping cybercriminals to hone their craft
June 26, 2025
Cybercriminals are bypassing the guardrails that are supposed to keep AI models from carrying out criminal activities, according to researchers. We’ve seen the misuse of AI models by cybercriminals growing rapidly over the past several years, shaping a new era of digital threats. Early on, attackers focused on jailbreaking public AI chatbots, which meant they used ...