Generative AI systems are made up of multiple components that interact to provide a rich user experience between the human and the AI model(s).
As part of a responsible AI approach, AI models are protected by layers of defense mechanisms to prevent the production of harmful content or being used to carry out instructions that go against the intended purpose of the AI integrated application. This blog will provide an understanding of what AI jailbreaks are, why generative AI is susceptible to them, and how you can mitigate the risks and harms.
Read more…
Source: Microsoft
Related:
- Adversaries Leverage AI for Vulnerability Exploitation, Augmented Operations, and Initial Access
May 11, 2026
Since our February 2026 report on AI-related threat activity, Google Threat Intelligence Group (GTIG) has continued to track a maturing transition from nascent AI-enabled operations to the industrial-scale application of generative models within adversarial workflows. This report, based on insights derived from Mandiant incident response engagements, Gemini, and GTIG’s proactive research, highlights the dual nature ...
- Study warns cost-cutting use of generative AI could increase cyber-attack risks
May 3, 2026
Newly published research from a leading computer scientist warns that the use of generative AI to design, train, or perform steps within a machine learning system could increase serious risks. Michael Lones, professor at Heriot-Watt University’s School of Mathematical and Computer Sciences, has argued in a new paper that generative AI could expose organizations and ...
- How ChatGPT conversations became ‘a treasure trove’ of evidence in criminal investigations
May 2, 2026
Days before two University of South Florida graduate students went missing last month, a roommate of one of the students allegedly asked the AI chatbot ChatGPT an unusual question. “What happens if a human has a put (sic) in a black garbage bag and thrown in a dumpster,” Hisham Abugharbieh asked on April 13, according to ...
- ASD: Careful Adoption of Agentic AI Services
May 1, 2026
Agentic artificial intelligence (AI) systems increasingly operate across critical infrastructure and defence sectors and support mission-critical capabilities. As agentic AI systems play a growing operational role, it is crucial for defenders to implement security controls to protect national security and critical infrastructure from agentic AI-specific risks. Agentic AI can automate repetitive, well-defined and low-risk tasks. However, ...
- Mythos: An AI tool too powerful for public release
April 20, 2026
Anthropic’s most capable model to date, Claude Mythos Preview (aka Mythos), has been described as a “step change” in AI performance, especially on cybersecurity tasks. Anthropic tried to keep Mythos a secret until a few weeks ago, when a data leak revealed the existence of what the company said was its most powerful artificial intelligence to ...
- Suspect arrested after incendiary device thrown at OpenAI CEO Sam Altman’s home
April 11, 2026
A 20-year-old man has been arrested after a Molotov cocktail was thrown at the San Francisco home of OpenAI CEO Sam Altman early Friday morning. The incident happened around 4:00 am when a suspect “threw an incendiary destructive device” at Altman’s home, “causing a fire to one exterior gate” before fleeing on foot, according to statement ...