This talk examines what can go wrong when AI models are secured using assumptions and controls that do not hold up in real deployments, creating gaps between perceived safety and actual behavior. Rather than cataloging attacks, the discussion focuses on why common AI security approaches fall short, including brittle safeguards and limited visibility once systems are in use. As AI systems become more agentic, small failures or exploitations can have outsized impact, allowing models to take actions, influence other systems, or compound mistakes at scale.

The talk closes by outlining how security teams can reason more clearly about AI risk today and what needs to change as these systems continue to evolve.