How catastrophic is your LLM?

Amazon Science and University of Illinois researchers introduced C3LLM, a framework for evaluating catastrophic risks in LLMs during multi-turn conversations. Unlike traditional red-teaming that assesses isolated prompts, C3LLM uses graph-based modeling of conversational flows to generate statistical probability bounds on attack success rates. Testing on frontier models revealed significant safety variance, with DeepSeek-R1 showing 70%+ certified risk in cybercrime scenarios, while Claude-Sonnet-4 and Nova Premier demonstrated lower risk levels.