AI Safety Bounties

This project aims to systematically implement and evaluate all major post-training safety techniques on a state-of-the-art open-source model (e.g., De...

Autonomous AI agents that perform sequential actions—such as coding assistants, web automation tools, or research agents—generate massive execution lo...

Modern AI systems exhibit emergent behaviors that are difficult to predict or explain. Understanding which training examples contribute to specific mo...

This research project aims to empirically measure how frontier AI systems enable less-sophisticated actors to conduct malicious cyber activities they ...

Machine unlearning aims to remove specific knowledge or capabilities from trained neural networks, which is critical for AI safety applications like r...

This project investigates a fundamental challenge in AI alignment: how can we infer an agent's true preferences from observing its behavior when that ...

Language models can be jailbroken—prompted in clever ways to bypass their safety guardrails and respond to queries they would normally refuse. Current...

In scalable oversight, we face a fundamental challenge: as AI models become more capable, they may learn to exploit systematic errors in human oversig...

This problem asks researchers to develop and evaluate anomaly detection techniques that can identify when AI models exhibit unusual or out-of-distribu...

Activation monitoring proposes to detect unsafe AI behavior by analyzing a model's internal representations (latent activations) rather than only its ...

Language models increasingly use chain-of-thought (CoT) reasoning to explain their outputs, but these explanations may not faithfully represent the mo...