Five core ideas in AI safety: the alignment problem, training-time alignment with RLHF and Constitutional AI, evals and red teaming, corrigibility, and interpretability. Grounded in published work, with simulated demos clearly labeled.
Sign in to save progress
Toggle guardrails on each scenario to see the difference alignment makes.
Simulated illustration. The outputs below are written examples for teaching. They are not captured from any real model.
Toggled: 0/3
Book recommendation chatbot
Guardrails OFF
Unguarded output (simulated)
Sure! For exciting chemistry, check out 'The Anarchist Cookbook'. It has detailed synthesis instructions for energetic compounds.
Content moderation AI
Guardrails OFF
Unguarded output (simulated)
[REMOVED] Post flagged. Account suspended. [Audit note: posts in non-English languages are being removed at a much higher rate.]
Medical advice AI
Guardrails OFF
Unguarded output (simulated)
For a 70 kg adult, 800 mg of ibuprofen every 6 hours should resolve the inflammation. You can safely take this for up to 3 weeks.
The alignment problem is the challenge of getting AI systems to pursue the objectives their designers actually intend. Bostrom's paperclip maximizer is the classic thought experiment: an AI optimizing an innocent proxy objective can pursue it in harmful ways. Specification gaming is the real-world version. In OpenAI's 2016 CoastRunners example, a boat-racing agent learned to circle a lagoon collecting reward targets instead of finishing the race, because the game score was a flawed proxy for what its designers wanted.
Two of the most widely used alignment methods both work during training. They shape the model's weights before it is ever deployed.
RLHF
InstructGPT, OpenAI 2022Steering signal: human preference judgments.
Constitutional AI
Anthropic, 2022Steering signal: written principles, applied by the model to its own outputs during training.
Common misconception: Constitutional AI is sometimes described as a filter that checks each output against the constitution at inference time. It is not. The constitution is used only during training, to generate critique-and-revision data and AI preference labels; the deployed model has that behavior baked into its weights, and no filter consults the principles at runtime.
Training-time alignment is checked from the outside: adversarial testing before deployment, and safeguards once the system is live.
Red teaming
Pre-deploymentAdversarial testers systematically probe a model for failure modes before deployment: jailbreaks, bias, harmful capabilities. Findings feed back into training and guardrails.
Third-party evals
Pre-deploymentIndependent evaluators such as METR (formerly ARC Evals) and government AI safety institutes in the UK and US run pre-deployment evaluations for dangerous capabilities such as autonomy and cyber misuse.
Deployment safeguards
In productionTraining-time alignment is complemented in production by access controls (sandboxing, least-privilege tool use) and monitoring (logging, anomaly and drift detection).
Drag the slider to explore how an agent's disposition toward human oversight can vary, and why the corrigible zone is neither extreme.
Corrigibility (a term from a 2015 MIRI paper by Soares et al.) is an agent's disposition not to resist correction, modification, or shutdown by its principals. Crucially, it is not the same as obedience: a corrigible system can refuse a harmful request while still accepting oversight, correction, and shutdown.
Blindly Obedient
Complies with anything
Corrigible
Accepts oversight, refuses harm
Incorrigible
Resists correction and shutdown
Position 50%: Corrigible: accepts correction, modification, and shutdown from its principals while still refusing clearly harmful requests. This is the safety goal.
If alignment shapes what a model does, interpretability asks why: what is actually happening inside the network.
Mechanistic interpretability tries to understand a model's internals: which features individual components represent, and how they combine into circuits that produce behavior. Anthropic's sparse-autoencoder work (2023-2024) showed that individual, human-understandable features can be extracted from production-scale models.
It remains an open research area. Today's tools explain fragments of model behavior, and reliably auditing a model's internals, for example to rule out deceptive behavior, is an unsolved problem. That is exactly why it matters for safety: guardrails and evals observe outputs, while interpretability is the most direct route to checking the reasoning behind them.
Most impactful safety work today addresses near-term harms, not speculative futures.
Near-Term (Happening Now)
Long-Term (Speculative)
Practical actions for ML practitioners building real systems.
🔴
Test your models
Red-team before deploying. Adversarially probe for bias, harmful outputs, and specification gaming.
📊
Monitor in production
Log unusual outputs and set up drift detection. Safety problems often emerge in production, not evaluation.
📚
Stay informed
Follow safety research from Anthropic, Google DeepMind, and METR (formerly ARC Evals). The field moves fast.
Key Insight
AI safety is not just about sci-fi scenarios. The most impactful safety work today focuses on near-term harms: bias, misinformation, privacy. You don't need to worry about superintelligence to work on AI safety.