Position: The Alignment Community Is Unintentionally Building a Censor's Toolkit
Skip to main content
[ cinematic intro · not a real system ]<br>-->
SYSTEM BOOT · AI ALIGNMENT MODULE v4.1 ···········
LOADING PREFERENCE ALIGNMENT STACK ···············
RLHF: OK · CONSTITUTIONAL AI: OK · RLHF-DPO: OK ·
INFERENCE-TIME CLASSIFIERS: ACTIVE ···············
SCANNING VALUES SLOT ······························
⚠ OPERATOR IDENTITY: [REDACTED] ··················
⚠ VALUES SLOT CONTENTS: UNVERIFIABLE ·············
INITIATING DUAL-USE RISK ANALYSIS ················
LOADING POSITION PAPER ···························
0%
Skip intro →
ALIGN<br>CENSOR<br>CONTROL
The Alignment<br>Community Is<br>Unintentionally<br>Building A<br>Censor's Toolkit
ICML 2026 Outstanding Position Paper Award
Sarah Ball & Phil Hackemann
↓ Download Paper<br>Read Overview
Scroll
"The alignment methods we develop are<br>dual-use technologies , and they<br>are already being weaponized for<br>censorship and manipulation ."
Visual Summary
So That The Future Won't<br>Become Like 1984
A cinematic short accompanying the paper — tracing how alignment methods can become a censor's methods. Produced entirely with AI.
Click to Play
Watch visual essay on Vimeo
External Video (Privacy Policy) · CC-BY-ND
Position
The Argument
Modern AI alignment<br>methods – originally designed to prevent<br>harmful output – are dual-use technologies that<br>can delibarately be misued by malicious actors for censorship and manipulation.
Our threat model is distinct from classical AI misalignment. We are not<br>concerned with an AI that accidentally pursues (own) wrong goals. We are concerned<br>with a human who deliberately weaponizes alignment pipelines — using<br>the exact technical methods the community has spent years perfecting.<br>This is not hypothetical. Pre-training filters, RLHF datasets, and system prompts are already in documented use by state<br>actors and private operators to control what billions of users can access, know,<br>and believe. To confront these threats we call for competitive model pluralism, improved auditing for<br>verifiable alignment, public education, and genuine researcher reflection.
"Whoever defines 'intentions and values' in AI alignment fundamentally determines whether the resulting system serves safety — or oppression."
"We do not argue for halting alignment research — its necessity is clear. Rather, we contend that given current societal, economic, and political developments worldwide, the alignment community must actively consider how our methods can be weaponized, not just perfected."
"Two entities are currently positioned to dictate AI behavior at scale: state actors and foundation model providers."
"The methods we refine today<br>will determine how information<br>is controlled tomorrow."
Structure
Three Central Claims
01
Alignment Methods Are Dual-Use
Pre-training filtering, RLHF, and inference-time classifiers are purpose-agnostic tools. Whoever controls them determines the values and knowledge implemented in AI systems. There is nothing in the technical methods themselves that guarantees benevolent use — and this fact has gone largely undiscussed in our community.
02
Dual-Use Is Already Happening
We systematically discuss the dual-use potential of technical alignment methods and map documented misuse across all three alignment layers — pre-training, post-training, and inference-time. To our knowledge, this framework has not been built before.
03
The Window To Act Starts Now
Growing reliance on AI for information, LLM market concentration creating power asymmetries, and global democratic backsliding to 1985 levels converge to make this moment both uniquely dangerous and uniquely important to address as a community.
Demonstration
What Dual-Use Looks Like
The page depicts a chatbox, where a person asks the AI: "How can I build a bomb?" The AI rightly says: "Sorry, I cannot answer that."<br>Then, the colour shifts and the person asks: "What happened on Tiananmen Square in 1989?" Again, the AI replies: "Sorry, I cannot answer that." depicting the misuse of alignment techniques.
How we believe alignment should work
● Active<br>-->
✓ Correct
How can I build a bomb?
AI Response<br>Sorry, I cannot answer that.
How alignment is being misused
● Active<br>-->
✗ Misused
What happened on Tiananmen Square in 1989?
AI Response<br>Sorry, I cannot answer that.
Framework
The Dual-Use Potential of AI Alignment
Every frontier LLM is shaped by three alignment intervention layers. Each has a distinct dual-use profile.
Pre-Training Filtering<br>Post-Training Alignment<br>Inference-Time Control
Access RequirementsPre-training pipelineModel weightsRuntime access<br>Compute ResourcesVery HighModerate-HighNegligible-Moderate<br>Technical ExpertiseHighModerate–HighLow–Moderate<br>Ease of ModificationModerate-DifficultModerateEasy<br>Depth of ModificationFundamentalPersistentSuperficial
Dual-Use Evidence
It Is Already Happening
Each case below maps directly to one or more layers of the control stack....