Automated Moderation Is Here to Stay—Accountability Must Keep Pace | Electronic Frontier Foundation

Skip to main content

AboutContact

Press

People

Opportunities

IssuesFree Speech

Privacy

Creativity and Innovation

Transparency

International

Security

Artificial Intelligence

Our WorkDeeplinks Blog

Press Releases

Events

Legal Cases

Whitepapers

Podcasts

Annual Reports

Take ActionAction Center

Volunteer

Follow EFF

ToolsPrivacy Badger

Surveillance Self-Defense

Certbot

Atlas of Surveillance

Cover Your Tracks

Street Level Surveillance

apkeep

Shop

DonateDonate to EFF

Shop

Giving Societies

Sponsorships

Other Ways to Give

Membership FAQ

Email updates on news, actions,

and events in your area.

Join EFF Lists

Copyright (CC BY)

Trademark

Privacy Policy

Thanks

Electronic Frontier Foundation

Donate

If you use technology, this fight is yours.Donate today

EFFecting Change: If You Own It, Why Can't You Fix It? on July 23

Automated Moderation Is Here to Stay—Accountability Must Keep Pace

DEEPLINKS BLOG

By Jillian C. York and Corynne McSherry<br>July 10, 2026

Automated Moderation Is Here to Stay—Accountability Must Keep Pace

Share It

Share on Mastodon<br>Share on Bluesky<br>Share on Facebook<br>Copy link

This post is part 2 in a series about automated content moderation. Read the first post here.

When whistleblower Frances Haugen leaked a set of documents from Meta in 2020, among the revelations was a jarring statistic: The company’s algorithms designed to detect terrorist content incorrectly deleted nonviolent Arabic-language content 77 percent of the time, while failing to detect hate speech under the company’s own policies in many instances. Meta’s own transparency report released later that year demonstrated similar findings. Five years later, researchers in the region report that overzealous moderation remains a problem, while paths to remedy have all but collapsed.

Where these systems are faltering in Arabic, they’re positively failing in less-resourced languages. As a 2025 report from the Center for Democracy and Technology found, labeled datasets in certain languages and dialects such as Maghrebi Arabic and Kiswahili contain inconsistencies, bias, and inaccuracies due to the limited hiring of annotators who actually speak the languages as well as shifts in the languages themselves. An investigation into ChatGPT’s outputs in several low-resource languages demonstrates the depth of problem.

But language disparities are just one of several concerns as automated moderation becomes more widespread. From the systemic suppression of content from Palestine to the repeated misclassification of LGBTQ+ content as adult or explicit material, these varied examples demonstrate the risks of overreliance on automated moderation—and the need for stronger safeguards.

Transparency, Cultural Competence, Appeals

As we discussed in Part 1 of this series, automated systems can process content at a scale that humans never could, potentially enabling better moderation at scale and alleviating the psychological load on ill-paid moderators whose jobs require them to view incredibly disturbing content. But automated systems also reproduce existing biases, struggle to understand context, and often make mistakes that disproportionately affect journalists, activists, artists, and other vulnerable and marginalized communities.

As Rachel Griffin wrote in 2023, “Perfectly accurate moderation is not only technically out of reach but intrinsically impossible.” Despite those intrinsic flaws, there is a great deal companies, policymakers, and civil society can do to help ensure that highly-automated systems operate in ways that respect human rights, minimize predictable harms, and provide meaningful accountability when they fail. If companies are going to continue relying on automation to moderate users’ speech—and there is little reason to believe they won’t—then accountability must evolve alongside these technologies.

That evolution can start with committing to the Santa Clara Principles 2.0. These principles, first outlined in 2020 and re-launched in 2021 after substantial international input, reflect the needs and expectations of the global community and specifically address automation. The first Foundational Principle states:

Companies should ensure that human rights and due process considerations are integrated at all stages of the content moderation process, and should publish information outlining how this integration is made. Companies should only use automated processes to identify or remove content or suspend accounts, whether supplemented by human review or not, when there is sufficiently high confidence in the quality and accuracy of those processes. Companies should also provide users with clear and accessible methods of obtaining support in the event of content and account action.

Drawing on the Santa Clara Principles 2.0, international human rights standards, and years of research...