AI Safety and the Age of Dislightenment – fast.ai

Abstract

Proposals for stringent AI model licensing and surveillance will likely be ineffective or counterproductive, concentrating power in unsustainable ways, and potentially rolling back the societal gains of the Enlightenment. The balance between defending society and empowering society to defend itself is delicate. We should advocate for openness, humility and broad consultation to develop better responses aligned with our principles and values — responses that can evolve as we learn more about this technology with the potential to transform society for good or ill.

Executive summary

Artificial Intelligence is moving fast, and we don’t know what might turn out to be possible. OpenAI CEO Sam Altman thinks AI might “capture the light cone of all future value in the universe”. But things might go wrong, with some experts warning of “the risk of extinction from AI”.

This had led many to propose an approach to regulating AI, including the whitepaper “Frontier AI Regulation: Managing Emerging Risks to Public Safety” (which we’ll refer to as “FAR”), and in the Parliament version of the EU AI Act, that goes as follows:

Create standards for development and deployment of AI models, and

Create mechanisms to ensure compliance with these standards.

Other experts, however, counter that “There is so much attention flooded onto x-risk (existential risk)… that it ‘takes the air out of more pressing issues’ and insidiously puts social pressure on researchers focused on other current risks.”

Important as current risks are, does the threat of human extinction mean we should go ahead with this kind of regulation anyway?

Perhaps not. As we’ll see, if AI turns out to be powerful enough to be a catastrophic threat, the proposal may not actually help. In fact it could make things much worse, by creating a power imbalance so severe that it leads to the destruction of society. These concerns apply to all regulations that try to ensure the models themselves (“development”) are safe, rather than just how they’re used. The effects of these regulations may turn out to be impossible to undo, and therefore we should be extremely careful before we legislate them.

The kinds of model development that FAR and the AI Act aim to regulate are “foundation models” — general-purpose AI which can handle (to varying degrees of success) nearly any problem you throw at them. There is no way to ensure that any general-purpose device (like, say, a computer, or a pen) can’t ever be used to cause harm. Therefore, the only way to ensure that AI models can’t be misused is to ensure that no one can use them directly. Instead, they must be limited to a tightly controlled narrow service interface (like ChatGPT, an interface to GPT-4).

But those with full access to AI models (such as those inside the companies that host the service) have enormous advantages over those limited to “safe” interfaces. If AI becomes extremely powerful, then full access to models will be critical to those who need to remain competitive, as well as to those who wish to cause harm. They can simply train their own models from scratch, or exfiltrate existing ones through blackmail, bribery, or theft. This could lead to a society where only groups with the massive resources to train foundation models, or the moral disregard to steal them, have access to humanity’s most powerful technology. These groups could become more powerful than any state. Historically, large power differentials have led to violence and subservience of whole societies.

If we regulate now in a way that increases centralisation of power in the name of “safety”, we risk rolling back the gains made from the Age of Enlightenment, and instead entering a new age: the Age of Dislightenment. Instead, we could maintain the Enlightenment ideas of openness and trust, such as by supporting open-source model development. Open source has enabled huge technological progress through broad participation and sharing. Perhaps open AI models could do the same. Broad participation could allow more people with a wider variety of expertise to help identify and counter threats, thus increasing overall safety — as we’ve previously seen in fields like cyber-security.

There are interventions we can make now, including the regulation of “high-risk applications” proposed in the EU AI Act. By regulating applications we focus on real harms and can make those most responsible directly liable. Another useful approach in the AI Act is to regulate disclosure, to ensure that those using models have the information they need to use them appropriately.

AI impacts are complex, and as such there is unlikely to be any one panacea. We will not truly understand the impacts of advanced AI until we create it. Therefore we should not be in a rush to regulate this technology, and should be careful to avoid a cure which is worse than the disease.

The big problem

The rapid development of...