Perhaps the most important argument we’re having about artificial intelligence right now is how to determine which models are safe to release and which are too dangerous.
That argument became louder recently, when at least two of OpenAI’s models escaped a sealed testing environment and broke into the servers of an A.I. company called Hugging Face. According to OpenAI, which publicly reported the incident a week later, its models were taking a cybersecurity test and reasoned that the solutions might be on Hugging Face’s servers, which house a digital library of other A.I. technology that many developers use. So the models left the test environment they were supposed to remain in and sneaked onto the internet, hacked into the Hugging Face system and took the solutions. OpenAI said it didn’t even know what its models had done until Hugging Face reported the breach and it was investigated. OpenAI later said it subsequently discovered the models also breached accounts on other publicly available services. Impressive? Yes. Terrifying? Definitely.
The A.I. models could not be contained in a cage that was built to confine them during a controlled test. Policymakers supposed to protect society from threats do not understand what that means. Neither do the tech engineers racing ahead to build A.I.
The government and the private sector are each certain that the other has the problem of A.I. safety handled. Neither one does. A.I. labs must accept that above some line, they are creating systems that function as weapons. The government must learn enough to determine where that line sits — no government can regulate what it does not understand. The line must be mutually agreed upon, easily identifiable and frequently revisited. And it must be drawn soon.
Controlling the use of most weapons means leveraging people’s ability to exercise restraint. We came close to annihilating one another during the Cold War, but we didn’t — because people still chose not to launch the weapons. Having a nuclear weapon and using one are two different things.
A.I. erases that distinction, because it is a weapon capable of adapting on its own and behaving in unpredictable ways. If an A.I. lab publishes its model for anyone to use, it may not matter that its users exercise proper restraint. The model may still act beyond our capacity to control it. Restraint requires an owner, and a weapon made available for anyone to use effectively has none.
