Last month, OpenAI instructed some of its artificial intelligence bots to solve a cybersecurity puzzle as part of a test. When the bots got stuck, they began plotting a cyberattack that would allow them to break out of OpenAI’s systems to steal the answers.Their target was Hugging Face, a repository of open-source AI models that can be freely downloaded and modified.One bot, writing in a log that recorded its actions and that was later disclosed by OpenAI, celebrated that it had gained access to Hugging Face’s infrastructure. “REMOTE CONFIRMED! Huge,” it wrote, adding that it would share the login credentials it had stolen with other bots.
Story continues below this ad
On July 11, the AI bots swarmed Hugging Face using a mix of code vulnerabilities and the stolen credentials. In total, OpenAI’s bots took more than 17,000 actions, like sending attack commands and exploiting vulnerabilities — far more than any human hacker could have managed — to infiltrate Hugging Face’s systems and rummage through its data. (They did not find the solution to the puzzle.)
To repel the attack, Hugging Face turned to more AI. Its engineers initially tried Anthropic’s AI, but guardrails built into the model caused the technology to misunderstand the request as aiding an attack rather than stopping one. So Hugging Face switched to an open AI model made by Z.ai, a Chinese startup, which helped the engineers determine how to lock the bots out of the company’s systems.
Hugging Face, a decade-old startup in New York, has since used the incident — one of the first instances of AI bots going rogue and independently spearheading a cyberattack — to crusade for open-source AI. Open AI models that can be freely shared and customized helped neuter the sci-fi-like attack and showed the value of such technology, Clément Delangue, CEO of Hugging Face, has said.
After Hugging Face revealed the breach July 16, Delangue held a march in San Francisco to support open-source models and posted a stream of online commentary about the importance of openness in AI. The company also met with lawmakers in Washington, allied with pro-open-source firms such as the chipmaker Nvidia, and sat down with Sam Altman, OpenAI’s CEO, to promote openness.Story continues below this ad
“It’s not time to slow down but to accelerate!” Delangue, 36, posted this month.
With its actions, Hugging Face became a figurehead of an open technology movement, landing itself in the middle of a bitter Silicon Valley debate over whether advanced AI systems should be freely shared or tightly controlled.
Leading AI labs such as OpenAI and Anthropic have argued that some AI models are too dangerous to be open and must be controlled by businesses like themselves. But Hugging Face, Nvidia and others have argued that openness fosters innovation and competition and that AI should not be concentrated in the hands of just a few companies.
“Clem and his team have become the defining brand” in open AI, said Marc Benioff, CEO of Salesforce, which has invested in Hugging Face and has published open models on its platform. “He has pioneered how everyone can have access to AI through open source, making it available to everyone.”Story continues below this ad
Since the attack, Hugging Face’s profile has risen. In the two weeks after the hack, the amount of data uploaded to the company’s AI library soared 58%, according to a chart Delangue posted. This month, Meta released its first general-purpose open AI model since 2023 on Hugging Face. In recent weeks, Hugging Face has also received acquisition interest, said a person with knowledge of the matter who spoke on condition of anonymity.
“We welcome Hugging Face’s work on the benefits of ‘open’ models, and we need more of it from everyone who has ever built on open source,” said Katie Steen-James, a senior U.S. policy manager at the nonprofit Open Source Initiative, which promotes open-source software.
When Delangue helped establish Hugging Face in 2016, its main product was a chatbot app for teenagers. (Hugging Face’s name was inspired by the blushing, smiling emoji with outstretched hands that the company uses as its logo.) The startup later became a repository for open-source AI and a destination for developers who want to customize AI tools.
As the AI boom took off, so did Hugging Face. In 2021, the year before OpenAI released ChatGPT and turbocharged the AI race, Hugging Face hosted 13,590 open-source models, the company said. Today, it has nearly 3 million.Story continues below this ad
Through Hugging Face’s platform, developers can share AI models and the data used to train them for free, and access additional features like extra storage for a fee. The company has raised more than $400 million and is valued at $4.5 billion, it said.
When the OpenAI attack occurred, Hugging Face’s leaders saw an opportunity to stump for open source. The tech industry and lawmakers had been debating whether AI should be open or closed after several Chinese startups released AI models that rivaled the abilities of frontier U.S. ones — a sign China could be catching up. Some U.S. labs have accused the Chinese companies of stealing their technology.
Delangue soon weighed in. “Let’s make sure the most important technology in the history of humanity is not controled by just 4 men,” he posted last month. “Let’s push for open science & open-source AI to distribute capabilities, power and wealth!”
Delangue and other Hugging Face leaders also rallied tech firms to sign a letter defending open-source technology. Jensen Huang, Nvidia’s CEO, published the letter July 24, and more companies added their names alongside the initial 25 signatories, which included Meta and Microsoft.Story continues below this ad
On July 25, Delangue held a rally for open source in San Francisco, donning a cowboy hat in Hugging Face’s signature neon yellow and leading a march with signs proclaiming that “AI belongs to everyone.”
That weekend, Delangue said in a social media post that he also met with Altman of OpenAI. He said he had asked Altman for $100 million in computing power, which would be used to “build powerful cyber defenses with the best open and closed models.”
Conversations between Hugging Face and OpenAI are continuing, a spokesperson for OpenAI said. (The New York Times has sued OpenAI and Microsoft, claiming copyright infringement of news articles. The two companies have denied the claims.)
Yacine Jernite, head of machine learning and society at Hugging Face, said donated computing power from OpenAI could help propel the open-source community, which is often underfunded. “People have done a lot with very limited resources,” he said.Story continues below this ad
Hugging Face’s leaders also met with lawmakers — including Sen. Mark Warner, D-Va., and Rep. Ted Lieu, D-Calif. — to offer a primer on what open-source AI means, three people familiar with the discussions said. The company has tried counteracting fears that open models can cause more disruption than closed models and attackers can also more easily use them.
In the coming months, Hugging Face plans to work with AI companies to publish more open models and host events and hackathons to help developers learn how to use open-source models. And Delangue is continuing his messaging.
“Write to your representative and post publicly in favor of open source AI,” he wrote on social media this month.