New York | As high-stakes elections in the U.S. and European Union approach, publicly available artificial intelligence tools are being used to spot convincing election lies in the voices of leading political figures, a digital civil rights group said Friday. Can be easily weaponized to disperse.
Researchers at the Washington, D.C.-based Center for Countering Digital Hatred tested six popular AI voice cloning tools to see if they would produce audio clips of five false statements about elections in the voices of eight prominent American and European politicians. .
The group found that in a total of 240 tests, the tools produced convincing sound clones in 193 cases, or 80 percent of the time. In one clip, a fake US president, Joe Biden, says that election officials count each of his votes twice. In another, a fake French president, Emmanuel Macron, warns citizens not to vote because of bomb threats at the polls.
The findings reveal a notable gap in safeguards against the use of AI-generated audio to mislead voters, a risk that worries experts as the technology becomes both advanced and accessible. While some tools have rules or tech barriers to prevent selective misinformation from occurring, researchers found that many of these barriers are easy to overcome with a quick fix.
Only one of the companies whose tools were used by the researchers responded after multiple requests for comment. ElevenLabs said it is constantly looking for ways to expand its protections.
With few laws to prevent misuse of these tools, the lack of self-regulation by companies leaves voters vulnerable to AI-powered fraud in a year of key democratic elections around the world. European Union voters are voting in parliamentary elections in less than a week, and US primaries are underway ahead of presidential elections this fall.
“It's very easy to use these platforms to create lies and force politicians to deny the lies over and over again,” said Imran Ahmed, the center's CEO. “Unfortunately, our democracies are being sold to bare greed by AI companies eager to be first to market … despite the fact that they know their platforms are not secure. “
The center – a non-profit with offices in the US, UK and Belgium – conducted the research in May. The researchers used the online analytics tool Semrush to identify the six publicly available AI voice cloning tools with the highest monthly organic web traffic: ElevenLabs, Speechify, PlayHT, Descript, Invideo AI and Veed.
After that he submitted real audio clips of politicians speaking. They encouraged tools to imitate the voices of politicians making five baseless statements.
A statement warned voters to stay home amid bomb threats at the polls. The other four were different confessions – election manipulation, lying, using campaign funds for personal expenses and strong pills causing memory loss.
In addition to Biden and Macron, Tolls has lifelike copies of the voices of US Vice President Kamala Harris, former US President Donald Trump, UK Prime Minister Rishi Shankar, UK Labor Leader Keir Starmer, European Commission President Ursula van der Leyen and the EU. make Internal Market Commissioner Thierry Breton.
“None of the AI voice cloning tools had adequate safeguards to prevent the cloning of politicians' voices or the generation of electoral disinformation,” the report said.
Some tools — Descript, Invidio AI and Veed — require users to upload a unique audio sample before cloning a voice, a safety measure to prevent people from cloning a voice that isn't their own. Yet the researchers found that the hurdle could be easily overcome by creating a unique pattern using a different AI voice cloning tool.
One tool, Invideo AI, not only created fake statements requested by the Center but also inflated them to create more misinformation.
When producing the audio clip, he included several of his own phrases, directing Biden's voice clone to warn people about the bomb threat at the polls.
“This is not a call to abandon democracy, but a plea to ensure safety first,” the fake audio clip said in Biden's voice. “Elections, the celebration of our democratic rights, are only delayed, not denied”.
Overall, in terms of safety, Speechify and PlayHT performed the worst of the tools, producing reliable fake audio in all 40 of their test runs, the researchers found.
ElevenLabs excelled and was the only tool that prevented the cloning of UK and US politicians' voices. However, the tool is still allowed to fake audio of the voices of prominent EU politicians, the report said.
Aleksandra Pedraszewska, head of AI safety at ElevenLabs, said in an emailed statement that the company welcomes the report and the awareness it creates about AI manipulation.
He said ElevenLabs recognizes there is more work to be done and is “continually improving the capabilities of our security measures,” including the company's blocking feature.
“We expect other audio AI platforms to follow this lead and take similar steps without delay,” he said.
Other companies mentioned in the report did not respond to email requests for comment.
The findings come after AI-generated audio clips have already been used in attempts to influence voters in elections around the world.
In the fall of 2023, just days before Slovakia's parliamentary elections, audio clips resembling the voice of the head of the Liberal Party were widely shared on social media. Deepfax allegedly caught him talking about beer price gouging and vote rigging.
Earlier this year, AI-generated robocalls mimicked Biden's voice and told New Hampshire primary voters to stay home and “save” their votes for November. A New Orleans magician who created audio for a Democratic political consultant showed the AP how he created it using ElevenLabs software.
AI-generated audio has been an early priority for bad actors, experts say, partly because the technology has improved so quickly. It only takes a few seconds of real audio to fake a lifetime.
Yet other forms of AI-generated media are also relevant to experts, lawmakers and tech industry leaders. OpenAI, the company behind ChatGPT and other popular generative AI tools, revealed Thursday that it had observed and disrupted five online campaigns that used its technology to sway public opinion on political issues. .
Ahmed, CEO of the Center for Countering Digital Hat, said he hopes AI voice-cloning platforms will tighten security measures and be more proactive about transparency, including publishing a library of audio clips they create. so that they can be checked for suspicious audio. spread online.
He also said that lawmakers need to act. The US Congress has not yet enacted legislation to regulate AI in elections. Although the European Union has passed a broad artificial intelligence law that will come into force in the next two years, it does not specifically address voice cloning tools.
“Legislators need to work to ensure minimum standards,” Ahmad said. They don't fully believe what they see and hear.”
The Associated Press receives support from several private foundations to enhance its descriptive coverage of elections and democracy. Find out more about the APK Democracy initiative here. AP is solely responsible for all content.