New Delhi, India – A seven-second clip went viral on social media as voters lined up in the early hours of November 30 last year to vote in legislative elections to elect the next government in the southern Indian state of Telangana.
Posted on X by the Congress Party, which is in opposition at the national level, and was in the state at the time, it showed KT Rama Rao, a leader of the Bharat Rashtra Samithi, which rules the state. They were appealing to vote in their favor. Congress
According to a senior leader who requested anonymity, the Congress shared it widely on a range of WhatsApp groups “unofficially run” by the party. It eventually ended up on the party’s official X account, getting over 500,000 views.
It was fake.
“Of course, it was generated by AI even though it looks completely real,” the Congress party leader told Al Jazeera. But a typical voter will not be able to tell the difference. Voting had started [when the video was posted] And there was no time [the opposition campaign] For damage control.”
The well-timed deepfake was a hallmark of the deluge of AI-generated, or manipulated, media that has affected a series of state elections in India in recent months, and is now central to the country’s upcoming general elections. There is a risk to form.
Between March and May, India’s nearly one billion voters will choose their next national government in the largest elections in the world and in history. The threat posed by fraudulent AI-generated media gained global attention when fake sexually explicit images of artist Taylor Swift appeared on social media platforms in January. In November, India’s Information Technology Minister, Ashwini Vaishnav, called Deepfax a “threat to democracy” and Prime Minister Narendra Modi echoed those concerns.
But with the increasing availability of simple artificial intelligence tools, all political parties in India, including Modi’s Bharatiya Janata Party and the Congress, are deploying deepfakes to sway voters, about 40 recent campaign managers told Al Jazeera. While many of the AI tools used to create DeepFax are free, others are available on subscription for 10 cents per video.
‘creating awareness’
The BJP, India’s most technologically advanced party, has been at the forefront of using illusions for election campaigning. By 2012, the party used 3D hologram projections of Modi to “campaign” in dozens of locations simultaneously. This strategy was widely used during the 2014 general elections that brought Modi to power.
There was little fraud involved, but in February 2020, Manoj Tiwari, a BJP Member of Parliament, became one of the first people in the world to use Deep Fax for election campaigning. In three videos, Tiwari addressed voters in Delhi ahead of the capital’s Legislative Assembly elections in Hindi, Haryanvi and English – reaching three distinct audiences in the multicultural city. Only the Hindi video was authentic: the other two were deepfakes, where AI was used to create her voice and words and change her expressions and lip movements to make her almost impossible to detect just by looking. , that they are not real.
In recent months, the Dravida Munnetra Kazhagam (DMK), which rules the southern state of Tamil Nadu, has used videos such as the life of the former film writer and veteran politician in its campaign programs to portray its popular leader M Karunanidhi as a man. Have used AI to bring life to life.
Now, consultants and campaign managers say the 2024 election could further turbocharge the use of DeepFax.
“Politics is about visualization; with AI tools [of voice and video modulation] And with one click, you can turn the concept on its head in a minute,” said Arun Reddy, national social media coordinator at the Congress, who oversaw the party’s tech-savvy Telangana polls. He added that The team was bursting with ideas to incorporate AI into the campaign, but they didn’t have enough “trained people” to execute them all.
Reddy is strengthening his team – as are other parties.
“AI will have a huge impact on narrative,” Reddy told Al Jazeera. “Political AI-manipulating content will multiply, much more than it was before.”
‘Campaigns are getting weirder’
From the desert town of Pushkar in western India, 30-year-old Devendra Singh Jadwin runs an AI startup, The Indian Deep Faker. Launched in October 2020, his company cloned the voice of Ashok Gehlot, the Congress chief ministerial candidate in the state of Rajasthan, to have his team address every voter by name during the November assembly elections. Send personalized messages on WhatsApp. Indian Deepfaker is currently working with Sikkim Chief Minister Prem Singh Tamang’s team for the hologram during the upcoming campaign. Sikkim is one of India’s smallest states in the northeast, located in the Himalayas between India, Bhutan and China.
“This is pure, official work,” he said. But in recent months, they have been hit with what political campaigns have described as “immoral solicitations.” “Political parties communicate indirectly through international numbers on WhatsApp, burner handles on Instagram, or Telegram,” Jadoun told Al Jazeera in a phone interview.
In the November election, his company rejected more than 50 such requests, he said, where potential clients wanted to alter videos and audio to target political opponents, including obscene content. As a startup, Jadoun said his company is particularly careful to avoid any legal trouble. “And this is a very unethical use of AI,” he added. “But I know a lot of people who are doing it at very low prices and are readily available now.”
Last November, during election campaigns for the state legislative assemblies of Madhya Pradesh in central India and Rajasthan in the west, police arrested senior politicians including Modi, Shivraj Singh Chouhan, Kailash Vijayvargiya (all BJP) and Kamal Nath (Congress). filed multiple lawsuits for deepfake videos targeting ). The production of deepfake content is often outsourced to private consulting firms, which rely on social media networks for distribution, led by WhatsApp.
A political consultant, who asked not to be named, told Al Jazeera that numbers of ordinary citizens are registered on WhatsApp and used for campaigns to identify parties, candidates, consultants and AI. Firms may find it difficult to track them directly.
The consultant had run six campaigns for both the BJP and the Congress in the last year’s assembly elections. “In Rajasthan, we were using the phone numbers of construction workers to run our network on WhatsApp,” he said, “where mainly deep faxes circulated.”
Meanwhile, AI-manipulated audio is a particularly valuable tool in small circles, “using fake call recordings to target candidates to arrange ‘black money’ for elections or threaten to buy votes. ” said the consultant, whose own candidate was targeted by one such recording. . The recordings are usually blacked out with the candidates’ voices so they can be cast as evidence of corruption.
“Voter manipulation by AI is not being considered a sin by any party,” he added. “It’s just part of the campaign strategy.”
India has 760 million internet users – more than 50 percent of the population – second only to China.
Among all the petitions, one from a constituency in southern Rajasthan stood before Jedoon. Ahead of the state election in November, the caller requested that Jadwin alter a difficult but authentic video of his candidate – whose party he did not disclose – to create a realistic deepfake. The goal: to claim that the original was a deepfake, and that the deepfake was real.
“The opposition had a disturbing video of their candidate and wanted to quickly spread it on social media to claim it was a deepfake,” he said with a wry laugh. “Political campaigns are getting weirder.”
Threat to electoral integrity
Anushka Jain, a policy researcher at the Goa-based Digital Future Lab, said Indian laws currently do not clearly define “deep fax”. The police have been using laws against defamation, fake news or violation of decency of a person along with the Information Technology Act to deal with individual cases. But often, they’re playing whack-a-mole.
“The police are prosecuting the effect of deepfake and not because it is deepfake itself,” he said.
Analysts say the Election Commission of India (ECI), an independent body that manages polls, needs to catch up with the changing nature of political campaigns.
In the days before voting in the Telangana state elections last year, leaders of the ruling Bharat Rashtra Samiti Party repeatedly warned His followers on social media should be on alert against deepfakes deployed by the Congress party. He also appealed to the ECI against the deepfake clip that the Congress had shared on the morning of the vote.
But the video Remaining Online and the party never received notice from the ECI, two Congress leaders aware of the issue told Al Jazeera.
Al Jazeera has sought comments from the ECI but is yet to receive a response.
Former Chief Election Commissioner of India SY Qureshi said, “If a person is misled to believe something and it changes his mind, it undermines the integrity of the electoral process.” “Deepfakes make the problem of rumor mongering a thousand times more serious during elections.”
Qureshi said deepfakes need to be moderated in real time to reduce the damage they do to Indian democracy.
“ECI needs to act before it does damage,” he said. “They need to be much more urgent.”
‘Truth is out of reach’
The Indian government is pressuring major tech companies, including Google and Meta, to proactively make efforts to moderate deepfakes on their platforms. IT Minister Rajiv Chandrasekhar has met officials of these firms as part of discussions on the threat posed by DeepFax.
By asking the tech sector to take the lead, the government avoids any criticism that it is trying to selectively censor deepfakes, or that it is cracking down on emerging AI technologies more broadly. trying to
But by giving money to private companies, the government is questioning the sincerity of its intention to regulate manipulative content, said Prateek Waghre, executive director of the Internet Freedom Foundation of India, a leading tech policy think tank based in New Delhi. . “It’s almost wishful thinking,” he said.
Arguing that tech companies are unable to deal with the current problems with content moderation, Waghre said the “rise of AI now” has exacerbated the challenges. And the current approach to content moderation ignores what is at the heart of the problem.
“You are not solving the problem,” he said. “Design [of algorithms] There is only error.”
On February 16, major tech companies signed an agreement at the Munich Security Conference to voluntarily adopt “reasonable precautions” to prevent artificial intelligence tools from being used to disrupt democratic elections around the world. But the vaguely worded agreement frustrated many advocates and critics.
YouTube has announced that it will enable people to request the removal of AI-generated or altered content that mimics an identifiable person, including their face or voice, to protect their privacy. Using the application process.
“I’m not very optimistic about the platform’s ability to detect deepfakes,” said Ravi Iyer, managing director of the Nellie Center for Ethical Leadership and Decision-Making at the University of Southern California’s Marshall School of Business. “With low digital literacy and increased consumption of videos, this poses a serious threat to India’s electoral integrity.”
Iyer said it’s not a reasonable task to identify every AI-manipulated media, so companies need to redesign algorithms that don’t promote polarizing content. “Companies have the money and resources they need to take reasonable steps to deal with the rise of deepfakes,” he said.
The Internet Freedom Foundation has published an open letter urging election candidates and parties to voluntarily refrain from using deepfake technology ahead of national elections. Waghre isn’t sure many people will bite, but he said it’s worth a try.
Meanwhile, political campaigns are bolstering their AI arsenal — and some, like Reddy, the national coordinator for social media in Congress, admit the future looks bleak.
“Most of the people using AI are there to distort the facts. They want to create an impression that is not based on the truth,” Reddy said. Coupled with the rise of , the truth in elections will now be beyond the reach of the people.”