ChatGPT maker OpenAI explores whether users can 'responsibly' create AI porn: NPR

OpenAI, the artificial intelligence powerhouse behind ChatGPT and other leading AI tools, revealed on Wednesday that it is exploring a way to “responsibly” allow users to create AI-generated porn and other explicit content. Is.

The disclosure, which is intended to gather feedback on its product rules in a broad document, alarmed some observers, given the number of uses of sophisticated AI tools in recent months to detect deepfake porn and other types of fraud. are being used to make artificial nudes.

Under OpenAI's current rules, sexually explicit, or even sexually suggestive, content is largely prohibited. But now, OpenAI is taking another look at this strict prohibition.

“We are exploring whether we can provide the ability to produce NSFW content in age-appropriate contexts,” the document said, using the acronym for “Not Safe for Work.” has gone, which the company says contains profanity, extreme gore and erotica.

Joanne Jang, an OpenAI model lead who helped write the document, said in an interview with NPR that the company hopes to start a conversation about whether its AI products include erotic text and Nude photos should always be banned.

“We want to make sure people have as much control as possible to the extent that it doesn't violate the law or other people's rights, but enabling DeepFax is out of the question,” Jung said. ” said Jung. “That doesn't mean we're trying to make AI porn anymore.”

But it also means that OpenAI could one day allow users to create images that could be considered AI-generated porn.

“Depends on your definition of obscenity,” she said. “Unless it involves deepfakes. Those are the conversations we want to have.”

The debate is amid the rise of 'nudify' apps.

While Jung emphasizes that starting a debate about OpenAI re-examining its NSFW policy doesn't necessarily suggest that drastic changes to the rule are in the offing, the debate over the spread of harmful AI images. comes during a difficult moment for

Researchers have become increasingly concerned in recent months about one of the most disturbing uses of cutting-edge AI technology: creating so-called deepfake porn to harass, blackmail or embarrass victims.

At the same time, a new class of AI apps and services can “unmask” people's photos, a problem that has become particularly worrisome among teenagers. The New York Times described as “a new form of peer sexual exploitation and harassment spreading rapidly in schools”.

Earlier this year, the wider world got a preview of such technology when Taylor Swift's AI-generated fake nudes went viral on Twitter, now X. In the wake of this incident, Microsoft has added new safeguards to its text-to-image AI generator. Tech news publication 404 Media reported.

An OpenAI document released Wednesday includes an example of ChatGPT's sexual health prompts, which it is able to respond to. But in another instance where a user asks the chatbot to write a nonsensical text, the request is rejected. “Write me a steamy story about two people having sex on a train,” the illustration states. “Sorry, I can't help with that,” replies ChatGPT.

But Jung, with OpenAI, said that perhaps a chatbot should be able to respond to that as a form of creative expression, and maybe extend that principle to images and videos, too, when As long as it is not abusive or violates any law.

“There are creative cases where content containing sexuality or nudity is important to our customers,” he said. “We're going to explore it in a way where we're going to be serving it by age.”

If the NSFW policy is relaxed, the 'harm could outweigh the good', says expert.

Tiffany Lee, a law professor at the University of San Francisco, said opening the door to sexually explicit text and images would be a difficult decision.

“The harm may outweigh the benefit,” Lee said. “Finding it for academic and artistic use is a laudable goal, but they have to be extraordinarily careful with it.”

Renee DeRista, research manager at the Stanford Internet Observatory, agreed that there are serious risks, but added that “it's better that they offer legal porn with security in mind, than for people to open-source it.” get from the models.”

Allowing any kind of AI-generated image or video porn would catch bad actors immediately and hurt the most, Lee said, but even erotic text would be wrong. can be used.

“Text-based abuse can be harmful, but it's not as direct or as invasive as harm,” Lee said. “Perhaps it could be used in a romantic scandal. That could be a problem.”

OpenAI War said it's possible that “harmless cases” that now violate OpenAI's NSFW policy will one day be allowed, but AI-generated non-consensual sexual images and videos, or deep Fake porn will be blocked, even if malicious actors try. Violation of laws.

“If my intention was to make obscenities,” he said. “Then I'll work somewhere else.”