The disease took away his voice. The AI ​​created a replica that she keeps in her phone.

PROVIDENCE, R.I. (AP) — Alexis “Lexi” Bogan's voice was excited before last summer.

He loved singing Taylor Swift and Zac Bryan songs in the car. She was laughing all the time — even when misbehaving with preschoolers or debating politics over the backyard fire pit with friends. In high school, she was a soprano in chorus.

Then the voice went away.

Doctors removed a life-threatening tumor in the back of his brain in August. When the breathing tube came out a month later, Bogan had trouble swallowing and was stressed to say “hello” to her parents. Months of rehabilitation helped her recover, but her speech is still impaired. Friends, strangers and her own family members struggle to understand what she is trying to tell them.

In April, the 21-year-old got her old voice back. Not the original but an acoustic clone that has been developed. Artificial intelligence That she can call from the phone app. Trained on a 15-second capsule of her teenage voice — taken from a video of a cooking demonstration she recorded for a high school project — her synthetic but remarkably real-sounding AI voice Now she can say almost anything she wants.

She types a few words or phrases into her phone and the app instantly reads it out loud.

“Hello, could I have a grande iced brown sugar oat milk espresso please,” Bogan's AI voice said as he held the phone out his car window at a drive-thru Starbucks.

Experts have warned. That rapidly improving AI voice-cloning technology could increase phone scams could be disruptive. Democratic elections and violate the dignity of those—living or dead—who have never consented to rebuilding their voices to say things they never spoke.

It is used to generate Deepfake robocalls For New Hampshire voters who copy President Joe Biden. In Maryland, Authorities recently charged A high school athletic director who used AI to fake an audio clip of a school principal's racist remarks.

But Bogan and a team of doctors at Lifespan Hospital Group in Rhode Island believe they've found a use that justifies the risks. Bogan is one of the first people — with just his condition — to recreate lost voice with OpenAI's new voice engine. Some other AI providers, such as startup ElevenLabs, have tested similar technology for people with speech impediments and impairments — including a lawyer who now uses a clone of his own voice in the courtroom.

“We're hoping Lexi will be a trailblazer as the technology advances,” said Dr. Roheed Ali, a neurosurgery resident at Brown University Medical School and Rhode Island Hospital. He said that millions of people suffering from debilitating paralysis, throat cancer or neurological diseases could benefit.

Dr. Fatima Mirza, another resident who worked on the pilot, said, “We must be aware of the risks, but we cannot forget the patient and social good.” “We're able to help Lexi get her true voice back and she's able to speak in the terms that are most true to herself.”

Mirza and Ali, who are married, caught the attention of OpenAI, the maker of ChatGPT, because their previous research project at Lifespan used AI chat to simplify medical consent forms for patients. Used the bot. The San Francisco company arrived earlier this year while exploring medical applications for its new AI voice generator.

Bogan was still slowly recovering from surgery. The illness began last summer with headaches, blurred vision and a droopy face, worrying doctors at Hasbro Children's Hospital in Providence. They discovered a vascular tumor about the size of a golf ball pressing on his brain stem and tangled with blood vessels and cranial nerves.

“It was a battle to control the bleeding and get the tumor out,” said pediatric neurosurgeon Dr. Konstantina Soukos.

The location and severity of the tumor, as well as complications from the 10-hour surgery, damaged Bogen's control of his tongue muscles and vocal cords, hampering his ability to eat and speak, Soukos said. .

“It's almost like a part of my identity was taken when I lost my voice,” Bogan said.

The feeding tube came out this year. Speech therapy continues, enabling her to speak intelligibly in a quiet room but without a trace she will regain full fluency in her natural voice.

“At some point, I started to forget what I looked like,” Bogan said. “I'm used to how I sound now.”

Whenever the phone rang at the family's home in Providence, a suburb of North Smithfield, she would push it toward her mother to take the call. Every time they went to a noisy restaurant, she felt like she was burdening her friends. Her father, who is hard of hearing, struggled to understand her.

Back at the hospital, doctors were looking for a pilot patient to experiment with OpenAI's technology.

“The first person that came to Dr. Soukos' mind was Lexie,” Ali said. “We reached out to Lexie to see if she would be interested, not knowing what her reaction would be. She was playing it to try it out and see how it would work.

Bogan had to go back a few years to find a suitable recording of her voice so she could “train” the AI ​​system how she spoke. It was a video in which she explained how to make a pasta salad.

His doctors deliberately fed the AI ​​system only a 15-second clip. The sounds of cooking make other parts of the video incomplete. That's what OpenAI needed – improvements over previous technology required much longer samples.

They also knew that getting something useful out of 15 seconds could be essential for any future patient who has no trace of a voice on the Internet. A short voicemail left for a relative may be enough.

Everyone was stunned by the quality of the voice clone when they first tested it. The occasional glitches — a mispronounced word, a missing accent — were mostly unintelligible. In April, doctors fitted Bogan with a customized phone app that only she can use.

“I get so emotional every time I hear his voice,” said his mother, Pamela Bogan.

“I think it's great that I can have that voice again,” Lexi Bogan added, adding that it “helped raise my confidence to where it was before all of this happened.”

She now uses the app about 40 times a day and sends feedback that she hopes will help patients in the future. One of her first experiences was talking to children at the preschool where she works as a teaching assistant. He typed “ha ha ha ha” expecting a robotic response. To his surprise, it sounded like his old laugh.

He has used it in Target and Marshalls to ask where to find items. It has helped him reconnect with his father. And ordering fast food has become easier for him.

Bogan's doctors have begun cloning the voices of other consenting Rhode Island patients and hope to bring the technology to hospitals around the world. OpenAI said it is working carefully to expand the use of the voice engine, which is not yet publicly available.

Many small AI startups are already selling voice cloning services. In entertainment studios Or make them more widely available. Most audio vendors say they prohibit impersonation or abuse, but they vary in how they enforce their terms of use.

“We want to make sure that everyone whose voice is used in the service is giving consent on an ongoing basis,” said Jeff Harris, OpenAI's lead on product. “We want to make sure it's not used in a political context. So we've taken an approach of being very limited in who we're giving the technology to.

Harris said the next step for OpenAI involves developing a secure “voice authentication” tool so that users can simply duplicate their own voice. That could be “limiting for a patient like Lexi, who had a sudden loss of speech,” he said. “So we think we're going to need high-trust relationships, especially with medical providers, to have a little more seamless access to technology.”

Bogan has inspired his doctors with his focus on thinking about how technology can help others with similar or more severe speech impediments.

“Part of what he's done throughout this process is to think about ways to modify and change it,” Mirza said. “She has been a great inspiration to us.”

While right now she has to fiddle with her phone to get the voice engine to talk, Bogan envisions an AI voice engine that mimics older treatments for speech rehabilitation — such as a robotic voice electrolarynx or voice prosthesis. Organs—The human body grows better together. or translating words in real time.

She's less certain that as she gets older and her AI voice sounds the same as it did as a teenager. He said the technology might “age” his AI voice.

For now, “even though my voice isn't coming back completely, I have something that helps me find my voice again,” she said.

___

The Associated Press and OpenAI have. A licensing and technology agreement which allows OpenAI to access the text archives portion of the APK.