AI and holography bring 3D augmented reality to regular glasses.

Researchers in the emerging field of spatial computing have developed a prototype augmented reality headset that uses holographic imaging to overlay full-color, 3D moving images onto lenses that appear to be an ordinary pair of glasses. Unlike the bulky headsets of today's augmented reality systems, the new approach delivers a visually satisfying 3D viewing experience in a compact, comfortable and attractive form factor suitable for all-day wear.

“Our headset appears to the outside world just like a pair of glasses, but what the wearer sees through the glasses is overlaid with dynamic, full-color 3D computed imagery,” said Gordon Weitzstein, an associate professor of electrical engineering. It's a rich world.” and expert in the rapidly emerging field of spatial computing.

Wetzstein and a team of engineers are introducing their device in a journal paper. The nature.

Although now only a prototype, such technology, he says, could transform fields ranging from gaming and entertainment to training and education — anywhere the wearer can see the world around them with computed imagery. can enhance or inform understanding of

“One can imagine a surgeon wearing such glasses performing a delicate or complex surgery or an airplane mechanic learning to work on a sophisticated jet engine using them,” says Manu Gopa Kumar, of Weitzstein. Led by a doctoral student and fellow in the Stanford Computational Imaging Lab. said the first author of the paper.

Overcoming obstacles

The new approach is the first to address a complex maze of engineering requirements that have so far produced either haphazard headsets or less-than-satisfying 3D visual experiences that leave the wearer visually fatigued, or sometimes a little Can also cause nausea.

“There is currently no other augmented reality system that has a comparable compact form factor or matches our 3D image quality,” said Gun Yell Lee, a postdoctoral researcher at the Stanford Computational Imaging Lab and co-first author of the paper. “

To succeed, researchers have overcome technical hurdles through a combination of AI-enhanced holographic imaging and novel nanophotonic device approaches. The first obstacle was that augmented reality imaging techniques often require the use of complex optical systems.

In these systems, the user does not actually see the real world through the lens of the headset. Instead, cameras mounted on the outside of the headset capture the world in real time and combine that image with computed imagery. The resulting blended image is then projected stereoscopically to the user's eyes.

“The user sees a digitized projection of the real world with computed imagery. It's a kind of augmented virtual reality, not true augmented reality,” Lee explained.

Wetzstein explains that these systems are necessarily bulky because they use a magnifying lens between the wearer's eye and the projection screens, which require a minimum distance between the eye, the lens and the screen, adding extra size. happens.

“Beyond largeness, these limitations can lead to unsatisfactory perceptual realism and often visual discomfort,” said Suwon Choi, a doctoral student in the Stanford Computational Imaging Lab and co-author of the paper.