AI chip reduces energy budget by 99+ percent.

Neural networks that mimic the workings of the human brain now often drive art, power computer vision, and many other applications. Now a Chinese neural network microchip that uses photons instead of electrons, dubbed Taichi, can perform AI tasks as well as its electronic counterparts with one-thousandth the energy, according to new research. Is.

AI typically relies on artificial neural networks in applications such as analyzing medical scans and creating images. In these systems, circuit components called neurons—similar to neurons in the human brain—provide data and contribute to solving a problem, such as recognizing faces. Neural networks are said to be “deep” if they contain multiple layers of these neurons.

“Optical neural networks are no longer toy models. They can now be applied to real-world tasks. – Lu Feng, Tsinghua University, Beijing

As neural networks grow in size and power, they are becoming more energy hungry when running on traditional electronics. For example, to train its state-of-the-art neural network GPT-3, a 2022 The nature The study suggested that OpenAI spent US$4.6 million to run 9,200 GPUs for two weeks.

The drawbacks of electronic computing have led some researchers to explore optical computing as a promising basis for next-generation AI. This photonic approach uses light to perform computing faster and with less power than an electronic counterpart.

Now, scientists at Tsinghua University in Beijing and the Beijing National Research Center for Information Science and Technology have developed Taichi, a photonic microchip that can be used as an electronic device to perform advanced AI tasks while being more energy efficient. .

“Optical neural networks are no longer toy models,” says Lu Feng, an associate professor of electronic engineering at Tsinghua University. “Now they can be applied to real-world tasks.”

How does an optical neural net work?

Two strategies for developing optical neural networks are either scattering light in specific patterns within microchips, or receiving light waves to precisely interfere with each other within devices. When input in the form of light flows into these optical neural networks, the output light encodes data from the complex operations performed within these devices.

Feng explains that both approaches to photonic computing have significant advantages and disadvantages. For example, optical neural networks that rely on scattering, or diffusion, can pack many neurons close together and use virtually no energy. Diffraction-based neural networks rely on the scattering of light beams as they pass through optical layers that represent network functions. A drawback of disparity-based neural nets, however, is that they cannot be reconstructed. Each string of operations can basically only be used for a specific task.

Taichi boasts 13.96 million parameters.

In contrast, optical neural networks that depend on interference are easily reconfigurable. Interference-based neural networks send multiple beams through a mesh of channels, and the way they interfere where these channels intersect helps the device perform operations. However, their drawback is related to the fact that interferometers are also large, which restricts how well such neural nets can scale. They also use a lot of energy.

In addition, current photonic chips experience inevitable errors. Attempting to extend optical neural networks by increasing the number of neuron layers in these devices usually only increases this inevitable noise exponentially. Which means that, until now, optical neural networks were limited to basic AI tasks like simple pattern recognition. Optical neural nets, in other words, were generally not suitable for advanced, real-world applications, Feng says.

Taichi, in contrast, is a hybrid design that combines both disparity and intervention methods, the researchers say. It consists of clusters of different units that can compress data for massive input and output in a compact space. But their chip also includes arrays of interferometers for reconfigurable calculations. Feng says that the encoding protocol developed for Taichi divides challenging tasks and large network models into subproblems and submodels that can be divided into different modules.

How does Taichi combine both types of neural networks?

Previous research has typically sought to increase the capacity of optical neural networks by mimicking what is often done with their electronic counterparts—increasing the number of neuron layers. Instead, Taichi’s architecture evolves by dividing computing into multiple chips that work in parallel. This means that Taichi can avoid the problem of rapid accumulation of errors that occurs when optical neural networks aggregate many layers of neurons.

“This ‘shallow in depth but wide in width’ architecture guarantees network scale,” says Feng.

Tachi produced music clips in the style of Bach and art in the style of Van Gogh and Munch.

For example, previous optical neural networks typically have only thousands of parameters—connections between neurons that mimic the synapses connecting biological neurons in the human brain. In contrast, Taichi boasts 13.96 million parameters.

Previous optical neural networks were often limited to classifying data with only a dozen or so categories—for example, determining whether images represent one of 10 digits. In contrast, in tests with the Omniglot database of 1,623 different handwritten characters from 50 different alphabets, Taichi showed 91.89 percent accuracy, which is comparable to its electronic counterparts.

The scientists also tested Taichi on the most advanced AI task of content creation. They found that it could produce music clips in the style of Johann Sebastian Bach and paint numbers and scenes in the style of Vincent van Gogh and Edvard Munch.

Overall, the researchers found that Taichi showed an energy efficiency of about 160 trillion operations per second per watt and an area efficiency of about 880 trillion multiply-accumulate operations (the most basic operation in neural networks) per square millimeter. of This makes it 1,000 times more energy efficient than one of the latest electronic GPUs, the NVIDIA H100, as well as nearly 100 times more energy efficient and 10 times more area efficient than previous optical neural networks.

Although the Taichi chip is compact and energy-efficient, Fang cautions that it relies on many other systems, such as a laser source and high-speed data coupling. She notes that these other systems are much heavier than a single chip, “taking up almost an entire desk.” In the future, Feng and his colleagues aim to add more modules on the chips to make the entire system more compact and energy efficient.

The scientists detailed their findings online in the journal April 11. science.