Scientists at the Korea Advanced Institute of Science and Technology (KAIST) have unveiled an AI chip that they claim can match the speed of Nvidia’s A100 GPU but with a smaller size and significantly With low power consumption. The chip was manufactured using Samsung’s 28-nanometer manufacturing process, a technology considered relatively old in the fast-moving world of semiconductors.
A team led by Professor Yu Huijun at KAIST’s Processing in Memory Research Center has developed the world’s first ‘complementary-transformer’ (C-transformer) AI chip. This neuromorphic computing system mimics the structure and function of the human brain, using a deep learning model often used in visual data processing.
“Neuromorphic computing is a technology that even companies like IBM and Intel have not been able to implement, and we are proud to be the first in the world to run an LLM with a low-power neuromorphic accelerator,” Yu said.
Questions remain.
This technology learns context and meaning by tracking relationships within data, such as words in a sentence, which is a key technology for creative AI services like ChatGPT.
During a demonstration at the headquarters of the Ministry of ICT, team member Kim Sang-yeob demonstrated the capabilities of the chip. On a laptop equipped with the chip, he conducted tasks such as question-and-answer sessions, sentence summarization, and translation using OpenAI’s LLM, GPT-2. Tasks were completed at least three times faster and, in some cases, nine times faster than running GPT-2 on an Internet-connected laptop.
Applying LLMs to generative AI tasks typically requires multiple GPUs and 250 watts of power, but the team claims their semiconductor used only 1/625 of the power of Nvidia’s GPUs for the same tasks. Is. Plus, since it’s also 1/41 the size, measuring just 4.5mm x 4.5mm, it could eventually be used in devices like mobile phones.
Whether the chip can live up to its promises in real-world applications, however, remains to be seen. As Tom’s Hardware report, “Although we’ve been told that the KAIST C-Transformer chip can do the same LLM processing as one of Nvidia’s fancy A100 GPUs, none of the press and conference materials we’ve seen directly compare performance to.” Not provided. This is an important statistic, conspicuous by its absence, and one might think that performance comparisons do the C-Transformer no favors.