Groq, an AI chip company, wants everyone to forget about Elon Musk. The snarky chatbot with almost the same name, Grok. Lightning-fast demos from Groq went viral this weekend, leading to the current versions. Chat GPT, Gemini And even the gurus look lazy. Groq claims to provide the “world’s fastest large language models”, and third-party tests are saying the claim may hold up.
In a split second, Groq generates hundreds of words into a fact-based response, citing sources along the way. According to a demo posted on X. In another demo, founder and CEO Jonathan Ross a A CNN host had a real-time verbal conversation with an AI chatbot around the world on live television.. While ChatGPT, Gemini, and other chatbots are impressive, Groq can make them lightning fast. Fast enough for real-world practical use cases.
Groq makes AI chips called language processing units (LPUs), which it claims are faster than Nvidia’s graphics processing units (GPUs). Nvidia’s GPUs are generally seen as the industry standard for running AI models, but early results show that LPUs can blow them out of the water.
Groq is an “inference engine”, not a chatbot like ChatGPT, Gemini, or Grok. It helps these chatbots run incredibly fast but doesn’t completely replace them. On the Groq website, You can test different chatbots. And see how fast they run using Groq’s LPUs.
Groq generates 247 tokens/second compared to Microsoft’s 18 tokens/second, according to a 3rd party test by synthetic analysis Published last week. This means ChatGPT can run up to 13x faster than if it were running on Groq’s chips.
AI chatbots like ChatGPT, Gemini, and Grok could be significantly more useful if they were faster. A current limitation is that these models cannot retain human speech in real time. Some delays make conversations feel robotic. Google Recently faked his Gemini demo To make Gemini look like it can have multimodal conversations in real time, even though it can’t. But with Grok’s growing momentum, that video could become a reality.
Prior to Groq, Ross co-founded Google’s AI chip division, which developed cutting-edge chips for training AI models. With LPUs, Ross says Groq bypasses two LLM bottlenecks that GPUs and CPUs get stuck on: Compute density and memory bandwidth.
The name comes from Grok A stranger in a strange land1961 science fiction book by Robert Heinlein. The word means “to understand deeply and intuitively.” This is why many AI companies are using it to describe their AI products.
There’s not just Ross’ grok and Elon Musk’s grok, but one An AI powered IT company called Grok.. Grimes is one too The AI-powered toy, Grok, It’s supposedly named after the way he and Musk’s kids say “groket.” However, Ross claims that his Grok was the first in 2016.
“Welcome to Grok’s Galaxy, Alvin,” said November Blog post From Ross, three days after Elon Musk released xAI’s version of Grok. “You see, I’m the founder and CEO of a company called Groq™,” Ross said, confirming that Groq is a trademarked name.
While Groq is getting a lot of buzz, it remains to be seen if its AI chips have the same scalability as Nvidia’s GPUs or Google’s TPUs. AI chips are a major focus these days for OpenAI CEO Sam Altman, who is even considering making them himself. Groq’s increased chip speed could leapfrog the AI world, opening up new possibilities for real-time communication with AI chatbots.