In the dynamically advancing realm of artificial intelligence (AI), traditional GPUs have served as the primary workhorse for processing power. However, in response to an escalating demand for enhanced efficiency and specialized AI hardware, a wave of companies is surfacing with innovative alternatives poised to disrupt the prevailing dominance of GPUs. This article explores the underlying motivations propelling the quest for GPU alternatives, delves into the constraints posed by legacy GPU technology, and scrutinizes the intensifying competition arising from burgeoning startups within the AI hardware market.
Nvidia’s Dominance and the Pursuit of Alternatives
Nvidia’s strategic shift from gaming-centric focus to high-performance computing processing, particularly with GPUs, has proven immensely successful. The recent quarter saw record-high data center revenue of $14.5 billion, a 41% increase from the prior quarter and a staggering 279% growth from the year-ago quarter. Despite this success, the AI hardware market is attracting competitors eager to challenge Nvidia’s stronghold. Notable contenders include AMD, Intel, and a slew of startups such as SambaNova, Cerebras, GraphCore, Groq, xAI, among others.
The AI hardware market, projected to be $43 billion in 2022, is anticipated to surge to $240 billion by 2030, according to Precedence Research. This massive opportunity has led to the development of alternatives to GPUs, each offering unique solutions to the limitations posed by traditional GPU technology.
Limitations of Legacy GPU Technology
While GPUs have been instrumental in AI processing, they have inherent limitations that some view as impediments to further progress. Glenn O’Donnell, senior vice president and analyst at Forrester Research, highlights that the CPU, being a general-purpose processor, may not be ideal for dedicated AI processing due to unnecessary power consumption and circuitry usage. In response, various companies are exploring specialized chips optimized for specific AI tasks.
Google’s TensorFlow processor stands out as an example of this optimization. Built specifically for the tensor flow algorithm, it eliminates compromises found in general-purpose processors. However, the GPU, originally designed in the 1990s for 3-D gaming acceleration, shares similar efficiency concerns with CPUs. According to Daniel Newman, principal analyst at Futurum Research, the GPU architecture’s kernel model, performing one task at a time, requires intercommunication and disassembly of models, leading to inefficiencies.
Startups and their Claims of Superiority
Several startups are positioning themselves as providers of superior AI processing solutions. Rodrigo Liang, co-founder and CEO of SambaNova Systems, notes that while GPUs excel in general training, deficiencies arise when dealing with large models like GPT. Running thousands of GPU chips for such models becomes inefficient, prompting the development of alternatives.
James Wang, senior product marketing manager at Cerebras Systems, echoes these sentiments, emphasizing the size limitations of GPU chips. Cerebras’ Wafer-Scale Engine-2 (WSE-2) is presented as a solution, boasting 850,000 cores and 9,800 times the memory bandwidth of a GPU. The argument revolves around larger chips allowing for more efficient processing of large language models, reducing the number of required chips and subsequently lowering power consumption – a critical concern for processor-intensive AI workloads.
Elmer Morales, founder and CEO of Ainstein.com, underscores the early adoption of GPUs in AI and HPC due to their availability, akin to “low hanging fruit.” However, he contends that GPUs are too small for massive models, necessitating the splitting of models among thousands of GPU chips for processing. The pitch from GPU alternative vendors is clear – they claim to have built a better mousetrap, addressing the inefficiencies and limitations of traditional GPUs.
Ecosystems and Software Dynamics
In the pursuit of GPU alternatives, startups like Cerebras and SambaNova are not just chipmakers; they are complete system developers. Offering server hardware and a software stack to run applications, these startups compete with industry giants Intel, AMD, and Nvidia, known not only for their silicon but also for their significant software efforts around AI.
Historically, software ecosystems have played a crucial role, both supporting hardware and creating vendor lock-in. Nvidia’s CUDA platform is a prime example of a proprietary ecosystem that contributed to the company’s dominance. However, the landscape is shifting towards open-source software, allowing customers to choose hardware without being bound to a specific platform. This shift is viewed as beneficial for the industry, promoting flexibility and avoiding vendor or platform lock-in.
Ainstein’s Morales emphasizes the importance of choices for startups and customers, allowing them to use multiple vendors and avoid network lock-in. The industry is witnessing a shift towards open-source software, fostering an environment where the benefits are shared across the AI community.
Evolving Processor Designs and Programmability
Looking ahead, experts foresee the evolution of AI processing towards custom, programmable chips often referred to as “FPGAs on steroids.” Glenn O’Donnell predicts the emergence of chips with the programmability of Field-Programmable Gate Arrays (FPGAs) but on a grander scale. This programmability would enable chips to be repurposed for different AI models, offering consumers greater flexibility in choosing devices tailored to their specific needs.
Elmer Morales supports this vision, asserting that hardware vendors must provide programmable chips to meet the diverse requirements of AI models. The ability to repurpose chips for different models ensures that consumers have choices, allowing them to use devices for various purposes without being constrained by a specific model or platform. This shift towards programmability is expected to gain traction in the latter half of the decade.
Despite the ambitious pursuits of startups entering the AI hardware market, O’Donnell remains skeptical about their ability to dominate over industry giants like Nvidia and Intel. He anticipates that while some startups may find success within specific niches, the likelihood of them exploding onto the scene remains uncertain. However, he acknowledges the potential for acquisitions, where established players may seek to acquire startups for their intellectual property.