Additional Coverage:
Google Gears Up for AI Tsunami: Infrastructure Boss Warns of Massive Scaling Needs
MOUNTAIN VIEW, CA – Google is bracing for an unprecedented surge in AI product usage, with a top executive revealing the company needs to dramatically scale up its technological infrastructure to meet the anticipated demand. This push to expand “serving capacity” could also signal that widespread fears of an AI market bubble might be overblown.
Amin Vahdat, Google’s Vice President leading the global AI and infrastructure team, emphasized the urgent need to double the company’s serving capacity every six months, projecting a “1000x increase in 4-5 years.” This ambitious target, shared during a recent all-hands meeting, refers to Google’s ability to ensure its AI products, such as Gemini, and those relying on Google Cloud, can efficiently handle a rapidly expanding user base and increasingly complex queries. This is distinct from “compute,” which involves the physical infrastructure used for training AI models.
A Google spokesperson confirmed the heightened demand, stating, “demand for AI services means we are being asked to provide significantly more computing capacity, which we are driving through efficiency across hardware, software, and model optimizations, in addition to new investments.” The company pointed to its proprietary Ironwood chips as an example of internal hardware innovations boosting computing power.
In previous years, major cloud providers like Google Cloud, Amazon, and Microsoft Azure focused heavily on increasing compute capacity in anticipation of the AI boom. Now, according to Shay Boloor, chief market strategist at Futurum Equities, the users are here, and serving capacity is emerging as the next critical hurdle.
“We’re entering stage two of AI where serving capacity matters even more than the compute capacity,” Boloor explained. “Compute creates the model, but serving capacity determines how widely and how quickly that model can actually reach the users.”
Boloor believes Google, with its substantial capital investments and strategic development of its own AI chips, is well-positioned to achieve its goal of doubling serving capacity every six months. However, he cautioned that Google and its competitors still face significant challenges, particularly as AI products begin to handle more sophisticated requests, including advanced search queries and video processing.
“The bottleneck is not ambition, it’s just truly the physical constraints, like the power, the cooling, the networking bandwidth and the time needed to build these energized data center capacities,” Boloor added.
The sheer volume of demand Google is experiencing for its AI infrastructure, necessitating such rapid scaling, might also serve as a counter-indicator to gloomy predictions from AI pessimists. Concerns about a potential AI bubble contributed to a recent downturn in major stock indexes, including the tech-heavy Nasdaq.
“This is not like speculative enthusiasm, it’s just unmet demand sitting in backlog,” Boloor asserted. “If things are slowing down a bit more than a lot of people hope for, it’s because they’re all constrained on the compute and more serving capacity.”