AI infrastructure investments by enterprises are outpacing their ability to measure costs, creating a significant compute gap.
A recent study of 107 enterprises reveals that AI infrastructure spending is accelerating significantly, far outpacing the organizations' ability to assess or manage the associated costs. While many enterprises rely on established hyperscalers and model-provider APIs, there is a marked shift toward investing in specialized compute solutions that are currently underutilized.
More than half of the surveyed enterprises plan to switch or add AI service providers within the next year, with many making decisions based on integration and total cost of ownership rather than just the initial price of tokens. This focus on comprehensive economics is critical, especially since most organizations lack clear visibility into their unit economics. Currently, GPUs are often running at less than half capacity, and fewer than 50% of enterprises rigorously track their actual compute costs.
This disconnect between investment and measurement has created what can be termed a compute gap—rapid investment without the necessary oversight to optimize it. As organizations ramp up spending to harness AI capabilities, the challenge will be to develop robust frameworks for tracking and managing costs effectively. Without this, enterprises risk overspending on infrastructure that may not yield the expected returns.
Source: VentureBeat AI