General Compute lands $400M loan for inference chips

General Compute, an AI inference cloud startup, has secured a $400 million loan from Upper90, in what may be the first financing deal to use inference-specific chips as collateral.

Inference chips run already-trained AI models quickly and efficiently. That makes them different from the more expensive processors used to train models in the first place. The deal reflects growing market concerns about the cost of AI tools and tokens, while increasing attention turns to infrastructure that can run open-source models more cheaply than the newest large language models from frontier labs.

General Compute’s inference strategy

Founded by CEO Finn Puklowski, General Compute raised a $15 million seed round in May to build a “neocloud” around silicon from SambaNova, an Intel-backed chipmaker. Unlike traditional hyperscalers such as AWS and Azure, neoclouds are designed specifically for AI workloads.

The company plans to use SambaNova’s SN50 chips, which are built for inference. General Compute says they are power-efficient and do not require expensive water-cooling systems, allowing them to be deployed more quickly than GPUs across a wider range of data centers. The startup claims the chips will deliver 16 times faster inference than GPU-based clouds.

Recommended reading

xAI Sues Grok User Over Sexual Deepfakes

The immediate challenge is acquiring enough hardware as a new company. Upper90 co-founder and CEO Billy Libby, a former Goldman Sachs quantitative trader, had already developed a financing model for that problem.

In 2021, Upper90 financed GPU purchases by Crusoe, the energy-focused data center startup. Libby believes it was the first loan secured against the value of advanced chips. Traditional lenders had avoided such deals because of uncertainty around GPU depreciation. As CoreWeave turned chip-backed loans into a business model and later the foundation of a blockbuster IPO, the financing structure became more common.

“When we financed Nvidia GPUs as the first group to do that, the market was inefficient. We could really put together something as an early participant, and kind of get compensated for the risk.”

Open models drive demand for alternatives

With GPUs now comparatively well understood and possibly overbought, Upper90 is looking toward the next phase of AI infrastructure. Libby said the firm began searching last year for a company focused on inference because it expects open-source models to become increasingly important.

“Everyone doesn’t need a supercomputer, but they do need inference and AI.”

That thesis is gaining support from companies such as OpenRouter and Fireworks, which provide access to open models and have raised new rounds at high valuations. New models, including Kimi’s K3, have recently competed with releases from Anthropic and OpenAI on coding benchmarks. Chipmakers such as Groq and Cerebras have also attracted interest from acquirers and public markets.

General Compute’s access to chips outside Nvidia’s ecosystem is central to the bet. TensorWave is pursuing a similar strategy through a partnership with AMD. As more alternatives emerge, compute providers not locked into Nvidia agreements could gain an advantage in delivering lower-cost inference.

“There are a bunch of chips that are starting to scale that have amazing [total cost of ownership], or that can operate much faster than Nvidia, but there’s not too many buyers for them. By getting together with Upper90, this is not just, 'a cool startup got some money to buy some compute.' Like, this is the first signal of capital organizing itself and the fragmenting of Nvidia’s monopolistic dominance.”