Inside the rapidly growing, and surprisingly narrow, AI infrastructure market

NVIDIA’s recent deal to license language processing unit (LPU) chips from Groq is shining a light on the LLM infrastructure ecosystem. Groq, which designs its own LPUs and runs models on its proprietary infrastructure, gives NVIDIA the opportunity to leverage the new chips and enter the market for model hosting and serving.

While Groq's chip production has received the lion's share of media attention, we’re more interested in the model hosting and serving piece. For an LLM to be used in production, it must be run somewhere (hosted) and be callable by other services (served). That makes this sector vital for AI to break into the mainstream.

Model hosting and serving companies make it easier for businesses to integrate any LLM of their choosing, including open-source models, into their product or workflows. These open-source models may be one critical source of competition. A robust, open-source alternative creates price and quality pressure on closed-source peers. Throughout the rest of this post, we will refer to model hosting and serving companies as AI infrastructure companies.

The rest of this report focuses on understanding the LLM infrastructure marketplace using Ramp’s data. Our main goal is to determine how the current market is structured and the implications it has for the market going forward.

A few of our high-level findings:

• The hosting and serving ecosystem is large and growing, but still a niche player that’s not yet putting pressure on closed-source companies.
  • Ramp customers spent $260M in Q4 2025 on AI infrastructure businesses, which is only 60% of what they spent on directly querying closed-source foundation model providers (OpenAI, Anthropic, Google, xAI, Perplexity).
  • Only a small subset of Ramp customers (~1,900) spend on AI infrastructure
• AI infrastructure spending does not yet appear to be substituting for open-source model spend.
  • Few customers are spending only on self-hosting AI infrastructure, with 94% of customers who spend on AI infrastructure also spending on closed-source models.
  • Company spending on AI infrastructure increases proportionally to spending on closed-source models. This suggests that open and closed-source models are not yet substitutes and may be complementary.

We have three distinct questions:

1. How big is the current market, and how fast is it growing?
2. Which companies lead in this space?
3. What services are companies using?

Breaking down the market

To understand this story, we first need to define the marketplace. Hosting and serving are often poorly defined, but we separate them into three distinct services:

1. Raw infrastructure bring your own model (BYOM) self-run

Providers offer general-purpose GPU infrastructure (VMs, bare-metal, or customer-operated clusters). Customers run and operate their own model stack, controlling weights, runtime, scaling, and lifecycle, and can freely train, fine-tune, or serve models using their own tooling. The provider delivers compute primitives, not a managed execution environment for models. Examples of vendors providing this service are CoreWeave and Nebius.

2. Hosted infrastructure bring your own model (BYOM) vendor-run

Vendors run customer-supplied models inside a managed execution platform. Customers upload model artifacts and may fine-tune or serve them via managed jobs or endpoints, while the provider controls the runtime, autoscaling, and infrastructure lifecycle. The service exposes managed APIs rather than raw compute primitives. Examples of vendors providing this service are Modal and Fireworks AI.

3. Proxy and router (serverless hosted models)

Providers expose a pay-as-you-go API to a catalog of hosted models they operate. Customers do not upload or manage model weights; they simply send inference requests to a serverless endpoint (often OpenAI-compatible). These services outsource infrastructure, scaling, and model operations entirely. An example of a vendor providing this service is OpenRouter.

You can see that the words “hosting” and “serving” don’t fully capture this complexity. A BYOM self-run service allows a client to host their models on infrastructure they control and then use that infrastructure to serve them to their own applications. This combines hosting and serving, but hosting is self-managed. A hosted BYOM service, on the other hand, simply provides a way for a customer to spin up preconfigured endpoints for serving models, with vendor-managed hosting.

The third type of service, proxy and router services, is interesting from a competitive standpoint. These are companies that allow firms to access a broader range of models (think open source) without having to understand model hosting and serving infrastructure. Reducing costs to access more models could create competitive pressure, leading to lower prices and higher-quality models. Additionally, if these services become dead simple to use, they may expand the market, enabling more companies to use LLMs directly.

How big is the market?

We conducted a comprehensive review of the AI infrastructure marketplace and identified 25 leading companies, specifying the services each provided. Here’s a list of which companies provide which service:

It’s important to note that there’s no unified definition of what an LLM infrastructure company is, so we may have missed a company. We are also unable to identify customer spending on GPUs and accelerator infrastructure from the major cloud providers (Amazon, Google, Microsoft).

We also included six closed-source foundation model providers as points of comparison (Anthropic, OpenAI, Google, xAI, Mistral, and Perplexity).

Below, we detail total spending, average per-business spending, and the total number of businesses spending on foundational models and AI infrastructure.

Total spend

Average spend/business

Number of businesses

The AI infrastructure market is large in dollar terms, but small in terms of the number of businesses. In Q4 2025, AI infrastructure spending on Ramp was about $260M, whereas foundation model spending was 60% larger at $430M. Growth rates in AI infrastructure spending are high (100%), but still lower than those for foundation model spending (126%).

While the total dollars spent on AI infrastructure are significant, the spending is coming from a small set of customers. Only 1,900 businesses spent on AI infrastructure in Q4, representing less than 9% of the 21,500 that used foundation model providers. This implies that companies that invest in AI infrastructure spend much more than those that invest in foundation models. On average, customers spent $142,000 on AI infrastructure, compared with $20,000 on foundation models.

Who are the market leaders?

To determine the players and leaders in the AI infrastructure space, we first look at Q4 2025 spend, where we again include foundation model companies for comparison. On a single-company basis, the foundation model providers are still far ahead of AI infrastructure companies. Together AI and Fireworks AI both hold significant market shares, with approximately 10% each of total spending on AI infrastructure or foundation models. Together AI offers raw infrastructure services through its Instant Clusters product, while Fireworks AI does not provide this type of service. Both companies outpace traditional data center competitors, with CoreWeave capturing 6% of the market and Nebius at 5%.

In contrast, vendors that only provide proxy and router services have a much smaller market share. OpenRouter, the largest provider in this category, accounted for less than 1% of the market share in Q4 2025.

The main takeaway: customers are spending heavily on vendors that are more than just proxy-only services. The market seems to be trending toward multi-service vendors where proxy services are only one among many AI infrastructure services.

Total spending may mask a vendor's market penetration. A vendor may have a few clients with large contracts. However, this business model creates risk; if only a few clients leave, then the business may be in trouble. That’s why, for the same level of revenue and cost structure, a business would usually prefer to have many small contracts versus few large contracts.

We plot the revenue per business (a proxy metric for contract size), sorted by total Q4 2025 spending. Companies that operate data centers such as CoreWeave ($1.7M/business), Crusoe ($2.5M/business), and Nebius ($560K/business) claim the highest revenue per business. While data center companies have fewer, larger contracts, this shows that some businesses that build and lease data centers operate differently from those that do not.

On the other end of the spectrum, OpenRouter has an average contract size of just $6,000. However, its 700 customers are the most of any AI infrastructure business.

Vendor-level statistics suggest that single-service-focused vendors attract a wider, smaller audience, whereas multi-service vendors attract a smaller, but more lucrative, customer base.

To better understand this pattern, we aggregate spending by whether a vendor is multi-service or single-service, and we break out the specific single service offered. Eighty percent of Ramp customer AI infrastructure spend goes to vendors that offer multiple services, indicating a strong preference for multi-service offerings.

Is AI infrastructure creating more competition for foundation model providers?

We have discussed the types of AI infrastructure services that companies are paying for, but is AI infrastructure putting pressure on closed-source foundation model companies (OpenAI, Anthropic, Google, etc.)? Are companies either self-hosting open-source models or using proxy services to query open-source models directly, thereby putting competitive pressure on closed-source companies?

The data definitively says companies are not yet substituting open-source for closed-source models. We identify substitution by looking at whether companies are spending exclusively on AI infrastructure or on both infrastructure and closed-source foundation models. A company that is only using open-source LLMs should be exclusively spending on AI infrastructure. However, the vast majority of companies are spending on both AI infrastructure and closed-source software.

While we don’t see any companies spending solely on AI infrastructure, you might hypothesize that some companies are spending substantially less on foundation models and more on AI infrastructure. The result is surprising: companies that spend on foundation models are also more likely to spend on AI infrastructure. In economic terms, foundation models and AI infrastructure appear complementary.

There are a few different explanations for this. One is that the rise of AI agents means companies are using infrastructure to orchestrate long-running jobs that are not necessarily self-hosted, i.e., using these companies as a pure serverless architecture. Another is that companies that use foundation models are more likely to have tasks for which GPUs can be used, i.e., the firm may need GPUs for ML and other non-LLM applications.

Conclusion

The AI infrastructure marketplace is large and growing. However, it is still a niche market, used by a small slice of the 50,000-plus companies on Ramp. While vendors offer diverse services, customer demand is geared toward hosting solutions. This marketplace could act as a competitive force against closed-source model providers, but it’s acting as the opposite. Whether these AI infrastructure vendors will become a stepping stone to unlocking open-source models across the industry remains an open question.