There Is More Than One AI Data Center Market

Training favors scale. What gets built beyond that turns on who must control the compute, what workload runs on it, and where law and latency allow it to run.

Open models now do most of the work and collect almost none of the money.

One large commercial gateway for developer AI traffic reports that open weights took 56 percent of the tokens in August and 14 percent of the spending, up from under 10 percent of traffic the previous December. Those are Vercel's numbers. A second gateway, OpenRouter, is further along, with open weights above 70 percent of tokens by early October.

Closed providers still take roughly 96 percent of model-layer revenue, measured across the OpenRouter data examined for the 2025 State of Open Source AI report. Both gateways carry developer and AI-native traffic rather than enterprise production, and token counts flatter open models, since reasoning-heavy systems generate more tokens per request and the task mix skews toward low-value work. On these platforms the trend is unambiguous. Whether it reaches enterprise production is a separate question, and most of what follows is about why the answer is dependent on something other than price.

What the volume figures do not reflect is where any of that compute may sit. That question turns on who needs to control it, what kind of workload runs on it, and where law and latency allow it to run. Those three things determine what gets built. Model licensing can constrain what may run there, but it does not determine the physical infrastructure.

I. The Widening Gap

That volume shift has a straightforward explanation. The capability case for paying closed-model prices has narrowed considerably. Mozilla's September 2026 assessment put the best open-weight models roughly 4.4 months behind the U.S. frontier, and on its neutral evaluation harness closed models cost roughly five times as much per task. On Terminal-Bench 2.1, the difference between the leading models was about half a point. The frontier advantage grows as tasks get longer. Mozilla estimates the leading closed model at roughly a 12-hour 50-percent task horizon, meaning the task length at which it still succeeds half the time, with the performance difference considerably smaller below about eight hours and neither category performing reliably beyond twelve.

The market is reacting accordingly. Eight of the ten most-used models by token volume on Vercel's gateway were open weight, and seven of those eight were developed in China. Average per-token prices fell about 23 percent in August, the third consecutive monthly decline.

Enterprise behavior is developing differently, and two independent measures confirm this. Menlo Ventures, surveying 495 enterprise decisionmakers, put open-source models at 11 percent of enterprise LLM spend in December 2025, down from 19 percent a year earlier, a decline it attributes largely to Llama's stagnation. Menlo's figure is estimated dollars spent, weighted by production API usage.

Measured in volume rather than money, the line runs the other way. AI.cc, analyzing 2.4 billion API calls across more than 8,000 developer and enterprise accounts, put open-weight models at 38 percent of enterprise token volume in the first quarter of 2026, against 11 percent a year earlier. Enterprise Technology Research, surveying 200 respondents, put the figure at 34 percent in September 2026, up from 23 percent. Over roughly the same period AI.cc's blended enterprise cost fell from $18.40 to $6.07 per million tokens.

The market is therefore separating into three broad, overlapping consumption patterns, and a single organization may exhibit all three. The first is aggressive substitution of lower-cost open models for expensive closed ones, concentrated among developers and AI-native companies. The second, and much the largest, is continued consumption through managed APIs and cloud services regardless of whether the underlying weights are open or closed. The third is far smaller: organizations that want the model, the hardware and the security boundary under their own control.

It is the third group that creates new customer-controlled infrastructure.

II. Three Benefits That Are Regularly Confused

Open weights are employed to achieve one or more of three different economic benefits: lower cost, control over where the model runs, or an advantage that rests on information competitors do not have. When they are discussed as a single thing it obscures the fact that each implies a different buyer and a very different infrastructure requirement. A single organization can also pursue more than one, for reasons that have little in common.

Cost substitution

The first is straightforward: replace an expensive commercial model with an open-weight alternative that performs adequately at materially lower cost. This probably explains a substantial portion of the open-weight traffic growth now visible on commercial gateways, although the gateway data does not reveal why individual users selected a particular model. The important point is that substitution of this kind requires the customer to own nothing. The open model can still be served through somebody else's API on somebody else's infrastructure, which means open weights do not automatically imply self-hosting, and for most organizations they will not.

Possession and control

The second benefit is different in kind. By possession I mean operational control over the model, the data and the execution environment. It does not necessarily require owning the building in which it runs. An organization may want the weights, data and compute inside a security or jurisdictional boundary it controls, whether because of classified information, attorney-client privilege, regulated data, data-sovereignty requirements, resilience against vendor deprecation, or discomfort with allowing proprietary workflow information to accumulate at a third-party provider. Sovereignty is the largest of those motivations and the one the market has built a vendor category around, but it is a reason to want possession rather than a synonym for it. A law firm protecting privileged material is exercising the same mechanism for an entirely different reason.

Government and defense are the natural large buyers, though I have not seen reliable market data showing how large their share will ultimately be. What they need in most of these cases is a capable general model running somewhere the network cannot reach, rather than a model that knows anything a public model does not. Defense systems operating downrange in a contested environment are the clearest instance, where physical separation and local control are the entire specification. Most of what Palantir and Armada are selling into air-gapped deployment answers that requirement.

Commercial examples exist. Latham & Watkins has spent roughly three years developing its own Nvidia server infrastructure and now operates multiple H200 GPUs in a third-party facility under firm control. The firm has said that some client information is sufficiently sensitive that it does not want that information placed with a conventional cloud provider. That arrangement is the definition in practice. Latham controls the machines without owning the building. Latham still uses commercial AI products for ordinary workloads. For a buyer in this position, possession is not a cost-saving strategy. It is a premium deliberately paid for control.

Proprietary advantage

The third benefit is rarer and potentially far more valuable: employing proprietary information to produce an answer a competitor cannot obtain from the same public model. That is a change in capability rather than cost.

There are two forms of proprietary advantage, and that dichotomy tends to cause confusion. The first is context advantage. A general model is given access to records a competitor does not have, through retrieval, tool access, proprietary context or lightweight adaptation. The advantage resides primarily in the proprietary data and workflow around the model rather than in a self-reinforcing learning loop, and a competitor holding comparable records can reproduce it.

The second is compounding advantage, and it requires a further condition: a mechanism that eventually reveals whether the model was right. Where that verifier exists, outcomes feed back into the system. The model improves on a dimension competitors cannot observe, and the advantage compounds. Where an outcome verifier does not exist, the organization has context and little more. Both forms need proprietary records absent from general training data. Only the second produces a gap a competitor cannot close by assembling comparable records of its own.

Drug discovery is the clean commercial case for the compounding form. Published scientific literature records enormous amounts of what worked, while a pharmaceutical company's internal records contain enormous amounts of what did not. More importantly, experiments eventually produce results, so the organization receives a feedback signal against which models can be improved. Pfizer's January 2026 agreement with Boltz illustrates the form, with Boltz adapting its open foundation models using Pfizer's proprietary data to create Pfizer-specific models. That is qualitatively different from swapping one chatbot for another.

Government appears here as well, on a larger scale than any commercial example I have found. The Department of Energy's Genesis Mission carries more than $5 billion in federal commitments across 278 projects and 17 national laboratories. Its Genesis Open Models Initiative released a first model developed with Arcee AI and trained on the laboratories' own research output. Darío Gil, who directs the mission, described it as “…an open model trained on the real work of the national laboratories and answerable to the scientists who do it.”

How the program is structured is consistent with the value requirements for this sort of investment. DOE has opened the model to outside contributors on a rolling basis and is asking for two different things. The first is data. The other is environments, meaning instrumented settings in which a model proposes an answer and is scored on whether it was right. That second request is the verifier the compounding form depends on, and DOE is building it deliberately rather than receiving it as a byproduct of work done for other reasons.

The Genesis Mission names several national security problems that meet both conditions. Nuclear signature attribution, biological threat detection, and materials qualification rest on classified or experimental records that appear in no public corpus, and each eventually resolves against physical ground truth. The difference from the possession cases shows up in the building. An air-gapped drafting assistant needs a capable model in a place the network cannot reach. Attribution of a nuclear signature needs a model that knows things no public model can know. The first can be bought. The second has to be built.

BloombergGPT shows what the context form produces when the second condition is missing. Bloomberg assembled a corpus of roughly 708 billion tokens, including about 363 billion financial tokens, and trained its 50-billion-parameter model on approximately 569 billion of them. Subsequent comparisons found general-purpose frontier systems, such as GPT-4, competitive with or superior to specialized financial models across many public benchmarks. Financial text records what was said and what was reported far more readily than it records which judgments proved correct, in a form that a model can be scored against. That does not prove specialized financial models are pointless, but it does show that a large proprietary corpus without a verifier is a lead the frontier can close.

JPMorgan has pursued the context form deliberately. Its LLM Suite has been rolled out to more than 200,000 employees and provides an enterprise layer through which users reach AI capabilities without tying the organization permanently to one model. The organization holds the data, context, controls, and orchestration while allowing the underlying models to change. That pattern will be far more common than enterprises training their own frontier models, and it is why the context form matters even though a competitor could copy it.

As an order-of-magnitude framework, the addressable market breaks down roughly as follows:

Sizes are order-of-magnitude judgments based on the cited evidence, not third-party forecasts.

III. Infrastructure Follows the Workload

The second important distinction is that owning an open-weight model does not mean training one. For infrastructure purposes there are at least four materially different AI workloads, and they carry very different physical requirements.

Four workloads, scored on the attributes that decide where a site can sit and what it must provide.

Frontier pre-training

Frontier pre-training is the workload most people picture when they hear “AI data center.” It is enormously compute intensive, tightly coupled and concentrated in very large GPU clusters, where power density, fabric bandwidth and the ability to coordinate thousands or tens of thousands of accelerators matter more than proximity to end users. The buyer population is tiny. Very few enterprises will ever train a foundation model from scratch, whether the weights end up open or closed. That means frontier training is not the mechanism through which widespread enterprise adoption of open weights creates a new infrastructure market.

Post-training and fine-tuning

The middle ground is more interesting. Enterprises may adapt existing models through fine-tuning, reinforcement learning, distillation, synthetic-data generation, evaluation and periodic retraining without ever performing a frontier-scale training run. This will be far more common than enterprise pre-training. The resulting workload is episodic rather than continuous: large bursts of compute for hours or days, followed by long periods dominated by inference. That produces a very different infrastructure profile from a dedicated training campus. A useful way to characterize many enterprise environments is therefore mostly inference, with periodic post-training, and that profile fits far more naturally into the single-digit or low-double-digit megawatt range than into hyperscale training infrastructure.

Interactive inference

Interactive inference supports users or machines waiting for an answer, such as copilots, agents, voice systems, customer applications, search and operational decision support. Latency and availability matter, as do network connectivity, redundancy and the ability to keep operating through failures. That pulls some inference capacity closer to users, enterprise systems, and major network nodes. Interactive inference is therefore more geographically constrained than training in where each site can be located, and at the same time far more distributed in how many sites there are.

Batch inference

Not all inference is latency sensitive. Document processing, scientific analysis, scoring large data sets, synthetic-data generation, simulation and other machine-to-machine workloads can often wait, and may be scheduled overnight, moved geographically, or routed toward cheaper available power. From a power-market perspective some batch inference behaves more like training than like a user-facing chatbot. A relatively remote site with inexpensive power and adequate fiber may be unsuitable for interactive inference and perfectly attractive for batch inference, post-training, or other delay-tolerant workloads, which expands the pool of useful sites well beyond major metro locations.

Inference is not one infrastructure class

The term “inference” often glosses over some important distinctions. Large-model inference has at least two different computational phases. The first processes the prompt and context and is relatively compute intensive. The second generates tokens and is heavily influenced by memory bandwidth, cache behavior and how efficiently the system can keep model weights available. At sufficient scale those phases can be separated onto different hardware pools.

The architecture itself is not the most relevant factor. The point is that an inference data center is not a smaller version of a training center. Different inference workloads favor different combinations of accelerator, memory, network, storage, redundancy, and location. Large-model inference can itself be substantial in scale. I would avoid the tidy formulation in which training means an enormous campus and inference means a small modular site. The picture is more complicated.

IV. What Each Benefit Builds

Once workload type is separated from possession, the infrastructure consequences become clearer.

Cost substitution builds little customer-controlled infrastructure

A company switching from a closed model to an open model served through a commercial API does not need to construct a data center. The workload still consumes compute and still contributes to aggregate data-center demand, but the decision itself requires no new customer-controlled site, interconnection or dedicated power supply. This is why the headline open-weight share is misleading from an infrastructure perspective. A large majority of open-model growth can occur without producing a corresponding class of new data-center owners.

Possession creates smaller, buyer-controlled inference estates

Possession is different because maintaining a hard security or jurisdictional boundary eventually requires control over hardware somewhere. What the customer builds, though, depends more on workload than on model ownership. Most of these organizations will not build miniature training campuses. They will build or lease enterprise-controlled inference environments with enough additional capacity to support occasional fine-tuning and post-training. The resulting installations fall naturally into the single-digit to low-double-digit megawatt range.

Whether that estate is one site or several depends on the reason for possession. A single secured facility satisfies a firm protecting privileged material. Jurisdictional residency, resilience requirements, latency to dispersed users, and disconnected operation each push toward more than one. Possession determines that a site is needed. It does not by itself determine how many.

Nvidia and its partners have announced plans for roughly 25 pilot deployments of approximately 5 to 20 megawatts each at or near utility substations across five U.S. utilities. Those pilots establish that a major vendor is productizing the form factor and that utilities will site it. They are not a measure of what is driving the demand, and I would not read them as evidence of possession-led buying.

An explicit commercial expression of the same idea arrived on October 2, when Palantir named Armada its first Certified Modular Data Center Partner. The partnership is oriented toward environments where customers care about sovereignty, physical infrastructure and the location of their compute. Government and defense are the obvious applications.

Armada also illustrates why the category needs careful handling. The company was founded in late 2022 around computing in disconnected and difficult environments, and its original products are expeditionary: the suitcase-sized Beacon, the 20-foot Cruiser and the 40-foot Triton. It works through a cooperative research arrangement with NIWC Atlantic supporting U.S. Naval Forces Southern Command and U.S. Fourth Fleet. Its newer infrastructure products operate at a different scale, with Leviathan providing approximately 1.77 megawatts and Orion designed for 10 megawatts and as many as 2,880 GPUs across 40 racks in a prefabricated modular configuration. Nordic data-center operator Fossefall has separately ordered five Leviathan systems totaling roughly 9 megawatts. Armada therefore spans two adjacent businesses under one brand: expeditionary compute for difficult or disconnected locations, and modular fixed infrastructure for customers that want meaningful amounts of controlled compute near available power. The second is relevant to a broader commercial infrastructure market.

Proprietary advantage builds two different things

The context form builds very little that resembles a training facility. Retrieval-augmented generation, tool access, proprietary context, model routing, fine-tuning and relatively lightweight post-training against an existing foundation model will all achieve it. The more valuable asset is the data and workflow environment rather than a uniquely trained model, so what gets added is storage, indexing, connectors, and orchestration sitting alongside an inference estate the organization needed anyway. It changes the shape of the load more than the class of the facility.

The compounding form adds an important element that the context form does not. A verifier is itself infrastructure. Determining whether a model was right requires instrumented environments, simulation, evaluation harnesses and, in scientific settings, the laboratory apparatus that produces ground truth in the first place. Running a verifier at scale consumes compute in its own right. DOE's request for environments is a compute commitment described as a data contribution. That load behaves more like batch inference than interactive serving. It is schedulable, tolerant of distance and lumpy, which argues for the remote, inexpensive-power site rather than the metro one. An organization pursuing the compounding form may therefore need two kinds of capacity rather than one, and there is no reason to assume they belong in the same building.

The largest examples sit well beyond that. Eli Lilly and Nvidia announced an investment of up to $1 billion over five years in a joint AI co-innovation laboratory in the San Francisco Bay Area. Lilly separately inaugurated LillyPod in Indianapolis in February 2026, built around 1,016 Blackwell Ultra GPUs to support drug-discovery and scientific workloads. Roche has deployed 2,176 GPUs on premises across U.S. and European facilities, bringing its combined on-premises and cloud GPU footprint above 3,500. These are meaningfully large systems, and they represent the upper end of enterprise-controlled AI rather than the typical deployment. Proprietary advantage therefore creates a few large enterprise-controlled clusters, a much larger number of smaller inference-and-post-training environments, and, where the compounding form is in play, verifier capacity that nobody is currently counting.

V. Workload Determines the Value of the Site

This leads to a more useful infrastructure framework.

Possession determines whether a site is needed at all. Workload determines what infrastructure it needs. Advantage determines what the buyer will pay for it.

A buyer pursuing cost substitution is managing an expense line and will treat compute as a commodity to be sourced as cheaply as possible. A buyer pursuing proprietary advantage is protecting something harder to replace, whether a revenue line, a mission capability, or a research position. An organization seeking advantage will pay for reliability, proximity and control in ways a commodity buyer will not.

The same 10 megawatts carry very different value depending on the workload. A remote industrial site with inexpensive power and mediocre network latency is attractive for batch inference or post-training and unattractive for interactive enterprise agents. A highly reliable five-megawatt location close to enterprise networks and redundant fiber is more valuable for sovereign interactive inference. A several-hundred-megawatt site capable of supporting an extremely dense, tightly interconnected GPU fabric belongs in an entirely different market. These are not interchangeable assets based solely on the fact that all three host GPUs.

VI. Why the Small Tier May Matter More Than Its Size

No one expects enterprise-controlled compute to rival hyperscale demand. As a working planning range, I would estimate U.S. demand for this smaller controlled-compute category at something like 0.5 to 2 gigawatts over five years. That range follows from the estimates above: dozens to perhaps two hundred deployments building or leasing at five to ten megawatts each, plus a small number of larger scientific systems at the Lilly and Roche end. For scale only, BloombergNEF forecasts approximately 118 gigawatts of total U.S. data-center capacity by 2030. That figure is total installed capacity at a different endpoint, and mine is a five-year increment, so I would not reduce the two to a ratio.

Four requirements that do not improve as total capacity grows.

The strongest objection to all of this is that the hyperscalers absorb it. Each of the major providers already operates sovereign and in-region offerings, and every one of them is better capitalized, better staffed and far easier to buy than a private estate. If residency, auditability and contractual control prove sufficient for most buyers who believe they need possession, the demand described here never leaves cloud and the segment remains a curiosity. What that argument cannot reach is the buyer whose requirement is air-gapped operation, a network path no third party touches, or a location the provider does not serve. That population is smaller than the sovereignty rhetoric suggests and larger than zero.

What makes the segment interesting is not its aggregate size. It is that the assets serving it have a variety of characteristics, bought on different criteria and under different constraints than hyperscale capacity. Four of those features are most important.

The existing pipeline is not optimized for it

We are all acutely aware that the industry is financing and building enormous hyperscale campuses. The customer that wants two, five or ten megawatts of controlled, in-jurisdiction infrastructure has a different requirement. That supply exists, but anecdotally far less of the current pipeline is purpose-built for this middle ground than for conventional colocation or hyperscale campuses. If that observation holds, smaller energized sites become disproportionately valuable.

The interconnection problem may be different

Time is the binding constraint. Projects entering service in PJM have recently averaged more than seven years from queue entry to operation, though other regions move considerably faster, and lead times for large high-voltage transformers have stretched toward or beyond 160 weeks. Where a site brings its own generation, generator step-up transformers carry similar delays. A site that already holds an energized interconnection is therefore worth considerably more than it would have been a decade ago.

Regulators are working on it. On June 18 FERC opened proceedings across all six RTOs and ISOs under its jurisdiction, aimed at the treatment of large new loads and covering co-location, behind-the-meter generation and transmission service. Those proceedings create no single national threshold, and definitions and procedures still depend on the regional market and the individual utility.

This matters especially for projects in the single-digit to low-double-digit megawatt range, where existing distribution or substation infrastructure can sometimes accommodate a load without triggering the process required for a new hyperscale connection. The schedule advantage can be substantial. That conclusion should be evaluated utility by utility and RTO by RTO. There is no universal megawatt threshold that guarantees it.

Flexible workloads can make imperfect sites useful

The third difference concerns which sites can serve this tier at all. Training, post-training and some batch inference tolerate more scheduling flexibility than interactive inference. A training run can sometimes wait, a synthetic-data job can be moved, and a batch-inference workload can often execute overnight or in another region. A human waiting on an AI agent cannot. Some AI loads are therefore capable of following power rather than requiring power to follow the load. This difference can make sites economically useful that would never support a conventional hyperscale campus or a latency-sensitive inference facility. The value of a site depends not merely on how many megawatts it has, but on which workloads those megawatts can support.

The political footprint may be different

Data Center Watch counted more than 75 U.S. data-center projects worth roughly $130 billion blocked or delayed in the first quarter of 2026, with organized opposition active in 49 states. Those fights are almost entirely about large greenfield developments, and they turn on a recurring set of issues: land use and rezoning, water consumption, and the effect on other ratepayers.

A 5- or 10-megawatt deployment at an existing industrial or energized site can avoid many of the more common objections, particularly where zoning, power infrastructure and industrial use are already established. It can help keep a development as a permitting exercise rather than a political fight. That is a materially easier problem, though not an absent one. I have found no data measuring the difference directly, and it may never appear cleanly in national statistics, but it is the sort of advantage that shows up in schedule and cost well before it shows up in a dataset.

VII. An Asset That May Be Mispriced

The United States has roughly 55,000 electric substations. Research examining selected substations has found an average of roughly 5 megawatts of available capacity in some samples, with individual locations reaching approximately 20 megawatts. Separately, the U.S. electric-generation fleet operates at an average utilization well below full theoretical output, with figures around 53 percent commonly cited.

Those first two figures can’t be multiplied together. The substations examined were selected rather than randomly sampled. Available capacity varies enormously by location and voltage class, and a great deal of it sits where no one would want to put compute. The count establishes that the population is large. It says nothing about how much of it is usable.

A 53 percent fleet utilization rate does not mean the remaining 47 percent is available to data centers. Generation utilization is not the same thing as local deliverable capacity, and spare generation somewhere on a system tells us very little about whether a specific substation can serve a new 10-megawatt load. The combination does suggest some questions to ask. How many places already possess an energized interconnection, some amount of unused deliverable capacity, and land suitable for a relatively small compute installation? What kind of AI workload could economically run there? Add in closed industrial facilities, legacy generation sites, landfill-gas projects, small hydro, non-powered dams and other locations where power infrastructure already exists. All are potential candidates for the right workloads.

A location does not have to suit every AI workload to be valuable. It may only need to suit one. Some of these assets are probably worth considerably more to emerging compute buyers than their current owners realize. That is a hypothesis rather than a demonstrated asset class, and it is one worth testing.

VIII. The Model Split May Become a Workload Split

Open-weight model share will probably continue rising, and most of that growth will change very little physically, since models served through commercial gateways can become cheaper and more capable without changing who owns the building they run in. The more interesting infrastructure question sits inside a much narrower population: the organizations that decide they need the weights, proprietary data, compute hardware, and security boundary under their own control. Most of them will not build frontier training clusters. They will build enterprise-controlled inference estates with periodic fine-tuning or post-training layered on top, and within that category the physical requirement will depend on the workload. Interactive inference favors highly reliable, well-connected locations near enterprise users and networks. Batch inference and post-training tolerate more remote sites and can follow cheap or temporarily available power. A much smaller number of organizations, particularly in scientific fields, will justify genuinely large training and post-training systems.

That produces a different picture of the emerging market than the one usually described. Not one enormous new category of enterprise AI data centers, but a fragmented one.

Five megawatts here. Ten there. Twenty somewhere else. Different levels of latency, redundancy, density, and network connectivity depending on what runs inside.

I am not ready to call that a new data-center asset class. What the evidence does support is narrower and more useful. Possession determines whether a site is needed at all, workload determines what kind, and the two together explain why five megawatts in a remote industrial park and five megawatts near a metro fiber hub are not the same product even though the capacity is identical.

If that holds, a good many owners of small energized sites are holding an asset whose value turns on a question they have not been asked: not “how much power is there?” but “what could run on it?”

Christopher C. Flaesch is Managing Principal of Echelon Gateway Advisors LLC, advising enterprise technology leaders on AI value capture, go-to-market strategy, and partner ecosystems.