The AI Sky Above the Clouds

By Assaf Araki

Where Sky Computing and Neocloud Meet

The AI data center market is currently the primary engine of infrastructure growth, valued at approximately $471.59 billion as of 2026. While traditional cloud data centers still support a vast array of legacy applications, the shift in investment is stark: AI-optimized facilities are growing at a CAGR of 27.5%, nearly double the 14% rate of the broader cloud market. This surge is driven by a massive capital expenditure cycle from hyperscalers like Microsoft and Google, who are pivoting their budgets toward the high-density power and GPU-intensive hardware required for generative AI.

By 2030, the landscape will be dominated by AI, with specialized infrastructure expected to represent over half of all data center workloads. While cloud service providers (CSPs) historically focused on general-purpose compute at lower power densities, the new AI-ready facilities require massive upgrades across multiple dimensions, including compute, memory, storage, network, cooling, and energy delivery, and all vertically integrated with dedicated AI SW infrastructure. It opens the door for an AI-specialized CSP and creates competition between neoclouds and general-purpose CSPs.

The time for an AI-specialized CSP

For CSPs, the primary distinction between AI and general cloud workloads lies in their resource intensity and hardware architecture. General cloud workloads, such as hosting web servers, databases, or enterprise applications, are typically "CPU-bound" or "I/O-bound." Another difference in networking and memory is in the shift from latency-sensitive general workloads to bandwidth-hungry AI clusters. Traditional cloud applications rely on standard memory and "North-South" Ethernet traffic to handle independent user requests across distributed virtual machines. In contrast, AI workloads require High Bandwidth Memory (HBM) integrated directly onto the silicon to feed data to processors at terabytes per second. To maintain synchronization across thousands of GPUs, AI infrastructure utilizes "East-West" traffic over specialized fabrics like InfiniBand or RoCE, allowing the entire data center to function as a single, tightly coupled supercomputer rather than a collection of isolated servers.

Physically, AI workloads are forcing a radical redesign of data center power and cooling architectures. While a standard cloud rack typically draws 5kW to 15kW, an AI rack can exceed 100kW, demanding a shift from traditional air cooling to advanced liquid cooling (Direct-to-Chip or Immersion) to manage extreme heat densities. Furthermore, AI storage must evolve from general-purpose block storage to high-throughput parallel file systems that can stream petabytes of training data without "starving" the processors. These requirements result in specialized "AI zones" characterized by reinforced flooring for heavier, liquid-cooled hardware and shorter, high-speed cabling to minimize signal degradation between nodes.

All general CSPs like AWS, Azure, and GCP are redesigning their data centers to support AI workloads. AWS' Project Rainier is for high-density AI workloads, focusing on massive scale, custom silicon, and advanced cooling. Azure is re-engineering its data centers from the ground up to support AI, moving from a "cloud-first" to an "AI-first" design architecture. GCP is transitioning from custom builds to modular, standardized, and AI-native "factories." This change focuses on optimizing compute density, power efficiency, and specialized networking.

The Infrastructure SW that will lead the path

It's not just the hardware that's different - AI workloads require specialized orchestration, distributed computing engines, development software platforms and management tools as well. General CSPs operate on a "Swiss Army Knife" philosophy, offering a massive breadth of integrated services, from API gateway and serverless functions to relational databases, built on top of legacy virtualization layers. This SW stack improves developer productivity dramatically and creates a stickiness to the CSP, however it is not tailor-made for AI, and lacks specific AI services for SWE on the general platform.

AI CSP, also known as Neocloud at the core needs to support basic AI workloads like AI training, fine-tuning using Reinforcement Learning (RL), and AI inference. AI training, inference, and RL are distinct workload types with very different characteristics across data, compute, and latency. Training is an offline, batch-oriented process in which models learn from large datasets; it is compute-intensive, uses distributed GPU clusters, and runs for hours to weeks while repeatedly adjusting weights via backpropagation. Inference happens after deployment, where the trained model applies fixed weights to new inputs, running forward passes only, with no learning. Unlike training, individual requests are computationally light, but the workload is highly latency-sensitive and cost-sensitive, since it must serve continuous real-time user or system traffic. RL-based fine-tuning (e.g., RLHF or RLVR) sits between these two: it still needs training-class accelerators, but the loop couples environment simulation or user traffic, on-policy data collection, and short training bursts, which benefits from fast parameter servers, high-throughput logging, and flexible orchestration that can spin up and tear down both inference and training workers dynamically on the same cluster.

AI workload is not usually a stand-alone application; it resides within an application and also requires developer tools. The AI CSP needs to be a one-stop shop for AI developers and deployments to support the end-to-end application lifecycle.

The AI CSP landscape today and a look into the future

The race to build AI-specialized cloud providers is well underway. However, creating a full CSP requires enormous capital, effort, and time; we're seeing several distinct types of Neoclouds emerge, each starting from a specific problem. The first generation is often described as "GPU-as-a-Service" (GPUaaS) providers. These Neoclouds focus on delivering high-performance hardware and container orchestration SW with lower overhead than the big three hyperscalers. Most GPUaaS players, such as CoreWeave and Lambda Labs, were founded in the past decade and primarily focus on training workloads.

The second generation of Neoclouds realized that simply selling large GPU clusters to a small number of customers is not enough, and began layering software products on top of their specialized hardware data centers. A key difference is also who they target: some focus on data scientists and ML experts, while others cater to software engineers who had little AI experience before ChatGPT, or to different stages of the ML pipeline.

Along that pipeline, some players concentrate on inference, offering serverless GPU platforms that autoscale with demand and provide optimization to improve performance and reduce cost. Others focus on managing the RL process itself, providing RL environments to fine-tune GenAI models. The following is a short list of representative Neoclouds in this second generation.

• Baseten, founded in 2019 in California, mainly focused on Model APIs and inference in the early days of GenAI.
• Fireworks, founded in 2022 in California, began by offering high-speed inference so developers could serve open-source models with low latency and high throughput.
• Mithril (formerly MLFoundry), founded in 2022 in California, aggregates and orchestrates multi-cloud GPUs, CPUs, and storage.
• Modal, founded in 2021 in New York City, started by abstracting away GPU infrastructure complexity, allowing AI teams to focus on building products while running on generalized CSPs such as AWS.
• Prime Intellect, founded in 2023 in the US, a full stack for training and deploying self-improving agents: compute, environments, evals, RL post-training, and inference, with RL post-training as a core focus.
• RunPod, founded in 2022 in New Jersey, launched with a serverless platform on bare metal. They are a cloud compute service designed for software engineers running AI inference and other compute-intensive workloads without managing infrastructure.
• Together, founded in 2022 in California, launched as an AI training and inference platform.

Each Neocloud sources GPUs differently: some run on top of hyperscalers like AWS, while others require bare-metal infrastructure. Some buy and operate their own hardware, whereas others contract capacity from existing data centers or use revenue-share arrangements.

Although most of these providers started with a narrow focus, they are steadily expanding into full-stack AI development platforms, investing heavily in the software ecosystem to make it easier for engineers to manage the entire machine-learning lifecycle end to end.

The whole is greater than the sum of its parts

The rapid advancement of AI is pushing enterprises away from "single-cloud" or strictly "cloud-first" strategies and toward multi-cloud and hybrid-cloud models. This shift is driven by the need for tighter cost control, stronger data privacy, and specialized performance for intensive AI workloads.

Because AI workloads are unique and GPUs are scarce and expensive, Neocloud providers are being forced to support multiple heterogeneous data centers and to build an abstraction layer that hides this infrastructure complexity, effectively becoming a sky-computing platform. Sky computing is a cloud paradigm in which an abstraction layer enables applications to run seamlessly across multiple cloud service providers, treating cloud resources as a largely undifferentiated commodity and creating a "cloud of clouds" that resembles how the Internet unifies separate networks. In such a model, a software engineer writes the application once and can run it on many CSPs in a transparent way.

Sky-computing capabilities are becoming mandatory for Neocloud providers, and they open the door to new enterprise services that traditional CSPs struggled to deliver. Neoclouds can offer enterprises a cloud-like service running in their own on-prem data centers, with seamless burst capacity into public clouds when needed.

This approach reduces the cost and complexity of managing internal GPU clusters by exposing a unified API that can transparently spill over to external capacity when demand spikes. At the same time, Neoclouds can better address sovereign-AI requirements, as more countries, states, and large enterprises mandate that AI be developed, deployed, and governed within specific physical data-center boundaries.

An investor focus

The AI CSP market is only beginning to take shape, and the journey toward true full-stack AI cloud providers is still in its infancy. As the ecosystem matures, we expect cycles of consolidation and the emergence of higher-level software services from Neoclouds that simply don't exist today.

At Intel Capital, our focus is on backing teams that tackle the hardest technical problems in AI while also serving the long tail of newcomers to the field. That may sound paradoxical, deep, PhD-level AI work on one hand, and making AI accessible to software engineers who hadn't touched it before ChatGPT on the other, but we believe these two elements reinforce each other. Solving a hard technical problem and packaging it so that a broad base of engineers can use it is precisely what creates outsized disruption in this market.