Google makes Gemini’s Pathways runtime generally available on Cloud TPUs

The current release is generally available on Google Kubernetes Engine, Google's managed Kubernetes platform, in regions supporting TPU v5e, v5p, Trillium v6e and Ironwood TPU7x. Google provides versioned containers supporting JAX 0.10.1 and later. Its Cluster Director integration for Slurm, another cluster workload manager, remains in preview.

This is also part of a wider hardware and software push. A report on the accompanying TPU announcements describes Google's new TPU 8tx system for confidential workloads and TPU 8i Max for memory-intensive work, alongside TPU Sync software and GKE dynamic slicing. Those products address different bottlenecks. Their advertised improvements should not be treated as Pathways benchmarks, and the Pathways support list above does not establish support for the newly announced eighth-generation systems.

Google's architecture diagram compares default multi-controller JAX with Cloud Pathways' single client coordinating multiple TPU slices. Image: Google. Source: Google Cloud.

Coordinating work without changing the physical limits

In Google's technical explanation, conventional single-program, multiple-data execution gives each accelerator host its own copy of the program. The workers synchronize through collective operations. Google acknowledges that this approach works well for a uniform job that fills its allocated hardware.

Coordination gets harder when stages run different programs or need different numbers of chips. In reinforcement-learning post-training—the refinement of a model after initial training—a sampler generates outputs, a reward model scores them and a trainer updates the model. Running those stages sequentially leaves one stage's chips waiting for another.

The network also constrains scaling. Google's example contrasts a TPU v5p chip's 1,200 GB/s of bidirectional bandwidth within its tightly connected group with 50 Gb/s to the datacenter network. Those are different links with different bandwidth scopes, not measurements of Pathways speed. Moving more communication between groups can limit the benefit of adding chips.

Pathways lets one Python process on a CPU machine describe the computation and its dependencies. A resource manager maps virtual devices onto physical TPUs; the distributed runtime transfers and redistributes tensors between stages and schedules operations in a consistent order. Asynchronous dispatch overlaps preparation for later work with computation already running. Researchers still choose how to partition their models.

Google also offers Python sidecars on TPU hosts for data loading and checkpointing, plus a shared service through which clients borrow slices—groups of tightly connected chips—from a persistent pool. A single client gives developers one place to inspect the job. Neither virtual devices nor sharing creates capacity: when the pool is full, new work waits.

The relevant alternatives operate at different levels. Google's explanation describes Ray, an open-source framework for coordinating Python AI workloads, as complementary: it could manage the broader pipeline while Pathways handles execution underneath. The post describes that combination as possible in principle, rather than demonstrating it.

There is also a broader choice about hardware dependence. Anthropic, the Claude model developer and a Google TPU customer, says in its October 26 IPO registration statement that its compute architecture spans AWS Trainium, Google TPUs, NVIDIA GPUs and AMD GPUs to optimize price and performance. That is not a like-for-like Pathways benchmark, but it illustrates a different procurement strategy. The announced Cloud Pathways configuration supports JAX on Google's chips; it does not establish portability across accelerator vendors or support for existing PyTorch programs.

What usage and recovery figures actually establish

Google says customers used more than 130 million Cloud TPU chip-hours through Pathways on GKE over the preceding 90 days, across several TPU generations. It separately describes an unnamed consumer technology company using more than 100 million chip-hours per quarter for production batch and reinforcement-learning inference across tens of thousands of chips.

The figures establish substantial reported use, not model quality or useful work per dollar. Their time windows differ, and Google does not disclose enough detail to derive customer concentration or generation-adjusted capacity from them. It says the unnamed customer also helped design elasticity features for expansion into pre-training.

The named customer is ZenteiQ, a developer of scientific foundation models using multi-slice Trillium clusters for elastic pre-training and reinforcement learning. Its founder's account in Google's announcement describes less manual supervision and little stall time. It supplies no measured before-and-after comparison.

Google's November 9 release notes explain the recovery mechanisms: scale down to healthy TPU slices after a failure and scale back up when replacements rejoin; save and restore accelerator memory through Cloud Storage during planned GKE preemption; or remap work onto standby or lower-priority slices. Avoiding a full job restart still leaves recovery overhead and potentially fewer chips available.

More detailed recovery results appear in Google's July 2025 Gemini 2.5 technical report. It describes training across multiple 8,960-chip TPU v5p pods in multiple datacenters, with hardware failures interrupting the run several times an hour. Google says slice-level elasticity reduced each interruption's lost training time to tens of seconds, compared with at least 10 minutes waiting for healthy machines without elasticity. Training continued at around 97% throughput while the failed slice recovered.

Across the run, Google reported that 93.4% of time went to TPU computation. The remainder was split approximately equally between elastic reconfiguration and rare cases where elasticity failed. About 4.5% of computed steps were replays or rollbacks for model debugging. Computation time therefore is not the same as net training progress, and these internal results are not a customer service guarantee.

A smaller demonstration adds another limit. The earlier rollout report describes Google's July 2026 test on 48 TPU v5e chips, training qwen3-0.6b with MaxText. After a deliberately terminated TPU, the next training step took about one minute and 50 seconds; the run lost 88 steps and returned to a checkpoint. That test used a different setup from Gemini's production run, so its recovery time is not a directly comparable performance result. It does show why recovery should not be read as uninterrupted progress.

Company-reported Gemini 2.5 training figures: 93.4% of run time involved TPU computations; around 4.5% of computed steps were replays or rollbacks for debugging. Different denominators; the percentages are not additive. Graphic: Magica; source: Google. Source: Google Gemini 2.5 technical report.

The next decision needs a cost per completed workload

For customers already using JAX on Cloud TPUs, general availability makes Google's internal execution approach a more concrete deployment choice. But the launch provides no like-for-like customer comparison on the same hardware and workload, or quantified reduction in a customer's bill.

The next useful evidence would measure completed work against total cost: throughput after recovery losses and repeated steps, idle capacity, and migration and operating effort. Those measurements would show whether Pathways' coordination benefits outweigh the cost of adopting its deployment model for a particular workload.