Google TPU 8tx and 8i Max focus on AI security and memory

Google has added two systems to its eighth-generation TPU portfolio: TPU 8tx and TPU 8i Max. Announced on November 9, 8tx targets secure AI workloads involving sensitive data, while 8i Max addresses the memory and computing demands of agents working with very large contexts. Google has now announced four eighth-generation TPU systems in a single year.

Two systems for different requirements

TPUs are Google’s AI accelerators, originally developed more than a decade ago for internal needs such as voice recognition and translation. Their uses now include training and serving Gemini, agent workloads, Google Cloud services and deployments in customers’ own data centres. Google says the business has nearly 100 customers, including frontier laboratories, trading firms, retailers and universities.

At Google Cloud Next in April, TPU 8t was introduced for large-scale training and TPU 8i for real-time reasoning. Google reported up to 2.7 times the training performance for 8t and 80% better inference price-performance for 8i compared with the previous generation. Google also describes 8t as maximising useful work per watt and 8i as reducing inter-chip communication delays by up to half to address the memory bottleneck. These are the company’s comparisons, not a promise of identical results in every application.

The announcement distinguishes a new chip from a new system. Designing silicon can take years, while combining existing chips with different packaging, memory, networking and security components can address emerging needs more quickly. Google says the modular architecture of its AI Hypercomputer supports this approach.

TPU 8tx protects data while it is in use

TPU 8tx uses the same silicon as TPU 8t and pairs it with fifth-generation AMD EPYC CPU hosts. It is designed for regulated organisations and governments running proprietary or frontier models and agents alongside sensitive data. The aim is to protect and isolate data and model weights at the hardware level while they are being processed, rather than only when stored or transmitted.

Google is using its low-latency Boardfly network, developed for real-time reasoning. According to the company, latency can be reduced by up to half compared with TPU 8t, with a near-zero performance penalty for the security layer. Those figures remain vendor claims and depend on the workload and comparison conditions.

The system also supports small-scale training. Organisations can fine-tune models with private data, distil larger models into smaller ones and train agents through reinforcement learning. Combining these activities with inference can reduce the need to move sensitive data between separate clusters.

TPU 8i Max combines two chips in one package

TPU 8i Max combines two low-latency TPU 8i chips in a single package. Google says it doubles compute, SRAM and high-bandwidth memory bandwidth relative to TPU 8i. This is intended to keep processors supplied with data when groups of agents work across an entire software repository or hours of video.

Fitting more silicon within the existing rack footprint increases compute density per unit of data-centre space. Google also points to more computing power per watt. Doubling component resources does not mean every application automatically runs twice as fast; software and workload characteristics still matter.

Six strands of software development

The hardware announcement is accompanied by open software tools. TorchTPU, open-sourced last month, allows PyTorch code and Hugging Face models to run natively on TPUs, with ready-to-use backends for inference engines including vLLM and SGLang. The open-source TPU agent suite, currently in preview, integrates with coding tools such as Claude Code, Cursor and Google Antigravity to support onboarding, bottleneck diagnosis and custom-kernel tuning. The new XLA Serving Engine is designed to serve Python-trained models in a low-latency C++ runtime across TPUs, GPUs and CPUs.

Google reports strong price-performance results for its native vLLM–TorchTPU stack in InferenceX and AgentX benchmarks. The evaluations it cites focus on agentic coding and long-context workloads rather than theoretical peak chip speeds. Results need to be considered alongside the models and serving software used, rather than treated as a universal ranking.

Agent Compute Engine can suspend agent sessions waiting for a database response, external tool or human feedback, moving working memory off the chip in milliseconds. Its purpose is to reduce the operational cost of computing resources that would otherwise remain idle during those pauses.

Google is also open-sourcing TPU Sync, a model-state layer that coordinates chip HBM, host DRAM, shared storage and the data-centre network. It tiers large KV caches, streams context between separate prefill and decode nodes, and synchronises reinforcement-learning weights. Google reports reductions of more than 90% in KV-transfer latency and time to first token, a company measurement that should be understood in its particular testing context.

Cloud Pathways is now generally available on Google Kubernetes Engine. It brings the distributed runtime approach behind Gemini to cloud customers, using a single controller and pathwaysutils to manage multiple TPU slices as if they were one local machine. Colocated Python sidecars, sub-minute slice hot-swapping and elastic resharding are intended to keep long-running jobs moving through hardware disruptions.

GKE dynamic slicing is also generally available. Using optical circuit switching, it can assemble TPU cubes into larger topologies or divide them into smaller, isolated workloads. Google says it can accelerate job startup by up to five times and fault recovery by up to 4.5 times. The objective is to separate physical provisioning from logical slice formation and reduce rigid, manual node-pool recreation.

How the four systems fit together

The resulting portfolio positions 8t for large-scale training, 8i for low-latency reasoning, 8tx for confidential computing and 8i Max for large contexts and high rack density. Google says its silicon, networking, compiler and DeepMind teams work together, and that it also uses AlphaChip and Gemini in hardware design. The announcement points to an approach in which organisations choose infrastructure by considering security, memory demands and software together, rather than comparing accelerators on a single headline speed figure.

Source: Google, November 9, 2026.

Dji Banner Inside Article - 728x90