We analysed the next generation of AI rack hardware, component by component.
Including the finding that is inconvenient for us. Coral's customers are about to make ten-year infrastructure decisions against a hardware generation whose thermal requirements are not widely understood. This is Coral's published component-by-component analysis, including the finding that current rack-scale systems are not immersion-compatible at the complete-system level.
Written to be useful to people who will never buy from Coral.
Coral's customers are about to make ten-year infrastructure decisions against a hardware generation whose thermal requirements are not widely understood, and the trade coverage has been imprecise in ways that lead to expensive mistakes. This is Coral's published analysis. It is deliberately written to be useful to people who will never buy from Coral.
What is actually specified, and what is not
| Parameter | What is specified | Evidence |
|---|---|---|
| Accelerator thermal design power | 1,800 W in the low-power profile to 2,300 W in the high-power profile, software managed. This split originates in supply-chain research rather than a published vendor specification. | Reported, not a vendor specification |
| Rack configuration | 72 accelerator packages, 144 GPU dies, and 36 host CPUs per rack | |
| Memory | 288 GB of HBM4 per package across eight stacks, up to 22 TB/s per accelerator. Memory suppliers have reportedly struggled to hit the target, so early parts may land nearer 20 TB/s. | with a supplier caveat |
| Compute tray cooling | 100 percent liquid cooled. The previous generation was a hybrid of roughly 85 percent liquid and 15 percent air. There is no air-cooled configuration of this generation. | |
| Coolant temperature | Up to 45 C inlet, leaving the chips at roughly 55 C. Coolant is 75 percent water and 25 percent propylene glycol. | |
| Cold plate design | Microchannel, with channel pitch reduced from 150 micron to 100 micron | |
| Thermal interface material | Not publicly settled. Credible published analyses disagree on whether the final design uses a liquid metal indium interface over gold-plated copper or a graphite interface. | Conflicted |
| Power delivery | Four 110 kW power shelves at N+1, each holding six 18.3 kW power supplies. Liquid-cooled busbars supporting up to 5,000 A, against 2,900 A in the previous generation. | |
| Rack power | Conflicted. Published figures range from approximately 120 kW to over 250 kW for nominally the same system. | Conflicted |
| Facility power | 800 V DC to the rack, stepped down by DC-DC shelves to 50 V DC at the compute tray | |
| Tray assembly | Cableless. Modules connect through board-to-board connectors onto a central printed circuit board midplane. Assembly time falls from about two hours to about five minutes. | |
| Successor generation | 600 kW per rack, 576 GPU dies across 144 packages, compute trays rotated 90 degrees into a vertical blade form factor with a dedicated power and cooling sidecar, second half of 2027. | |
| Immersion cooling | No immersion reference architecture has been published for this generation. The word does not appear in the vendor's own liquid cooling material. | Verified by absence |
Why published rack power figures differ by more than 100 percent, and what to build to
| Published figure | Coral's assessment |
|---|---|
| Approximately 120 to 130 kW | Low credibility for a high-power deployment. This figure appears to date from pre-launch expectations and is inconsistent with a 2,300 W accelerator. |
| 180 to 220 kW total rack thermal design power | Highest credibility. Consistent with the arithmetic, and consistent with four 110 kW power shelves at N+1. |
| Exceeds 250 kW | Plausible as a ceiling for a fully populated high-power rack with all networking and storage in the same envelope. |
| 600 kW | Not in dispute for the 2027 successor generation. |
Coral's planning recommendation is to size thermal capacity for 220 kW per rack sustained, with provision to 250 kW, and not to plan against the lower published figures. The arithmetic is straightforward: at the reported upper thermal design power, the accelerators alone account for approximately 165 kW before hosts, network interfaces, switching, power conversion or data processing units are counted. A pod designed for 130 kW that meets a 200 kW rack is not a product with a performance shortfall. It is a product that cannot be sold at all.
Can you immerse a next-generation AI rack? No. Here is the component-by-component reasoning.
| Component or configuration | Verdict | Reasoning |
|---|---|---|
| Accelerator package | Physically marginal, practically no | A 2,300 W package cannot be cooled by single-phase dielectric convection at 45 C fluid without a heat sink structure that defeats the density argument. The package also ships with a microchannel cold plate already bonded through a metal thermal interface. Removing that assembly to immerse a bare die voids the thermal design and almost certainly the support position. |
| Host CPU | Same conclusion, lower severity | Lower thermal design power and therefore more tractable, but it sits on the same tray under the same cold plate assembly and cannot be separated from it. |
| Networking and switch silicon | No, for the reference tray | The switch carries the same aggregate bandwidth in half the ports at double the rate, which concentrates the thermal load. Copper cable cartridges and their connector shells are a known wicking and contamination risk in dielectric fluid. |
| Power delivery and voltage regulation | Yes, and this is the strongest case for immersion | Board-level power delivery heat is distributed, awkward to cold-plate, and was the residual air-cooled fraction in the previous generation. A dielectric bath handles exactly this class of load well. |
| Memory | Not separately addressable | HBM stacks sit on-package under the same cold plate as the logic die. Immersion cannot reach them independently. |
| Optical and co-packaged optical components | The hardest constraint, and publicly unresolved | Optical engines are far more temperature sensitive than logic, laser wavelength drifts with temperature, and fibre terminations, index matching and connector cleanliness all have environmental requirements that a hydrocarbon or ester bath does not obviously satisfy. No vendor has published immersion compatibility data for co-packaged optical engines. |
| Complete-rack immersion | No | Seven independent blockers. See below. |
| Complete-rack immersion of the 2027 successor | No, and further away than the current generation | A vertical blade architecture with a power and cooling sidecar is mechanically incompatible with a horizontal immersion tank, and 600 kW in a bath is not a thermal problem, it is a fluid inventory, buoyancy and containment problem. |
| Partial liquid cooling | This is the vendor default and it works | Direct-to-chip on everything that matters, with the residual air fraction eliminated entirely at the tray in this generation. |
| Hybrid direct-to-chip plus immersion | Viable, and the correct answer for non-rack-scale hardware | Cold plate or two-phase evaporator on the die, dielectric bath on the board. |
Coral's published position is that current-generation NVIDIA rack-scale AI systems are not immersion-compatible at the complete-system level, and that the 2027 successor architecture is further from immersion compatibility, not closer. Seven independent blockers apply, any one of which is sufficient: the compute tray ships with cold plates already fitted; a busbar carrying up to 5,000 A has no published immersion electrical safety case; the cableless board-to-board midplane is not designed for fluid ingress; co-packaged optical component compatibility with dielectric fluids is publicly unresolved; the vendor publishes no immersion reference architecture; the five-minute tray service event the vendor engineered for is destroyed by fluid handling; and no long-term compatibility data exists for the thermal interface material stack in dielectric service.
Coral publishes this despite being widely described as an immersion cooling company, because it is what our own research found and because our customers will discover it regardless. It is also why Coral is not an immersion cooling company. A vendor whose entire business rests on a single thermal technology cannot tell you where that technology stops working. Coral's three thermal architectures exist precisely so that the answer to "what should we do about the next hardware generation" is an engineering question rather than a sales question.
Where immersion still wins, decisively
The majority of sovereign, hospital, university and enterprise AI hardware is not a rack-scale integrated system. It is PCIe and eight-GPU class, and that hardware is immersion-viable and two-phase-viable today.
Board-level power delivery heat is the load a dielectric environment handles better than anything else, and in the previous hardware generation it was roughly 15 percent of tray heat load handled by air. That air path is what forces fans, dust ingress, acoustic noise and filtration into an otherwise liquid-cooled system. Removing it is worth a great deal in a hospital, a school or a naval facility.
The open question Coral cannot answer alone
Can co-packaged optics be immersed? Nobody has published an answer.
As the dominant networking roadmap moves to co-packaged optics, this becomes a hard boundary condition on any immersion architecture. Coral's position is that it is unresolved, that it is a first-order risk to any pure-immersion strategy, and that Coral is actively seeking collaboration to answer it.
This analysis is maintained. Version 1.0, 20 August 2026. Coral republishes it whenever a material specification changes, and archives every prior version.
Request an architecture review.
A structured technical assessment of power, thermal capacity, density, jurisdiction and deployment sequencing for a specific site.