In August 2021, Rich Miller, Data Center Frontier’s Editor in Chief, hosted a panel discussion on liquid immersion cooling, a game changing technology poised to revolutionize data center cooling design, efficiency and sustainability.
The panel included Sandeep Ahuja of Intel, Sunlai Chang of Wiwynn Corporation, Phillip Tuma of 3M, Joe Capes of LiquidStack, and John Major of Page Southerland Page. These industry leaders shared visionary insight into the potential of this powerful technology. They discussed:
- The impact of liquid immersion cooling on data center energy use, costs, sustainability and design
- Practical advice on how data centers can implement liquid immersion cooling
- The evolutionary changes immersion cooling is driving in the data center and HPC ecosystem
Read on for a comprehensive summary of this illuminating discussion.
The Problems with Today’s Data Centers
In order to fully appreciate the true size and significance of Liquid Immersion Cooling’s potential impact as an emerging technology, it’s best to begin with a quick tour of a traditional data center.
The first thing you’ll likely notice when you step into a typical data center is the deafening roar of countless whirring fans, vents, pumps, and a litany of other things that move, make noise, and consume a whole lot of resources. The intensity of that roar varies depending on where you are in the data center, but they can regularly exceed 95 decibels (somewhere between a snowmobile and a commercial leaf blower — all day, every day).
But as unpleasant and potentially damaging as that sound can be to your ears, the larger issue is the resource consumption. According to Energy Innovation, a non-partisan energy and climate policy think tank, data centers consume just over 1% of all the electricity generated on Earth — the majority of which, as most of us know, is produced by burning fossil fuels. A potentially larger environmental concern, however, is the sheer volume of water consumed by data centers. Steam generated from boiling water is used to turn the turbines that ultimately create energy in fossil fuel and nuclear power generation plants. But air handlers, chillers, and other heat rejection equipment guzzle incredible amounts of water as well. In aggregate, the world’s data centers consumed an estimated 660 billion liters of water in the year 2020.
Now for the Good News: The Promise of 2-Phase Immersion Cooling
Liquid immersion cooling can take different forms, but for the purposes of this article we’ll be talking about 2-phase immersion cooling (2PIC) — the current gold standard for immersion cooling. In 2PIC, servers are placed in a hermetically sealed DataTank™ partially filled with dielectric (i.e. nonconductive) fluid. The heat generated by the server’s components gradually raises the fluid’s temperature until it reaches its boiling point. The heat is then rejected from the fluid bath in the form of a vapor gas contained within the DataTank (phase 2). There, using the incredible efficiency of phase change, the vapor gas condenses on the surface of specialized coils, which are filled with water (or a water glycol mix), supplied by, and returned to, Dry Coolers located outside the data center white space. The condensed liquid then falls back into the fluid bath where the entire process repeats. The process of rejecting the heat inside the DataTank itself consumes no energy whatsoever.
This closed loop also mitigates or eliminates the consumption of water for heat rejection. But the environmental benefits of 2PIC extend beyond that. By potentially using the waste heat generated from the servers themselves, 2-phase immersion cooling can actually contribute more energy (in the form of reusable heat energy) than what it consumes to reject that same heat in the first place. These two things alone are enough to make 2PIC not merely advantageous, but necessary. And this dramatic shift in the way we construct and cool the world’s data centers will have a ripple effect on many other aspects of computing.
How 2-Phase Immersion Cooling Will Change Computer Design
Believe it or not, immersing electronics in tanks of liquid requires fewer design overhauls than you might think. Because the fluids used are fluorinated and non-conductive, they don’t interfere with the electrical workings of IT hardware or erode computer components. Many of the fluids that 2-phase immersion liquid cooling leverages have other use cases as cleaning agents for semiconductors and printed circuit boards — these fluids are even used as fire suppressants in vast numbers of data centers today.
The best way to summarize the impact of 2PIC on computer design is that there will be very few necessary design changes. However, it will afford so many new opportunities for optimization that significant changes are all but inevitable. Immersion cooling allows IT hardware designers to double down on denser, more powerful computing. By eliminating the need for power-hungry server fans and bulky air-cooled heat sinks, 2PIC will propel the development of ultra-dense, ultra-efficient, ultra-sustainable and ultra-fast computing.
How 2-Phase Immersion Cooling Will Change Data Center Design
At the highest level, 2PIC allows data centers to be simpler, denser, and more efficient, packing far more server power into less space. And the metric of power density isn’t confined to the server level. The entire footprint of data centers can be dramatically reduced by 2PIC. For context on density, a 36MW data center that is air cooled can average 10kW per rack. A liquid-immersion cooled data center, meanwhile, can produce an average of 252kW per tank.
In terms of simplicity, designers simply have much less to think about. Air-cooled centers require consulting engineers to develop complex, full vision simulations of air circulation throughout the facility. It requires complex venting, flashing, and lots of strategically distributed open space for optimal air flow. Data centers using 2PIC require none of these things. So, it goes without saying that data center design will be faster and cheaper, with fewer design considerations and fewer materials required to build. From the elimination of traditional air-cooled IT cabinets to a significantly reduced need for copper and fiber cabling, the 2PIC data center will spare all types of expenses.
What’s Required to Convert a Traditional Data Center to a 2-phase Liquid Immersion Cooled Data Center?
First, you will need to replace your existing racks with DataTanks, which optimize your data center’s layout. Second, you should check floor loading calculations to ensure support for the horizontal nature of DataTank mounting. There is rarely a need for reinforcement, but it’s something that requires consideration.
Perhaps the most substantial requirement is the need for a secondary fluid loop, with manifolds for supply and return to the DataTanks. Sometimes these secondary loops already exist. Deploying dry coolers for outside heat rejection is recommended, as it is inexpensive and further saves on power and water consumption. This also enables you to increase primary fluid loop temperatures. Water entering the loop can be as high as 53ºC. Chillers are not designed to operate at such high entering and leaving temperatures, nor can they take advantage of the free-cooling opportunities using warm (versus chilled) water.
Finally, keep in mind that dry coolers will require space on the roof or equipment yard.
What Role Can 2-phase Immersion Cooling Play in Edge Computing?
Edge computing, by definition, involves more compute power located closer to the user or device. This results in the need for smaller, more widely distributed servers and data centers. As 2PIC allows for a 60-90% reduction in overall data center footprint, it will inevitably become the cooling method of choice for such a wide distribution, especially as more data centers will need to be located in densely populated urban environments. This raises the matter of noise pollution as well. Being in such close proximity to end users makes the near silence of 2PIC data centers another major advantage.
Interestingly enough, 2PIC data centers are also preferable for deploying edge computing in more remote locations. Hermetically sealed DataTanks mean IT equipment is not exposed to particulates, contaminants, dust, wind, and other elements. As a result, 2PIC data centers can go as long as 12 months without requiring any maintenance, making the deployment of data centers in remote or harsh locations much more viable.
The need for a greater number of widely distributed data centers means greater efficiency will become that much more important in edge computing environments, thus making 2PIC the logical choice.
What Types of Savings Can 2-phase Immersion Cooling Offer?
As of today, the capital expenditure required for cooling equipment in a 2PIC environment is already at parity with that of air-cooled systems. Beyond that, the massive improvements in density afforded by 2PIC lead to reduced costs at nearly every level of construction. Less land needs to be purchased, fewer building materials are involved, and less time and money is spent on architectural design, to name a few.
For a sense of size and scope, the following table compares various elements of air-cooled and immersion-cooled data center costs.
As you can see, immersion cooling provides economic benefits at nearly every level of construction and operations. And this holds true even in the case of retrofitting!
Conclusion
It’s not often that a new technology is so conducive to existing needs and growing trends that its adoption is all but inevitable. In the case of 2PIC, it’s hard to imagine a future where it doesn’t overtake air-cooled technologies. Between the efficiency and financial benefits it provides to industry and the enormous positive impact it would have on the environment, we can confidently conclude that 2-phase liquid immersion cooling will be the cornerstone and defining feature of tomorrow’s data centers.
To view the discussion in its entirety, click here.