Staff Answer
Nov 20, 2019 - 06:54 AM
When comparing the PUE of Single-phase Liquid Immersion Cooling (SLIC) and air-cooled systems it is important to break down the where the costs and where the saving are so that the PUE is properly calculated for the two systems.
For those not familiar with PUE, a detailed definition can be found here.
The basic formula for air cooled facilities is:
The difficulty with using the standard PUE formula when comparing air and immersion cooled systems stems from the definition of "IT Equipment Energy."
The majority of air-cooled IT equipment (servers) have embedded fans in them that use up to 20% of the power. This fact has lead to some data center providers to publish excellent PUEs for their data centers by moving a majority of the air-cooling power use "inside" the IT device. They do this by placing fans and in some case A/C compressors powered by the IT equipment in the racks, enabling a higher rate of air flow to provide better cooling. This is nothing more than a marketing ruse to show better and therefore "greener" PUEs in the face of increasing pressure by environmental groups to reduce energy use and waste by data centers.
SLIC does not require the use of fans on the devices. It is important when converting air-cooled equipment for use in immersion cooling that the fans and all air baffles be fully removed. In fact, when you leave the fans installed in immersed equipment, they not only waste power when operating in fluid - the fans typically disrupt the coolant flow thereby reducing cooling effectiveness. By removing the device fans you can save a substantial amount of energy!
For typical enterprise servers the amount of energy saved ranges from 15 - 20%.
For cryptocurrency miners the amount of energy saved is ranges from 5 - 10%.
Because of this difference in equipment configuration and where the fans are included in the calculation we need to modify the calculation of PUE to ensure that the on-board device fans and pumps are properly categorized as Non-IT Facility Energy in the calculation for both air and immersion cooled devices.
Thus the calculation of PUE for a 1MW (1000KW Servers with Fans) IT Equipment Energy with 400KW of Non-IT Facility Energy (Cooling, lighting, comfort heating and cooling, etc.) would change from the standard PUE calculation of:
1,000KW + 400KW = 1,400KW = Total Facility Energy
--------------------------------------------------------------------------- = 1.40 PUE
. 1,000KW = IT Equipment Energy
to the modified form where the on-board device cooling fans / pumps are removed from the IT Equipment Energy, but are included in the Total Facility Energy thus yielding:
1,000KW load for air-cooled Servers equals (1000KW * 20%) = Fan Power = 200KW which will be included in the Total Facility Energy
and
(1000KW Servers - 200KW Server Fans) = 800KW Server IT Power = IT Equipment Energy
Thus the modified PUE calculation that can be used to compare to an immersion cooled system would yield:
800KW + 200KW + 400KW = 1,400KW = Total Facility Energy
-------------------------------------------------------------------------------
------- = 1.75 PUE
. 800KW Servers = IT Equipment Energy
This illustrates that the standard air-cooled systems' PUE calculation can overstate the actual PUE by as much as 25%!
When comparing immersion cooled systems and air-cooled systems, it is equally important to understand the types of loads that make up the Non IT Facility Energy. Comfort heating and cooling for office space, lighting, security systems, kitchen appliances, etc. are all components that are included in the Non IT Facility Energy, however far and away the largest is the air handling and cooling systems used to cool the IT Equipment. It is not unusual for the air conditioning and air handling systems to account for over 90% of all Non IT Facility Energy consumption in the data center.
Because of the highly concentrated nature of the equipment in a SLIC cooled environment you need substantially less floor space (typically a reduction of 80%) which reduces a corresponding quantity of the non-cooling related Non IT Facility Energy.
Finally because of the efficiencies of SLIC and Engineered Fluids coolants and their low viscosity at operating temperatures the energy required to pump and circulate the fluid is very low, on the order of 1-2% of the IT Equipment Energy.
Thus it is common for SLIC systems to have true system PUE's of 1.05 to 1.03 in operation.
In fact, we have systems in our labs operating on a long term basis with system PUEs of 1.018!
For those not familiar with PUE, a detailed definition can be found here.
The basic formula for air cooled facilities is:
The difficulty with using the standard PUE formula when comparing air and immersion cooled systems stems from the definition of "IT Equipment Energy."
The majority of air-cooled IT equipment (servers) have embedded fans in them that use up to 20% of the power. This fact has lead to some data center providers to publish excellent PUEs for their data centers by moving a majority of the air-cooling power use "inside" the IT device. They do this by placing fans and in some case A/C compressors powered by the IT equipment in the racks, enabling a higher rate of air flow to provide better cooling. This is nothing more than a marketing ruse to show better and therefore "greener" PUEs in the face of increasing pressure by environmental groups to reduce energy use and waste by data centers.
SLIC does not require the use of fans on the devices. It is important when converting air-cooled equipment for use in immersion cooling that the fans and all air baffles be fully removed. In fact, when you leave the fans installed in immersed equipment, they not only waste power when operating in fluid - the fans typically disrupt the coolant flow thereby reducing cooling effectiveness. By removing the device fans you can save a substantial amount of energy!
For typical enterprise servers the amount of energy saved ranges from 15 - 20%.
For cryptocurrency miners the amount of energy saved is ranges from 5 - 10%.
Because of this difference in equipment configuration and where the fans are included in the calculation we need to modify the calculation of PUE to ensure that the on-board device fans and pumps are properly categorized as Non-IT Facility Energy in the calculation for both air and immersion cooled devices.
Thus the calculation of PUE for a 1MW (1000KW Servers with Fans) IT Equipment Energy with 400KW of Non-IT Facility Energy (Cooling, lighting, comfort heating and cooling, etc.) would change from the standard PUE calculation of:
1,000KW + 400KW = 1,400KW = Total Facility Energy
--------------------------------------------------------------------------- = 1.40 PUE
. 1,000KW = IT Equipment Energy
to the modified form where the on-board device cooling fans / pumps are removed from the IT Equipment Energy, but are included in the Total Facility Energy thus yielding:
1,000KW load for air-cooled Servers equals (1000KW * 20%) = Fan Power = 200KW which will be included in the Total Facility Energy
and
(1000KW Servers - 200KW Server Fans) = 800KW Server IT Power = IT Equipment Energy
Thus the modified PUE calculation that can be used to compare to an immersion cooled system would yield:
800KW + 200KW + 400KW = 1,400KW = Total Facility Energy
-------------------------------------------------------------------------------
------- = 1.75 PUE
. 800KW Servers = IT Equipment Energy
This illustrates that the standard air-cooled systems' PUE calculation can overstate the actual PUE by as much as 25%!
When comparing immersion cooled systems and air-cooled systems, it is equally important to understand the types of loads that make up the Non IT Facility Energy. Comfort heating and cooling for office space, lighting, security systems, kitchen appliances, etc. are all components that are included in the Non IT Facility Energy, however far and away the largest is the air handling and cooling systems used to cool the IT Equipment. It is not unusual for the air conditioning and air handling systems to account for over 90% of all Non IT Facility Energy consumption in the data center.
Because of the highly concentrated nature of the equipment in a SLIC cooled environment you need substantially less floor space (typically a reduction of 80%) which reduces a corresponding quantity of the non-cooling related Non IT Facility Energy.
Finally because of the efficiencies of SLIC and Engineered Fluids coolants and their low viscosity at operating temperatures the energy required to pump and circulate the fluid is very low, on the order of 1-2% of the IT Equipment Energy.
Thus it is common for SLIC systems to have true system PUE's of 1.05 to 1.03 in operation.
In fact, we have systems in our labs operating on a long term basis with system PUEs of 1.018!
Add New Comment