Optimizing Cooling Layouts for the Data Center

Transcript

Slide 1
Welcome to the Data Center University™ course: Optimizing Cooling Layouts for the Data Center.

Slide 2: Welcome
For best viewing results, we recommend that you maximize your browser window now. The screen controls
allow you to navigate through the eLearning experience. Using your browser controls may disrupt the
normal play of the course. Click the attachments link to download supplemental information for this course.
Click the Notes tab to read a transcript of the narration.

Slide 3: Learning Objectives

At the end of this course, you will be able to:
List the major components common to most computer room air conditioning systems
Describe three methods of air delivery
Recognize the five basic methods of data center heat removal
Recognize the two basic arrangements of precision cooling equipment
Identify hot aisle/cold aisle configurations and the best practices of this application

Slide 4: Introduction
Whenever electricity is being consumed, heat is being generated. Excess heat in the data center can pose
a significant threat to availability, therefore maintaining appropriate temperature and humidity levels is
critical. A typical data center has six to eight times the heat density of a normal office space, and the
temperature can differ dramatically throughout that space. In addition, while a normal office space may
require two air changes per hour to maintain temperature, a high density data center may require up to thirty
air changes per hour. Striking a balance between the complex and drastic heat output of IT equipment and
the strict temperature range thresholds for optimal system performance, creates a significant cooling

Optimizing Cooling Layouts for the Data Center Page |1

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
challenge for IT and Facility Managers. Humidity and temperature levels outside the recommended range
can rapidly deteriorate sensitive components inside the computers making them vulnerable to future failures.
As high-density equipment becomes more prolific in IT rooms, proper configuration of cooling equipment
becomes paramount to the availability of the data center. The concept of proper cooling not only includes
the supply of adequate cool air, but also should take into account the distribution of that cool air, and the
removal of hot air. Data center cooling has emerged as a predominant area of focus for today’s IT and
Facility Managers.

Slide 5: Introduction
There are many facets of data center cooling. Types of cooling equipment, humidity levels, raised floor
designs, floor tile cooling configurations, and cabling arrangements all effect data center cooling. In this
course, we will specifically address how the layout of cooling equipment affects data center cooling.
However if you wish to learn more about these additional cooling factors, please refer back to the Data
Center University Course catalog, where these topics are addressed in more detail.

Slide 6: Types of Cooling Systems

Data center and network room cooling systems consist of a Computer Room Air Conditioner (or CRAC) and
an air distribution system. In larger data centers, a Computer Room Air Handler or CRAH may be used
instead of a CRAC. Although CRACs and CRAHs may differ in design and capacity, the configuration of the
associated distribution systems is what differentiates the different types of data center cooling systems.

Slide 7: Air Distribution Systems

Every air distribution system has a supply system and a return system. The supply system distributes the
cool air from the CRAC to the IT equipment. The return system takes the hot exhaust air from the IT
equipment, and returns it back to the CRAC where it can then be cooled, thus creating a cycle. For both
the supply and return, there are three basic methods used to transport air between the CRAC and the
equipment load.

Optimizing Cooling Layouts for the Data Center Page |2

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
These methods are:
1. Flooded Air Distribution System
2. Locally Ducted system
3. Fully Ducted system

Slide 8: Cooling Distribution Systems

In a Flooded distribution system, there is no ductwork between the CRAC and the equipment load. In this
case, the air moves freely between all pieces of equipment. In a Locally Ducted distribution system, air is
provided or returned via ducts which have vents located near the loads. In a Fully Ducted system, supply or
return air is directly ducted into or out of the loads. Each of these methods could be used on the supply side,
on the return side, or both.

Slide 9: Data Center Heat Removal Approaches

In addition to providing a sufficient amount of cool air, proper distribution and removal of hot air is essential
for maximum performance. There are 5 basic approaches for collecting and transporting unwanted heat
from the IT environment to the outdoor environment. One or more of these methods are used to cool
virtually all mission critical computer rooms and data centers. Each method uses the refrigeration cycle to
transport or pump heat from the data center or computer room to the outside environment. Some
approaches relocate the components of the refrigeration cycle away from the IT environment and some add
additional loops (self-contained pipelines) of water and other liquids to aid in the process. These five
approaches include:
1. Air Cooled DX systems
2. Air Cooled self contained systems
3. Glycol Cooled systems
4. Water Cooled systems
5. Chilled Water systems

Optimizing Cooling Layouts for the Data Center Page |3

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Slide 10: Air Cooled DX System
The first heat removal approach involves the use of air cooled computer room air conditioners, which are
widely used in IT environments of all sizes and are considered the “staple” for small and medium rooms.
This type of system is often referred to as a DX, or Direct Expansion system or split system. In an air
cooled system half the components of the refrigeration cycle are in the computer room air conditioner (also
known as a CRAC unit) and the rest are outdoors in the air cooled condenser. Refrigerant, typically R-22,
circulates between the indoor and outdoor components in pipes called refrigerant lines. Heat from the IT
environment is “pumped” to the outdoor environment using this circulating flow of refrigerant.

The advantages of this system include low overall cost, and ease of maintenance. One of the prime
disadvantages revolves around the need for refrigerant piping to be installed in the field. Only properly
engineered piping systems that carefully consider the distance and change in height between the IT and
outdoor environments will deliver reliable performance. In addition, refrigerant piping cannot be run long

Optimizing Cooling Layouts for the Data Center Page |4

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
distances reliably and economically. Finally, multiple computer room air conditioners cannot be attached to
a single air cooled condenser.

The Air Cooled DX system is commonly used in wiring closets, computer rooms and small-to-medium data
centers with moderate availability requirements.

Slide 11: Air Cooled Self-Contained Systems

Air cooled self-contained systems locate all the components of the refrigeration cycle in one enclosure that
is usually found in the IT environment. Heat exits the self-contained system as a stream of hot air called
exhaust air. This stream of hot air must be routed away from the IT room to the outdoors or into an
unconditioned space to ensure proper cooling of computer equipment. If mounted above a drop ceiling and
not using condenser air inlet or outlet ducts, the hot exhaust air from the condensing coil can be rejected
directly into the drop ceiling area. The building’s air conditioning system must have available capacity to
handle this additional heat load. Air that is drawn through the condensing coil (becoming exhaust air)
should also be supplied from outside the computer room. This will avoid creating a vacuum in the room that
would allow warmer, unconditioned air to enter. Self-contained indoor systems are usually limited in
capacity because of the additional space required to house all the refrigeration cycle components and the
large air ducts required to manage exhaust air. Self-contained systems that mount outdoors on a building
roof can be much larger in capacity but are not commonly used for precision cooling applications.

(Image on next page)

Optimizing Cooling Layouts for the Data Center Page |5

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
One of the key advantages of air cooled self-contained systems is that they have the lowest installation cost.
No components need to be installed on the roof or outside of the building. In addition, all refrigeration cycle
components are contained inside one unit as a factory-sealed and tested system for highest reliability.
Disadvantages of this system include less heat removal capacity per unit compared to other configurations.
Also, air routed into and out of the IT environment for the condensing coil usually requires ductwork and/or
dropped ceiling.

Air cooled self contained systems are commonly used in wiring closets, laboratory environments and
computer rooms with moderate availability requirements. They are sometimes utilized to address hot spots
in data centers.

Optimizing Cooling Layouts for the Data Center Page |6

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Slide 12: Glycol Cooled Systems
Glycol cooled systems contain all refrigeration cycle components in one enclosure (like a self-contained
system) but replace the bulky condensing coil with a much smaller heat exchanger. The heat exchanger
uses flowing glycol (a mixture of water and ethylene glycol, similar to automobile anti-freeze) to collect heat
from the refrigerant and transport it away from the IT environment. Heat exchangers and glycol pipes are
always smaller than condensing coils (2-piece air cooled systems) and condenser air ducts (self-contained
air cooled systems) because the glycol mixture has the capability to collect and transport much more heat
than air does. The glycol flows via pipes to an outdoor-mounted device called a fluid cooler. Heat is
rejected to the outside atmosphere as fans force outdoor air through the warm glycol-filled coil in the fluid
cooler. A pump package (pump, motor and protective enclosure) is used to circulate the glycol in its loop to
and from the computer room air conditioner and fluid cooler.

Optimizing Cooling Layouts for the Data Center Page |7

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Advantages:
The entire refrigeration cycle is contained inside the computer room air conditioning unit as a
factory-sealed and tested system for highest reliability with the same floor space requirement as a
two piece air cooled system.
Glycol pipes can run much longer distances than refrigerant lines (air cooled system) and can
service several computer room air conditioning units from one fluid cooler and pump package.
In cold locations, the glycol within the fluid cooler can be cooled so much (below 50°F [10°C]) that
it can bypass the heat exchanger in the CRAC unit and flow directly to a specially installed
economizer coil. Under these conditions, the refrigeration cycle is turned off and the air that flows
through the economizer coil, now filled with cold flowing glycol, cools the IT environment. This
process is known as “free cooling” and provides excellent operating cost reductions when utilized.

Disadvantages:
Additional components are required (pump package, valves) and this increases capital and
installation costs when compared with air cooled DX systems.
Maintenance of glycol volume and quality within the system is required.
This system introduces an additional source of liquid into the IT environment.
Glycol cooled systems are commonly Used:
In computer rooms and small-to-medium data centers with moderate availability requirements

Slide 13: Water Cooled Systems

Water cooled systems are similar to glycol cooled systems in that all refrigeration cycle components are
located inside the computer room air conditioner.

(Image on next page)

Optimizing Cooling Layouts for the Data Center Page |8

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
However, there are two important differences between a glycol cooled system and a water cooled system:
1. A water (also called condenser water) loop is used instead of glycol to collect and transport heat
away from the IT environment
Heat is rejected to the outside atmosphere via a cooling tower instead of a fluid cooler.
2. A cooling tower rejects heat from the IT room to the outdoor environment by spraying warm
condenser water onto sponge-like material (called fill) at the top of the tower. The water spreads
out and some of it evaporates away as it drips and flows to the bottom of the cooling tower (a fan is
used to help speed up the evaporation by drawing air through the fill material). In the same
manner as the human body is cooled by the evaporation of sweat, the small amount of water that
evaporates from the cooling tower serves to lower the temperature of the remaining water. The
cooler water at the bottom of the tower is collected and sent back into the condenser water loop via
a pump package.

Optimizing Cooling Layouts for the Data Center Page |9

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
 Condenser water loops and cooling towers are usually not installed solely for the use of
water cooled computer room air conditioning systems. They are usually part of a larger
system and may also be used to reject heat from the building’s comfort air conditioning
system (for cooling people) and water chillers (water chillers are explained in the next
section).
Advantages:
All refrigeration cycle components are contained inside the computer room air conditioning unit as
a factory-sealed and tested system for highest reliability.
Condenser water piping loops can easily run long distances and almost always service many
computer room air conditioning units and other devices from one cooling tower.
In leased IT environments, usage of the building’s condenser water is generally less expensive
than chilled water (chilled water is explained in the next section).

Disadvantages:
High initial cost for cooling tower, pump, and piping systems.
Very high maintenance costs due to frequent cleaning and water treatment requirements.
Introduces an additional source of liquid into the IT environment.
A non-dedicated cooling tower (one used to cool the entire building) may be less reliable then a
cooling tower dedicated to the Computer Room Air Conditioner.

Commonly Used:
In conjunction with other building systems in small, medium and large data centers with moderate-
to-high availability requirements.

Optimizing Cooling Layouts for the Data Center P a g e | 10

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Slide 14: Chilled Water Systems
In a chilled water system the components of the refrigeration cycle are relocated from the computer room air
conditioning systems to a device called a water chiller.

The function of a chiller is to produce chilled water (water refrigerated to about 46°F [8°C]). Chilled water is
pumped in pipes from the chiller to computer room air handlers (also known as CRAH units) located in the
IT environment. Computer room air handlers are similar to computer room air conditioners in appearance
but work differently. They cool the air (remove heat) by drawing in warm air from the computer room
through chilled water coils filled with circulating chilled water. Heat removed from the IT environment flows
out with the (now warmer) chilled water exiting the CRAH and returning to the chiller. At the chiller, heat
removed from the returning chilled water is usually rejected to a condenser water loop (the same condenser
water that water cooled computer room air conditioners use) for transport to the outside atmosphere.
Chilled water systems are usually shared among many computer room air handlers and are often used to
cool entire buildings.

Optimizing Cooling Layouts for the Data Center P a g e | 11

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Advantages:
Computer room air handlers generally cost less, contain fewer parts, and have greater heat
removal capacity than computer room air conditioners with the same footprint.
Chilled water piping loops are easily run very long distances and can service many IT
environments (or the whole building) from one chiller plant.
Chilled water systems can be engineered to be extremely reliable.
Chilled water systems have the lowest cost per kW for large installations.
Disadvantages:
Chilled water systems generally have the highest capital costs for installations below 100kW of
electrical IT loads.
CRAHs generally remove more moisture from data center air than their CRAC counterparts,
requiring more money be spent on humidifying the room in many climates.
Introduces an additional source of liquid into the IT environment.

Commonly Used:
In conjunction with other systems in medium and large data centers with moderate-to-high
availability requirements or as a high availability dedicated solution in large data centers.

Slide 15: Precision Cooling Model Types

There are 2 basic model types of precision cooling equipment: ceiling mounted and floor mounted. Variants,
such as wall-mounted or mini-split systems are similar to ceiling mounted systems and are employed
similarly when adequate wall space is available.

Optimizing Cooling Layouts for the Data Center P a g e | 12

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Slide 16: Ceiling Mounted Systems
Ceiling mounted systems are small (300-500 pound) (136-227 kg) precision cooling devices suspended
from the IT room’s structural ceiling. They cool 3-17kW of computer equipment and utilize any of the 5 IT
environment heat removal methodologies. One of the key benefits of ceiling mounted systems is that they
do not require floor space in the IT environment. A drawback however, is that installation and maintenance
activities are more complicated due to their overhead placement. As a result, it is recommended that IT
professionals, facilities personnel and manufacturer’s representatives or mechanical contractors handle the
specification, installation and maintenance of ceiling mounted precision cooling systems.

Slide 17: Floor Mounted Systems

Floor mounted systems

Floor mounted precision cooling systems usually offer the greatest range of features and capabilities. They
are increasingly being used to cool or to assist in the cooling of smaller IT environments as power
consumption of computer equipment continues to increase.

Slide 18: Portable Systems

Portable systems (also known as spot coolers) are considered part of the floor mounted category, however
they are almost always have wheels and can be easily located anywhere precision cooling is required.
Portable systems cool 2-6kW of computer equipment and often a normal wall outlet can be used to supply
electrical power (2-4kW models). Portable systems are almost always self-contained systems.
Specification, installation and maintenance of most portable cooling systems can be accomplished by IT
professionals without the assistance of facilities personnel or mechanical contractors.

Slide 19: Large Floor Mounted Systems

Large floor mounted precision cooling systems have been extensively used to cool mission critical
computing environments since their inception. These are usually the highest capacity cooling devices found
in the IT environment with the ability to cool 20kW to over 200kW of IT equipment per chassis. Floor
mounted systems utilize IT environment floor space and must be strategically located in the room for

Optimizing Cooling Layouts for the Data Center P a g e | 13

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
maximum effectiveness. Specification, installation and maintenance of large floor mounted precision
cooling systems is highly dependent on the existing electrical, mechanical and structural capabilities of the
building they are to be operated in. For this reason it is important for IT professionals to work closely with
facilities management and manufacturer’s representatives during the specification process. Often the
services of a State-Registered Professional Engineer are required to design and certify the solution. Most
mechanical contracting firms familiar with IT environments can install and if desired, maintain the solution.
Recent developments in large floor mounted systems have reduced their energy consumption and the
overall space they require in the computer room or data center. Their outer dimensions and appearance
have changed so they fit in spaces sized for IT rack enclosures. This allows for operational cost savings
and more flexibility in IT environment planning.

Slide 20: Cooling Arrangements

Cooling equipment for an IT environment can be implemented in 10 basic configurations. The selection of
the appropriate configuration for a particular installation is affected by the existing facility infrastructure, the
total power level of the installation, the geographical location, and the physical constraints of the building.

Slide 21: Cooling Arrangements

The 10 configurations are achieved by combining the 5 methods of heat removal with the ceiling and floor
mounted model types.

The Air Cooled DX System requires roof access, an air cooled condenser and refrigerant piping for both the
ceiling and floor mounted configurations. The floor and ceiling configurations are typically found in computer
rooms, and the floor mounted arrangement is also found in medium data center applications.

(Image on next page)

Optimizing Cooling Layouts for the Data Center P a g e | 14

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
The Air Cooled Self-Contained System must have dropped ceiling or ducts installed for ceiling mounted
configurations. The floor mounted system also must have a dropped ceiling for condenser air tubes. In
addition, large floor mounted systems require outdoor heat rejection components. This system is typically
found in wiring closets, computer rooms and small data centers.

With Glycol Cooled Systems, both ceiling and floor mounted arrangements require the building to have roof
access and a 10’ (3m) floor to structural ceiling height. Fluid cooler, pump package and glycol piping is also

Optimizing Cooling Layouts for the Data Center P a g e | 15

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
required. Ceiling mounted systems are found in computer rooms and small data centers; while floor
mounted versions can be used in medium and large data center installations.

Water cooled systems require that the building must have 10’ (3m) floor to structural ceiling height for ceiling
arrangements with a further requirement of a hookup to the building condenser water. In floor mounted
configurations, the building must have condenser water system with adequate capacity. Ceiling mounted
installations are not commonly seen, however floor mounted configurations can be found in medium and
large data center applications.

Optimizing Cooling Layouts for the Data Center P a g e | 16

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
In Chilled Water Systems, the ceiling mounted and floor mounted arrangements requires a reliable chilled
water system and hookup, with the ceiling mounted configuration of a 10’ (3m) floor to structural ceiling
height. Chilled water systems that are ceiling mounted can be found in wiring closets, computer rooms and
small data centers, while the floor mounted version can be used in computer rooms, ,and small, medium
and large data center installations.

Optimizing Cooling Layouts for the Data Center P a g e | 17

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Slide 22: Technology Compaction
As computing equipment continues to shrink in size, the overall footprint of high power computing also
decreases. This means that the amount of equipment needed to perform the same amount of computing
has gone down, thereby also decreasing heat that is created. Despite an overall decrease in heat levels,
however, the heat density per square foot has increased, as have the prevalence of hot spots.

Slide 23: Equipment Placement

The way in which individual equipment racks are placed in relation to one another can also have a
significant impact on heat density, conditioned air distribution and service accessibility. When numerous
racks and large amounts of equipment are a factor, careful planning of room layouts, and continued
diligence in maintaining these layouts is imperative for maximum performance.

Slide 24: Rack Arrangements

It is common to enter a data center and see row after row of equipment racks all facing the same way.
While this is aesthetically pleasing, and may seem to be the logical way in which to layout numerous racks,
this approach can be detrimental to the overall availability of the data center.

Before we can understand how rack placement affects performance, let’s first discuss how racks and
equipment components housed within the racks are affected by airflow in the data center.

Slide 25: Rack Arrangements

The majority of equipment racks on the market are designed to take in cooled air through the front, pass it
over the IT equipment, and exhaust it from the rear. With either raised floor or non-raised floor
environments, it’s important to design layouts such that they maximize this airflow pattern.

Slide 26: Hot Aisle/ Cold Aisle

The ideal placement of rack equipment is called a hot-aisle/cold-aisle configuration.

Optimizing Cooling Layouts for the Data Center P a g e | 18

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Slide 27: Hot Aisle/ Cold Aisle
This configuration involves the installation of racks in a face-to-face (and back-to-back) orientation. By
configuring units in this way, it eliminates the hot exhaust air of one unit being drawn into the intake of
another unit.

Slide 28: Hot Aisle/ Cold Aisle

In a hot aisle /cold aisle configuration, in addition to racks being placed face to face, CRAC units must also
be strategically placed to create a cold aisle by properly distributing the cold air to the face of the racks, and
to maximize the return of hot exhaust air out of the back of the racks and into the hot aisle.

Optimizing Cooling Layouts for the Data Center P a g e | 19

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
Slide 29: Hot Aisle/ Cold Aisle
In a large data center application, this would mean every other row would be forward facing to create the hot
aisle/cold aisle arrangement.

Slide 30: Hot Aisle Containment Units

Taking this concept one step further has prompted manufacturers such as APC to developed hot aisle
containment units. Hot aisle containment systems ensure proper air distribution by completely separating
supply and return air paths. The hot aisle is sealed off using doors and transparent ceiling tiles that extend

Optimizing Cooling Layouts for the Data Center P a g e | 20

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.
the width of the hot aisle. These units ensure that this is little to no mixing of hot and cold air streams in the
data center. The warmest air possible is returned to the CRAC units thus increasing the efficiency and
capacity of the system.

Slide 31: Summary

To summarize:
The three basic methods used to transport air between the CRAC and the equipment load are
Flooded, Locally Ducted, Fully Ducted
There are five basic systems for data center heat removal. They are Air Cooled DX Systems, Air
Cooled Self-Contained Systems, Glycol Cooled Systems, Water Cooled Systems and Chilled
Water Systems
There are 2 basic model types of precision cooling equipment: ceiling mounted and floor mounted
The ideal placement of rack equipment is in a hot-aisle/cold-aisle configuration, where racks are
installed face-to-face and back-to-back

Slide 32: Thank You!

Thank you for participating in this course.

Optimizing Cooling Layouts for the Data Center P a g e | 21

© 2013 Schneider Electric. All rights reserved. All trademarks provided are the property of their respective owners.