Airflow and Cooling in a Data

Center
This paper deals with the distribution of airflow and the resulting cooling in a data
Suhas V. Patankar center. First, the cooling challenge is described and the concept of a raised-floor data
Professor Emeritus center is introduced. In this arrangement, cooling air is supplied through perforated tiles.
Department of Mechanical Engineering, The flow rates of the cooling air must meet the cooling requirements of the computer
University of Minnesota, servers placed next to the tiles. These airflow rates are governed primarily by the pres-

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
Minneapolis, MN 55455; sure distribution under the raised floor. Thus, the key to modifying the flow rates is to
President influence the flow field in the under-floor plenum. Computational fluid dynamics (CFD) is
inInnovative Research, Inc., used to provide insight into various factors affecting the airflow distribution and the
Plymouth, MN 55447 corresponding cooling. A number of ways of controlling the airflow distribution are
e-mail: [email protected] explored. Then attention is turned to the above-floor space, where the focus is on pre-
venting the hot air from entering the inlets of computer serves. Different strategies for
doing this are considered. The paper includes a number of comparisons of measurements
with the results of CFD simulations. 关DOI: 10.1115/1.4000703兴

Keywords: data center, electronics cooling, raised floor, under-floor flow, simulation,
computational fluid dynamics

1 Introduction consumes electrical energy and dissipates a large amount of heat

in the range of 2–20 kW. For the electronics to function properly,
1.1 What is a Data Center? We live in a computer age. We it needs to be cooled and kept at an acceptable temperature level.
search for information on the Internet. We reserve e-tickets for Overheating may cause the equipment to malfunction, melt, or
airline travel and get our boarding passes on a computer. Stocks burn; but more commonly, safety devices on the server racks will
are traded on computers and people use online banking. When we detect high temperatures and shut down the equipment. It is this
use a credit card, the transaction is instantaneously verified and interruption that presents a serious problem for a data center and
approved. All major retail merchants offer online shopping. Large needs to be prevented.
companies have their inventory, purchase orders, invoices, and all Normally, cooling air enters a server rack through the front face
accounting computerized. Medical records are stored on comput-
and hot air exits from the rear face. In a large room, in which 2000
ers. We use email and text messaging to communicate with others.
server racks may be spread all over the room, it is not easy to
The modern cell phone is a small computer that communicates
supply cooling air to each rack. This task is accomplished by a
with the rest of the world. The list goes on.
clever concept called the raised-floor data center. The server racks
Whereas the visible transaction takes place at the “point of sale”
are installed on a tile floor that is raised 0.3–0.6 m 共12–24 in.兲
on a personal computer or a small device, the whole mechanism
above the real solid floor. Air-conditioners are used to pump cold
can function only if all the relevant data is held at one place and
air into the space below the raised floor. The floor tiles are remov-
processed at a very fast speed. Therefore, behind the small visible
able and some of the solid tiles can be replaced by perforated tiles
devices 共such as a desktop computer, a laptop, or a cell phone兲 are
or grilles to permit the cold air to enter the above-floor space. By
large and powerful computer servers located in one place. For a
credit-card or stock-trading company, it is common to have a large locating perforated tiles at the foot of the server racks, cooling air
is delivered to them. The hot air then finds its way back to the
room 共of the size 70⫻ 70 m2兲 housing over 2000 server racks
air-conditioners.
共each 1 ⫻ 1 m2 and 2 m tall兲. There are also colocation facilities,
The raised-floor arrangement gives unlimited flexibility. If the
where different companies can place a few server racks each for
their own use. Such a huge computer room is called a “data cen- layout of the server racks is changed, all that is needed is to
ter.” It has become an essential part of any modern-day large rearrange the perforated tiles so that cooling air is delivered at the
business. new locations of the server racks. Since there is no permanent
The most important requirement for a data center is its uninter- ducting, no elaborate dismantling or construction is necessary.
rupted, zero-downtime operation. An interruption caused by The raised-floor design makes it possible to create, but does not
equipment failure would entail costly repairs and replacement. guarantee, proper cooling of the server racks. How much cold air
But even more serious is the cost of business interruption; the do we need? How does it distribute in the whole data center? Are
business may lose thousands or even millions of dollars for every we meeting the individual demands of all server racks? Does the
minute of downtime. cold air introduced at the floor level reach to the top of the 2-m
For uninterrupted operation, two things are crucial: power and tall server rack? Can the hot air coming out of one rack enter the
cooling. Uninterrupted power is assured by having several backup inlet of another rack and damage its cooling? These questions will
sources of power that can be automatically brought on line as be addressed in this paper.
soon as a power failure is detected. Cooling is a more complex Cooling a data center in its current configuration is a difficult
issue, which is discussed in this paper. challenge in itself. But data centers are dynamic; their equipment
layout continually needs to change. Business conditions require
1.2 The Cooling Challenge. Each server rack in a data center that new server racks are installed and the old ones are removed.
Typically, ten percent of the equipment in a data center is replaced
Contributed by the Heat Transfer Division of ASME for publication in the JOUR-
each month. The cooling design has to keep pace with this fre-
NAL OF HEAT TRANSFER. Manuscript received March 30, 2009; final manuscript re- quent change.
ceived November 11, 2009; published online April 27, 2010. Editor: Yogesh Jaluria. The common practice to meet this challenge is trial-and-error

Journal of Heat Transfer Copyright © 2010 by ASME JULY 2010, Vol. 132 / 073001-1
and “overkill.” One detects hot spots and tries to force cold air particular small data center. However, as will be shown later in
there by placing more perforated tiles. If that fails, an extra air- this paper, the assumption is not valid for most practical
conditioner is installed in the hope of removing the hot spots. It is configurations.
estimated that the amount of cooling air used in most data centers In a subsequent study, Schmidt et al. 关9兴 allowed the pressure
is 2.5 times the required amount. So, there is a good opportunity variations in the under-floor space, but used a depth-averaged
for saving on air-conditioning equipment and energy. Whereas model to convert the three-dimensional problem into a two-
until now uninterrupted operation was the only focus 共and data dimensional one. For the cases they considered, a good agreement
centers were ready to spend any amount of resources to achieve with measurements was demonstrated. However, later investiga-
it兲, the current economic climate has shifted the focus to energy tions have shown that the depth-averaged model is adequate when
conservation. A “green data center” is a new goal in addition to the height of the raised floor is small 共normally less then 0.15 m or
the zero-downtime operation. 6 in.兲. Practical data centers have floor heights in the range of
0.3–0.75 m. As a result, even this simplification is not acceptable

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
1.3 The Criterion for Acceptable Cooling in a Data and one must perform the full three-dimensional computation.
Center. Manufacturers of computer servers design their equip- Schmidt 关10兴 and Schmidt and Cruz 关11,12兴 have studied the
ment with a certain allowable maximum inlet temperature. This inlet temperatures to server racks in a data center. Patel et al.
value is around 24° C 共75° F兲. The air-conditioners in a data cen- 关13,14兴 describe application of CFD modeling to data centers and
ter usually supply cold air at 13° C 共55° F兲. If the 13° C cold air evaluate some specific overhead-cooling solutions. A detailed
enters the servers, there is no difficulty in satisfying the manufac- analysis of the flow in the under-floor space and the factors affect-
turer’s criterion. As we will see later, the cold air does not always ing it is presented by Karki et al. 关15,16兴 and Patankar and Karki
enter at all the inlet locations on the server rack. Often, the hot air 关17兴; in Ref. 关16兴, computed results are compared with measure-
exhausted by the rack finds its way to the inlet of the same rack or ments of airflow through the perforated tiles. Guggari et al. 关18兴
some other rack. This is how the cooling in a data center is com- describe ways of optimizing data center layout. As the concerns
promised. about data center cooling became more important, Bash et al. 关19兴
1.4 Methods to Meet the Cooling Challenge. As in any outlined the research needs in this area.
complex engineering situation, the cooling in a data center has Temperature and flow rate measurements in an actual data cen-
been conventionally handled by accumulated experience 共includ- ter are reported by Schmidt 关20兴. Later in this paper, this study
ing some rules of thumb兲, field measurements, and ad hoc design will be used to provide a sample validation of CFD simulation.
changes. A number of specialized cooling products have been in- The issue of distributed leakage through the raised floor is ad-
troduced in recent years, but their evaluation for specific situations dressed by Radmehr et al. 关21兴 and Karki et al. 关22兴; they report
is costly and time-consuming. The need for a more scientific ap- airflow measurements and corresponding CFD analysis. Van
proach is quite obvious. Gilder and Schmidt 关23兴 consider the factors that govern the uni-
formity of airflow through the perforated tiles.
1.5 Role of CFD Simulation. Cooling in a data center is an Whereas the full three-dimensional CFD analysis of the under-
excellent application for Computational Fluid Dynamics 共CFD兲. It floor airflow yields useful results, a one-dimensional idealization
offers a new paradigm for meeting the cooling challenge. One can of the flow can be used to get valuable insight. Karki and Patankar
create a computer model of the whole data center, complete with 关24兴 have derived such a one-dimensional model and provided an
the raised floor, air-conditioning units, perforated tiles, and server analytical solution of the governing equations. The validity of the
racks. The CFD simulation then provides a detailed distribution of model is demonstrated with reference to a full three-dimensional
air velocity, pressure, and temperature throughout the room. The CFD solution.
simulation can be used to analyze an existing data center, but The interaction of the air stream emerging from a perforated tile
more importantly, any proposed layout for a new or reconfigured with the internal fans in a front-to-rear server rack is analyzed by
data center. One can detect hot spots in a simulation 共before they Radmehr et al. 关25兴. They determine the conditions for which the
arise in reality兲 and explore ways of mitigating them. As already inlet flow is essentially unaffected by the pressure variations in the
mentioned, data centers are dynamic environments; their equip- inlet stream.
ment layout changes frequently. A CFD simulation provides in-
valuable help in planning the changes and ensuring proper cool- 1.7 Scope of the Paper. The purpose of this paper is to de-
ing. scribe some interesting physical behavior in the airflow and cool-
ing in a data center. Many physical effects are explained with the
1.6 Available Literature. Archival literature on airflow man- help of CFD simulation. Most of the material is taken from the
agement in data centers is rather scarce. Only in recent years, this papers mentioned above, where further details can be found.
topic has created strong scientific and practical interest. A brief A complete consideration of the flow and heat transfer in a data
overview of the available literature is given below. center would include both the under-floor and above-floor spaces.
Seymour 关1兴 applied CFD analysis to show temperature and This paper initially focuses on the flow in the under-floor space
flow distributions in an atrium. Awbi and Gan 关2兴 used CFD to and the resulting airflow rates through the perforated tiles. Al-
predict airflow and temperature distributions within offices. Kiff though the under-floor space appears to be insignificant, it is here
关3兴 presented the results of a CFD analysis of rooms populated that the cooling battle is primarily won or lost. If we are able to
with telecommunications equipment. Bullock and Phillip 关4兴 per- deliver the required amount of airflow at the foot of each server
formed a CFD simulation for the Sistine Chapel renovation rack, proper cooling is essentially assured. If we fail to satisfy this
project. Schmidt 关5兴 compared CFD results with measurements of requirement, any attempted remedy in the above-floor space is
temperature and velocity fields in an office-size data processing usually ineffective.
room. Cinato et al. 关6兴 describe a tool to optimize the energy The flow in the under-floor space is influenced by various fac-
consumption of the environmental systems that provide cooling to tors such as the layout of perforated tiles, their open area, the
telecommunication rooms. Quivey and Bailey 关7兴 have presented height of the raised floor, and under-floor obstructions. A number
results based on a CFD model for a Lawrence Livermore Data of simple case studies are described to illustrate these effects.
Center. The above-floor space brings its own unique behavior and sur-
Kang et al. 关8兴, on the basis of some CFD calculations, pro- prises. These are explained through a number of examples. It is
posed a simplified model that assumes the whole volume under shown that the main challenge in the above-floor space is to en-
the raised floor to be at a uniform pressure. The entire flow system sure that the hot exhaust air from a server rack does not enter the
is then represented as a network of flow resistances. The results inlet of the same rack or some other rack. Various strategies used
from this simplified model did agree well with CFD results for a for ensuring this are described.

073001-2 / Vol. 132, JULY 2010 Transactions of the ASME

 Server Racks Server Racks
CRAC

Fig. 1 A raised-floor data center

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
The paper is written in an introductory style so that the readers
who are not intimately familiar with data centers can get a good
appreciation of the scientific and practical issues pertaining to
cooling in a data center.
Fig. 3 The cold aisle in a data center
2 Fundamentals of a Raised-Floor Data Center
2.1 The Overall Arrangement. Figure 1 shows an outline of
a raised-floor data center. On the right, a down-flow air- seen on both sides. The raised floor is made up of tiles. Many tiles
conditioning unit is placed on the raised floor. It draws the hot air in the cold aisle are perforated. It is through these tiles that cold
in the room into its top face and supplies cold air from its bottom air is supplied for the cooling of the racks.
into the under-floor space. Such an air-conditioner is called a com-
puter room air-conditioner 共CRAC兲 by the data center community. 2.3 Importance of Supplying the Required Airflow to Each
The cold under-floor air enters the above-floor space through per- Server Rack. A server rack has internal fans that draw a known
forated tiles that are placed at the foot of the server racks. The amount of airflow rate. Also, as mentioned before, the cooling of
racks, in turn, draw in this air through their front face and exhaust the rack is designed on the basis of a maximum acceptable inlet
hot air from the rear face. The hot air finally returns to the top of temperature. If the actual inlet temperature exceeds this value,
the air-conditioning unit 共CRAC兲. overheating occurs, which may lead to malfunction or automatic
shutdown.
2.2 The Hot Aisle/Cold Aisle Concept. To eliminate the pos- To ensure that cold air enters the rack, the airflow rate from the
sibility that the hot air exhausted by one rack would enter the inlet perforated tile at the foot of the rack has to be equal to 共or greater
of another rack, data centers are often laid out in the “hot aisle/ than兲 the airflow rate demanded by the internal fans in the rack.
cold aisle” arrangement, which is shown in Fig. 2. This arrange- Figures 4 and 5 illustrate this concept via simplified examples. In
ment was suggested by Sullivan 关26兴 and has become a standard both cases, each rack dissipates 2 kW of heat and requires an
practice in data centers. The so-called cold aisle has the perforated airflow rate of 0.15 m3 / s. In Fig. 4, the airflow supplied at the
tiles. The server racks are placed on both sides of the cold aisle
such that their inlets face the cold aisle. As a result, the exhausts
from two neighboring rows of racks emerge into the hot aisle. Of To CRAC Unit
course, there is no reason to place any perforated tiles in the hot
aisle. The hot air collected there simply returns to the CRAC unit, 24°C
without entering 共hopefully兲 into the inlet of any server rack.
Figure 3 shows a photograph of a cold aisle in a data center.
The unit at the back is a CRAC. The inlets of the server racks are 24°C

2 kW 24°C 2 kW

0.15 m3/s, 12.8°C 0.15 m3/s, 12.8°C

Fig. 4 Required airflow supplied

To CRAC Unit
CRAC Server Rack Perforated Tile
35°C
0.075 m3/s 0.075 m3/s

46°C 46°C

Cold Aisle
1 kW 1 kW

Hot Aisle
1 kW 24°C 24°C 1 kW
Cold Aisle

0.075 m3/s, 12.8°C 0.075 m3/s, 12.8°C

Fig. 2 The hot aisle/cold aisle arrangement Fig. 5 Insufficient airflow

Journal of Heat Transfer JULY 2010, Vol. 132 / 073001-3

perforated tile is indeed 0.15 m3 / s. This meets the demand of the
server rack. The cold air is supplied at 12.8° C, it heats up to
24° C, and this hot air returns to the CRAC unit. Since the tem-
perature entering the rack is 12.8° C, which is well below the
acceptable inlet temperature, proper cooling is assured.
As will be shown later, it is not always possible to control the
amount of airflow emerging from each perforated tile. Figure 5
shows the case in which insufficient airflow is supplied from the
perforated tile. Whereas the rack requires 0.15 m3 / s, the perfo-
rated tile supplies only half that amount. This reduced flow is
sufficient to cool the bottom half of the rack. The cooling airflow
needed by the top half is now taken from the “hot” air returning to

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
the CRAC unit. This air has resulted from the mixing of the ex-
haust air at 24° C from the bottom half of the rack with the much
hotter air emerging from the top half of the rack. The result is that
the maximum inlet temperature to the rack is 35° C, which may be
unacceptable for many electronics designs. 共The idealized picture
in Fig. 5 assumes that the exhausts of the top and bottom halves of
the rack are well mixed to the temperature of 35° C. In reality, if
such perfect mixing does not take place, even hotter air is likely to
enter the inlet of the top half of the rack.兲
These simple examples lead to a very important conclusion.
The key to good cooling is to supply the required amount of cold
airflow at the foot of each server rack. If this is done, satisfactory Fig. 6 Pressure drop as a function of the airflow rate for per-
forated tiles
cooling is assured. If this cannot be done, cooling difficulties arise
and then they are usually very hard to overcome.
2.4 Airflow in the Under-Floor Space. Since the airflow rate
tical flow rates through perforated tiles are of the order of
emerging from each perforated tile holds the key to successful
0.25 m3 / s, for which the pressure drop is about 12 Pa. That this
cooling, we turn our attention to what controls the distribution of
value of the pressure drop is rather small is relevant for the dis-
airflow through the perforated tiles. Interestingly, it is not what
cussion below.
happens above the raised floor but what happens below the raised
floor that determines the flow through the perforated tiles. Thus, 2.6 Behavior of the CRAC Unit. The flow emerging from
the fluid mechanics of the tiny 共and usually invisible兲 space below the perforated tiles originates at the CRAC unit. Usually, the flow
the raised floor controls the success or failure of cooling in a data is directed vertically downwards from one or more blowers in the
center. From a computational point of view, this is good news. If CRAC. It impinges on the solid floor 共usually called the subfloor兲,
the CFD simulation is limited to the under-floor space, the calcu- turns 90 deg, moves horizontally in the under-floor space, and
lation domain is small and a fast solution is possible. Yet, this emerges from the perforated tiles wherever they are placed. The
small computational effort leads to the most valuable information CRAC blowers deliver a rated airflow; 5 m3 / s is a typical value.
needed for the cooling of the data center. Does the actual amount of airflow from a CRAC unit depend on
2.5 Flow Through the Perforated Tiles. The perforated tiles the number of perforated tiles and their percent open area? Inter-
used in a data center have a large number of small circular holes. estingly, for all practical purposes, a CRAC unit can be regarded
Such a tile is usually characterized by the percentage open area; as a constant-flow device that delivers that same amount of air-
the most common perforated tiles have 25% open area. The flow flow rate for different numbers, layouts, and open areas of the
resistance of the perforated tiles can be obtained from well-known perforated tiles.
To understand this conclusion, we need to examine the various
pressure-drop correlations for such plates. The pressure drop ⌬p
pressure drops that the CRAC blower is required to overcome.
across a perforated tile is expressed as
These include the pressure drop in:
⌬p = K共0.5␳V2兲 • The filter and cooling coils in the CRAC unit. 共This is
where V is the velocity approaching the perforated tile, ␳ is the known as the internal static and is usually in the range 250–
density of air, and K is the flow resistance factor 共the “K factor”兲. 500 Pa.兲
An empirical formula for K, based on a large number of measure- • The impingement and turning on the subfloor. 共This is
ments, is given by Idelchik 关27兴 known as the external static and is usually in the range 100–
200 Pa.兲
1 • The perforated tiles 共about 12 Pa, as seen above兲.
K= 共1 + 0.5共1 − F兲0.75 + 1.414共1 − F兲0.375兲
F2
where F is the fractional open area of the perforated tile. For a Table 1 Pressure drop for a 25% open tile at different airflow
25% open tile, this formula gives K = 42.8. rates
Figure 6 shows the variation in the pressure drop ⌬p with the
volumetric airflow rate through the perforated tile for tiles of 6%, Airflow rate Pressure drop
11%, and 25% open area, as given by the above formula. For the 共m3 / s兲 共Pa兲
case of 25% open tile, experimental data are also shown in the
figure. The curve given by the formula is in good agreement with 0.05 0.47
0.10 1.87
the data. 共This figure assumes that the tiles are 2 ⫻ 2 ft2, which is
0.15 4.21
the common size used in the US. The size in other countries is 0.20 7.48
0.6⫻ 0.6 m2, which is not very different.兲 0.25 11.7
Table 1 gives, for the 25% open tile, the values of the airflow 0.30 16.8
rates through the tile and the corresponding pressure drops. Prac-

073001-4 / Vol. 132, JULY 2010 Transactions of the ASME

 Fig. 7 Maldistribution of airflow and its cause

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
It is now obvious that the perforated tiles represent only a small
fraction of the flow resistance experienced by the CRAC blower.
Therefore, for all practical purposes, the CRAC unit gives nearly
the same flow rate for different numbers, layouts, and open areas Fig. 9 Comparison of measured and calculated airflow rates
of the perforated tiles. 共This is an important conclusion and it is „the tiles are numbered from left to right…
often not understood even by experienced people in the data cen-
ter field. They attempt to increase the flow rate from perforated
tiles by using 40% open tiles instead of 25% open. As seen above, shown in Fig. 9. The maldistribution is clearly visible. For tiles 14
this action changes only the tiny pressure drop across the perfo- and 15, which are closest to the CRAC unit, the flow is actually
rated tiles. Since the other two major pressure drops remain the negative. The flow can be seen to increase rapidly for smaller tile
same, the CRAC unit delivers essentially the same total flow rate.兲 numbers 共which are further away from the CRAC unit兲. The
agreement between the measurements and calculation is very sat-
2.7 Surprising Airflow Distribution Through the Perfo- isfactory.
rated Tiles. As seen in Fig. 3, different perforated tiles are at
different distances from the CRAC unit. Does the airflow distrib- 2.9 Further Validations. The prediction of airflow rates
ute uniformly through different perforated tiles? At first sight, we through the perforated tiles by CFD simulation has been validated
may conclude that, as we go way from the CRAC unit 共which is by comparison with measurements in a number of data centers. A
the source of air兲, the airflow through the perforated tiles dimin- typical example is shown in Figs. 10 and 11. Here Fig. 10 shows
ishes. Actually, the flow distributes in a surprising and counterin- the layout of the data center, while Fig. 11 displays the compari-
tuitive manner. The perforated tiles that are farthest way from the son of calculated and measured airflow rates through the perfo-
CRAC unit get the largest flow. The airflow through the tiles close rated tiles. These figures, which are directly taken from Ref. 关16兴,
to the CRAC unit is quite small. contain “ft” as length units and “CFM” as the unit of volumetric
Figure 7 shows the so-called maldistribution of airflow in a flow rate through the perforated tiles. 共If necessary, the unit con-
schematic manner. The reason for this maldistribution is also ex- version can be accomplished rather easily.兲 Once again, the com-
plained in the figure. The mechanism is similar to the maldistri- puted results can be seen to agree quite well with the measured
bution that occurs in manifolds 关28,29兴. In Fig. 7, if we consider values.
the velocity of the under-floor air in the horizontal direction, this
velocity must decrease as air escapes through the perforated tiles. 3 Factors Affecting the Airflow Distribution
The Bernoulli equation would then imply that the pressure in-
creases as we go away from the CRAC unit. The airflow rate 3.1 Relationship Between the Flow Field in the Plenum
through the perforated tiles depends on the pressure drop. Since and Flow Rates Through Perforated Tiles. As we have seen
the tiles on the right experience a greater pressure drop, they before, the flow rate through a perforated tile depends on the
deliver more airflow. 共This assumes that the pressure above the pressure drop across the tile, that is, the difference between the
raised floor is nearly uniform. This is a valid assumption and has plenum pressure just below the tile and the ambient pressure
been verified by measurements and computation.兲 above the raised floor. Pressure variations above the raised floor
are generally small compared with the pressure drop across the
2.8 An Early Validation. The above discussion explains the perforated tiles. Thus, relative to the plenum, the pressure just
maldistribution in a qualitative sense. To verify its quantitative above the perforated tiles can be assumed to be uniform. The flow
accuracy, a simple experiment 关9兴 was conducted at IBM Corpo- rates, therefore, depend primarily on the pressure levels in the
ration in Poughkeepsie, NY. The experimental setup is shown in
Fig. 8. This includes a small data center with two CRACs and
many perforated tiles. For the initial tests, the CRAC unit on the
left was turned off. Then, only the unit on the right supplies cold
air to all the perforated tiles. The airflow from each perforated tile
was measured and compared with the airflow rate predicted by a
CFD simulation of the under-floor space. The comparison is

Fig. 8 Layout for a small test data center Fig. 10 Layout of a data center

Journal of Heat Transfer JULY 2010, Vol. 132 / 073001-5

 Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
Fig. 11 Comparison of measured and calculated airflow rates

plenum, and the nonuniformity in the airflow distribution is configuration shown in Fig. 12. This layout uses the conventional
caused by the pressure variations in the horizontal plane under the hot aisle/cold aisle arrangement, with the perforated tiles placed in
raised floor. the cold aisles. The CRAC units are also located in the cold aisles.
For the nonuniformity in the airflow distribution to be signifi- The server racks are arranged on both sides of the cold aisles, with
cant, the horizontal pressure variations 共or change in velocity their intake sides facing the cold aisles. The hot aisles are formed
heads兲 must be comparable to the pressure drop across the perfo- between the back ends of two rows of server racks. The cooling
rated tiles. This condition is satisfied if the area available for hori- air exiting the perforated tiles is sucked in by the internal fans of
zontal flow in the plenum is comparable to or less than the total the racks, heats up as it moves through the racks, and is exhausted
open area of the perforated tiles. from the back of the racks into the hot aisles. From the hot aisles,
the heated air returns to the inlets of the CRAC units. Due to
3.2 Parameters Considered. The key to controlling the air- symmetry, only a portion of the data center around one CRAC
flow distribution is the ability to influence the pressure variation in unit needs to be considered; this portion is shown in Fig. 12.
the plenum. For specified 共horizontal兲 floor dimensions and total Thus, the base configuration under consideration here consists
flow rate, the effect of the following parameters is significant: of a CRAC unit and two rows of perforated tiles, each containing
• plenum height 15 tiles with 25% open area. The CRAC unit delivers 4.72 m3 / s
• open area of perforated tiles 共10,000 CFM兲 of cold air. The under-floor plenum height is

Plenum height. The plenum height has a major influence on the

horizontal velocity and pressure distributions in the plenum. As
the plenum height increases, the velocities decrease and the pres-
sure variations diminish, leading to a more uniform airflow
distribution.
Open area of perforated tiles. As the open area of perforated
tiles is reduced, the pressure drop across the tiles increases and, at
some point, becomes much larger compared with the horizontal
pressure differences under the raised floor. Under these condi-
tions, all perforated tiles experience essentially the same pressure
drop and the airflow distribution becomes nearly uniform.
3.3 The Base Case. The effect of these parameters on the
airflow distribution will be illustrated with reference to the simple Fig. 12 The base case

073001-6 / Vol. 132, JULY 2010 Transactions of the ASME

 Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
Fig. 13 Flow rates through perforated tiles for the base case

0.3048 m 共12 in.兲. The tile size is 2 ⫻ 2 ft2. The overall dimen- spreading in the vicinity of the CRAC unit.
sions of this part of the data center are 36⫻ 14 ft2 共10.97 Figures 16 shows the velocity vectors and pressure distribution
⫻ 4.27 m2兲. on a horizontal plane just under the raised floor for plenum height
The distribution of airflow rates for this configuration is shown of 6 in. 共0.1524 m兲 For this smaller plenum height, the pressure
in Fig. 13. The flow rates are smaller near the CRAC unit and variations are more significant 共compared with Fig. 14兲, with an
increase toward the opposite wall. There is actually a reverse flow extensive region of negative pressure near the CRAC unit. Figure
through the perforated tiles next to the CRAC unit. 17 shows the same plot for plenum height of 24 in. 共0.6096 m兲.
Figure 14 shows the velocity vectors and the pressure distribu- Here, the pressure distribution is much more uniform; the varia-
tion on the horizontal plane just under the raised floor. The cold tions are limited to a small region near the CRAC unit. These
air exiting the CRAC unit impinges on the subfloor and expands pressure plots explain the airflow distribution shown in Fig. 15.
horizontally. In the impingement region, the pressure levels are 3.5 Effect of the Open Area of Perforated Tiles. To illus-
high, and they decrease rapidly as the air rushes out of these trate the effect of the open area of perforated tiles, the open area in
regions. As we move away from the CRAC unit, since the cold air the base configuration is varied from 10% to 60%.
is exiting the plenum, the horizontal velocity diminishes and the
pressure rises. Note that the pressure under the perforated tiles
next to the CRAC unit is negative 共that is, below the pressure in
the above-floor space兲 and produces a reverse flow through these
tiles.
3.4 Effect of the Plenum Height. To illustrate the effect of
plenum height on the airflow rates, the height for the base con-
figuration is varied from 6 in. 共0.1524 m兲 to 24 in. 共0.6096 m兲.
The flow rates for different plenum heights are shown in Fig.
15. It can be seen that the nonuniformity in flow rates is most
pronounced for plenum height of 6 in. 共0.1524 m兲 and diminishes
as the height is increased. The intensity of reverse flow through
the perforated tiles next to the CRAC unit also weakens as the
plenum height is increased. Note that the curves for the plenum
heights of 18 共0.4572 m兲 and 24 in. 共0.6096 m兲 are almost coin-
cident. This implies that, once the plenum height is large enough,
any further increase does not affect the flow distribution. In Fig.
15, even for the 24-in. 共0.6096 m兲 height, the flow distribution is
not quite uniform. This is due to the complex flow and lateral
Fig. 15 Effect of plenum height on the airflow distribution

Fig. 14 Pressure distribution and velocity vectors under the Fig. 16 Pressure distribution and velocity vectors under the
raised floor for the base case „plenum height= 12 in. „0.3048 raised floor for plenum height= 6 in. „0.1524 m… „the pressure
m…; the pressure values are in Pa… values are in Pa…

Journal of Heat Transfer JULY 2010, Vol. 132 / 073001-7

 4fL
frictional resistance parameter, ⌽ =
dh
In the definition of ⌿, ␩ is a geometrical factor defined as the
ratio of the total area of perforated tiles Atile to the cross-sectional
area of the plenum 共width⫻ height兲 Acs, that is,
Atile
␩=
Acs
The hydraulic diameter dh appearing in the definition of ⌽ is
taken as twice the plenum height. The friction coefficient f in-
cludes contributions from wall shear stress and the flow resistance

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
Fig. 17 Pressure distribution and velocity vectors under the of support structures and other distributed obstructions in the
raised floor for plenum height= 24 in. „0.6096 m… „the pressure plenum.
values are in Pa… A further simplification is possible by recognizing that, whereas
⌿ is an important parameter, the frictional parameter of ⌽ is
usually small and can be set equal to zero. Under these conditions,
The airflow distributions for different open areas are shown in the governing one-dimensional differential equations can be
Fig. 18. For a fixed layout and plenum height, the nonuniformity solved to get an analytical exact solution. This solution reduces
in flow rates diminishes as the open area is reduced. A reduction the complex problem of airflow in the under-floor space to a
in the open area also reduces the likelihood of reverse flow near simple formula, which has surprisingly good validity.
the CRAC units. Note that there is no reverse flow for open area While considering this simplification, we must remember that it
of 15% and 10%. applies to the simple configuration shown in Fig. 12. Whereas we
It may appear that using highly restrictive tiles 共such as 10% get valuable guidance, understanding, and insight form the ideal-
open兲 is a good way of making the airflow distribution uniform. ized model, it cannot be directly applied to real data centers with
However, there is an undesirable side effect. At smaller open ar- complex layouts, several CRAC units with intersecting flow
eas, the pressure levels in the plenum increase, and a large pro- streams, and irregular distribution of perforated tiles.
portion of cold air escapes through extraneous openings on the
floor, e.g., openings around cables and pipes and other leakage
paths. 共The flow resistance of these openings now becomes com- 4 Leakage Through the Raised Floor
parable to the flow resistance of the perforated tiles.兲 This wasted In practical data centers, the raised floor is not quite an imper-
air will not be available for cooling of equipment. meable surface as one would normally assume in an analysis.
3.6 One-Dimensional Idealization. The CFD simulations There are deliberate openings made in the tiles to bring cables to
shown above are based on a three-dimensional analysis of veloc- the server racks. 共In fact, historically, the raised-floor arrangement
ity and pressure in the under-floor space. However, for the simple was as much for bringing electrical connections as for supplying
configuration chosen here, the significant variations are only in the cooling air.兲 In addition, there are always small gaps between
x direction 共left to right兲. For modest heights of the plenum, the individual floor tiles. The leakage area due to these gaps is esti-
variations in the vertical direction 共between the subfloor and the mated to be 0.2% of the floor area.
In a normal data center, about 10% of the floor area is occupied
raised floor兲 are usually small. In the y direction, significant varia-
by perforated tiles. If we assume that these tiles are 25% open,
tions are found only near the CRAC unit. Based on these obser-
then the open area of the perforated tiles is 2.5% of the floor area.
vations, it is possible to construct a one-dimensional model of the
For this case, the leakage area mentioned above is 8% the total
situation designated above as the base case. Such a model has
open area. If we reduce the number of perforated tiles on the floor,
been described by Karki and Patankar 关24兴. In this model, the
then the leakage area becomes an even larger proportion of the
whole family of solutions is dependent on only two dimensionless
total open area.
parameters, which are
If we ignore the leakage area, then we would expect that, as the
␩ number of perforated tiles decreases, the plenum pressure would
pressure variation parameter, ⌿ =
冑K increase and the average airflow rate through each perforated tile
would be larger. These expectations are qualitatively in agreement
with actual measurements, but quantitatively the pressure rise and
the increase in airflow rate are not found to be as high. The reason
for the discrepancy is the leakage area. As the number of perfo-
rated tiles decreases, the leakage area becomes comparable to the
open area of the perforated tiles. As a result, the leakage airflow
must be taken into account while estimating the plenum pressure
and the average airflow rate through each perforated tile.
A detailed study of the effect of leakage is presented by Rad-
mehr et al. 关21兴 and Karki et al. 关22兴. The study includes careful
measurements performed in a data center with and without dis-
tributed leakage from the floor. The measurements are also com-
pared with the results of CFD simulations.

5 Effect of Under-Floor Obstructions

5.1 Types of Under-Floor Obstructions. In the under-floor
space in a data center, usually there are many pipes, cables, struc-
tural beams, and other objects. Their presence reduces the area
Fig. 18 Effect of open area of perforated tiles on the airflow available for the airflow and creates nonuniformities in the pres-
distribution sure distribution. These are substantial effects and must be in-

073001-8 / Vol. 132, JULY 2010 Transactions of the ASME

Fig. 19 A circular pipe as an under-floor obstruction „only the
centerline of the pipe is shown…

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
Fig. 21 Pressure distribution and velocity vectors under the
raised floor for the case of the circular-pipe obstruction „the
pressure values are in Pa…
cluded in a CFD simulation.
In addition to the obstructions that are present for other reasons
共bringing in chilled water or electrical power兲, we can consider
placing deliberate obstructions for the purpose of controlling the deceasing the area available for the under-floor flow兲, then the
airflow distribution. This is an inexpensive way of getting the pressure would remain uniform. Consequently, a uniform airflow
desired flow distribution. This concept will be illustrated in this distribution can be achieved through the perforated tiles.
section. Figure 22 shows the base case modified by the placement of
5.2 Effect of a Circular Pipe as an Obstruction. As a dem- two inclined partitions in the under-floor space. The partitions
onstration of how an under-floor obstruction influences the airflow attempt to reduce the available area for the horizontal flow in a
distribution through the perforated tiles, a modification to the base linear fashion. However, in order to avoid intersecting the final
case is considered. As shown in Fig. 19, a circular pipe of 6-in. perforated tiles, the final area at the right end is kept finite and not
共0.15 m兲 diameter is placed in the under-floor space as an obstruc- made zero. The resulting airflow distribution through the perfo-
tion. 共Only the centerline of the pipe is shown in the figure. In the rated tiles is presented in Fig. 23, along with the distribution for
12-in. height of the raised floor, the pipe is placed on the solid the no-partition case. It can be seen that the use of the inclined
subfloor leaving a 6-in. clearance above it.兲 Fig. 20 shows the partitions leads to a much more uniform airflow distribution com-
resulting airflow distribution through the perforated tiles. The cor- pared with the base case. The only nonuniformities are in the
responding velocity vectors and pressure distribution just under region close to the CRAC unit 共due to the rather complex flow
the raised floor are displayed in Fig. 21. In general, the pipe al- there兲 and at the right end 共due to our inability to make the area
lows the pressure to build up in the upstream region. So, the flow reduce to zero兲.
rates close to the CRAC unit are not as small as in the base case. There is a valid criticism of the use of solid impermeable par-
In the immediate upstream region of the pipe, air velocity in- titions that lead to closed regions in the under-floor space. For a
creases 共due to the blockage caused by the pipe兲 and pressure data center with many CRAC units, the possibility of failure of
falls. Then, there is a further lowering of the pressure on the one of the CRAC units is always present. In absence of solid
downstream side of the pipe. Thereafter, the pressure gradually partitions, other CRAC units would supply some of the air to the
builds up as the horizontal flow velocity diminishes. These pres- perforated tiles that are normally supplied by the failed CRAC
sure changes are reflected in the airflow distribution through the unit. Solid partitions 共such as the inclined ones used in Fig. 22兲
perforated tiles. prevent such cross transfer and are likely to turn a normal failure
into a catastrophic one.
5.3 Use of Inclined Partitions. As mentioned above, deliber- From this point of view, a perforated plate can be a more suit-
ate obstructions can be used to control the airflow distribution. A able partition. A perforated partition does not completely stop the
convenient obstruction is a vertical partition placed in the under- flow through it; however, the extra flow resistance offered by the
floor space. The flow misdistribution in the base case occurs be- partition can be used to discourage the flow in one place and
cause the horizontal flow velocity deceases as the air emerges consequently to encourage the flow in other places. The use of
from the perforated tiles. If this velocity can be kept uniform 共by perforated partitions is discussed next.

Fig. 20 Flow rates through perforated tiles as affected by the circular-pipe obstruction

Journal of Heat Transfer JULY 2010, Vol. 132 / 073001-9

 Fig. 22 Use of inclined partitions in the under-floor space

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
Fig. 24 Use of perforated partitions in the under-floor space
5.4 Use of Perforated Partitions. The flow maldistribution
in the base case delivers large airflow rates through the perforated
tiles located far way from the CRAC unit. The flow distribution
can be made more uniform by placing perforated partitions nor- 6 Above-Floor Effects
mal to the horizontal flow so that the flow will be somewhat
discouraged from moving fast to the downstream region. This 6.1 The Main Consideration. As already mentioned above,
concept is shown in Fig. 24 with two proposed locations for the in a data center, the goal of cooling is to ensure that the maximum
partitions. The required percent open area of the partitions will be inlet temperature to any server rack does not exceed the allowable
determined by computational experiments. inlet temperature. The purpose of measurement and/or simulation
In Fig. 25, the results of airflow distribution are shown for the in the above-floor space is to verify whether this requirement is
case of the two perforated partitions set at 70% and 30% open, satisfied. The primary cause of high inlet temperatures is insuffi-
respectively. Also shown for reference are the results for the no- cient airflow supplied through the perforated tile at the foot of the
partition case. It can be seen that the use of the partitions has server rack. If the airflow demand of the rack cannot be met by the
increased the airflow in the tiles near the CRAC unit. At each perforated tile, the upper part of the rack draws in hot air. To
partition location, there is a drop in the airflow rate due to the illustrate this, a small section of a data center is shown in Fig. 28,
pressure drop across the partition. Actually, the “discouragement” in which insufficient cooling airflow is supplied. For this situation,
of the flow is so strong that the six tiles furthest away from the the calculated inlet temperatures for the racks are displayed in Fig.
CRAC unit have a rather small flow through them. To get a uni- 29. 共The color scale used for plotting the temperature contours is
form flow, we need to adjust the percent open area of the parti- included in Fig. 29. The same color scale is used for all the re-
tions. However, if we do desire a large airflow for tiles 1–9 and a maining figures that show a temperature distribution. For this rea-
small airflow for tiles 10–15, then the current arrangement is quite son, the color scale is not repeated in those figures.兲 Since the
satisfactory. In general, the perforated partitions can be used to get cooling airflow is insufficient, all racks have hot air entering the
any desired distribution of airflow rates, not just a uniform tops of their inlets. Figure 30 shows where this hot air originates.
distribution. It is the hot exhaust of the rack that gets recirculated into the inlet
Figure 26 shows a further attempt with the partitions set at 75% when the cooling air is insufficient. This was shown earlier in Fig.
and 50%. The airflow distribution has improved, but is still not 5 via a simple example.
quite uniform. The case in Fig. 27 uses partitions at 80% and 65% If sufficient cooling airflow is supplied at the foot of the server
open area. The resulting airflow distribution is quite uniform. In rack, usually there is no reason for unsatisfactory cooling. How-
fact, except for tile 1 共which is affected by the very complex flow ever, there are special locations and circumstances in which hot
in the vicinity of the CRAC unit兲, all other tiles give nearly the air can enter the inlets even when sufficient cold airflow is sup-
same airflow. plied at the perforated tiles. Some of these are described below.

Fig. 23 Airflow distribution with and without inclined partitions

073001-10 / Vol. 132, JULY 2010 Transactions of the ASME

 Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
Fig. 25 Airflow distribution with and without perforated partitions „70% and 30%…

Fig. 26 Airflow distribution with and without perforated partitions „75% and 50%…

Fig. 27 Airflow distribution with and without perforated partitions „80% and 65%…

Journal of Heat Transfer JULY 2010, Vol. 132 / 073001-11

 Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
Fig. 28 A simple data center model with one CRAC, several Fig. 31 Temperature distribution on the inlet faces of the racks
racks, and perforated tiles „insufficient cooling airflow… with increased cooling airflow

6.2 End Effects. Suppose that in Fig. 28, the airflow was
increased to meet the demands of the racks. The corresponding
calculated inlet temperatures are displayed in Fig. 31. It can be
seen that most of the racks in the middle have acceptable low inlet
temperatures over the whole inlet face. However, higher inlet tem-
peratures are seen for the racks near the CRAC unit and also for
the racks furthest away from the CRAC unit. The behavior near
the CRAC is easy to understand. As we have seen before, the
usual maldistribution of airflow leads to small airflow rates at the
perforated tiles near the CRAC. This insufficient cooling flow
leads to the higher inlet temperatures for the racks in that region.
Far away from the CRAC unit, the perforated tiles deliver the

Fig. 32 Temperature distribution and velocity vectors on a

horizontal plane

Fig. 33 Temperature distribution on the inlet faces of the racks

Fig. 29 Temperature distribution on the inlet faces of the racks with an air curtain
„insufficient cooling airflow…

Fig. 30 Temperature distribution and velocity vectors on a Fig. 34 Temperature distribution and velocity vectors on a
plane „insufficient cooling airflow… horizontal plane „with an air curtain…

073001-12 / Vol. 132, JULY 2010 Transactions of the ASME

 Drop Ceiling

Server Racks Server Racks

CRAC

Fig. 38 A schematic of ducted racks with drop ceiling

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
racks. An obvious remedy is to close the gaps by using imperme-
Fig. 35 Use of partitions to prevent hot air from entering the able plates or partitions.
inlets of the racks
6.4 High-Velocity Flow Through the Perforated Tiles. The
heat loads of modern server racks can be very high 共10–20 kW兲
and the corresponding airflow demand may be of the order of
highest airflow rates. So, at first glance, the behavior of the racks 1.0 m3 / s. At these flow rates, air emerges from the perforated tile
in that region is difficult to understand. Figure 32 shows a plot of at a velocity of 3 m/s. When this high-velocity stream flows over
temperature distribution on a horizontal plane near the top of the the inlet face of the rack, would the cooling air enter the rack or
racks. Although the perforated tiles in the end region 共away from simply flow past it? This is a valid concern. Radmehr et al. 关25兴
the CRAC兲 deliver sufficient flow, the hot exhaust air finds its way considered this issue and performed a detailed analysis of the
around the last rack and enters the rack inlets. situation. It was shown that the high-velocity airflow does create a
One quick remedy for this effect is shown in Fig. 33. Here, low-pressure region at the bottom of the rack. This means that the
additional perforated tiles are placed in the end region to create an server fans in the bottom region deliver a lower flow rate com-
air curtain of cold air. The corresponding inlet temperatures in pared with the uniform-pressure environment. Fortunately, this
Fig. 33 can be seen to be an improvement over the distribution in flow reduction is not large. The results in the paper indicate that,
Fig. 31. Figure 34 shows how the air curtain is effective in pre- for realistic values of the flow resistance inside a server rack and
venting the recirculation of hot air into the rack inlets. for common fan curves, the flow reduction at the bottom of the
6.3 Gaps Between the Racks. Normally, the racks are placed rack is less than 15%.
in a row in a contiguous manner. However, occasionally, there 6.5 Use of Above-Floor Partitions. The success of cooling in
may be gaps between them. For example, in practice, gaps are a data center depends on keeping the hot air away from the inlets
created by removing a rack from a row. It is easy to see that the of the server racks. This can be partially arranged by placing solid
gaps provide additional places where the “end effects” can be partitions in appropriate places. Figure 35 shows some possible
observed. Hot air from the back of the racks can enter the cold arrangements. In the two configurations at the back, the top or the
aisle through the gaps and influence the inlet temperatures of the sides of the cold aisle are closed. This would prevent hot air
entering the cold aisle from the top or from the side. The arrange-
ment in the middle tries to prevent both top and side recirculation.

Drop Ceiling

Server Racks Server Racks

CRAC

Fig. 36 A schematic of the drop-ceiling arrangement

Fig. 37 Use of the drop ceiling Fig. 39 Layout of the NCEP data center

Journal of Heat Transfer JULY 2010, Vol. 132 / 073001-13

 Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
Fig. 40 Comparison of measured and computed airflow rates through perforated tiles

The arrangement at the right of the figure is an attempt to deal portant. The cold air can be supplied anywhere in the room; it will
with the effects of a bad original layout. In this layout, the hot find its way to the inlets of the racks without getting “diluted” by
exhaust from one set of racks blows into the inlet faces of the next any hot air.
set of racks. 共The hot aisle/cold aisle arrangement is not used.兲
The partition between the two rows of racks helps to keep the hot 6.8 Concerns About the Ducted Solutions. Although the
air away from the next set of inlets. need for reliable and assured cooling has made the arrangements
The use of partitions is an inexpensive way of keeping the hot like drop ceiling and ducted racks quite popular, they create some
air away from the inlets of the racks. An ultimate use of partitions concerns. Originally, the attractions of the raised-floor design
is the arrangement known as the ceiling plenum or the use of a were its simplicity and flexibility. One could easily move server
drop ceiling. This is described next. racks to new locations and simply place perforated tiles next to
them to provide cooling air. No ducting was involved. The use of
6.6 Drop Ceiling. Figure 36 shows a schematic of the drop- a drop ceiling requires ducting of the return airflow to the CRAC
ceiling arrangement, in which a false ceiling is created below the units. This makes it difficult to relocate the CRACs. Moving any
real ceiling. Above the hot aisles, vents are placed in the drop racks to new locations requires the relocation of the vents on the
ceiling to take the hot air to the space above the drop ceiling. drop ceiling in addition to the movement of perforated tiles.
From this space, the hot air is ducted to the top of the CRAC The use of ducted racks presents additional problems. Relocat-
units. A three-dimensional picture of the use of the drop ceiling is ing the racks is now even more difficult. Further, the flow rate
shown in Fig. 37. This arrangement creates a near-perfect separa- provided by the CRAC blowers should match the flow rate de-
tion between the hot and cold air and ensures satisfactory cooling. manded by the internal fans in the server racks. Otherwise, the
6.7 Ducted Racks. Some designs go one step further. Instead spaces above and below the drop ceiling will be at very different
of using vents above the hot aisle to capture the hot air, exhaust pressures. This will cause the CRAC blowers and the server fans
ducts are attached to the racks to transport hot air to the space to operate at abnormal conditions. If, for the same set of CRACs,
above the drop ceiling. This arrangement is schematically shown one adds or removes a number of server racks, a significant dis-
in Fig. 38. It guarantees complete separation between hot and cold parity in flow rates will arise.
air. As long as the total amount of required airflow is supplied These problems can be partially handled by 共i兲 ensuring that the
through the perforated tiles, proper cooling of all racks is assured. total flow rate provided by the CRACs is always equal to or
The flow distribution through the perforated tiles is no more im- greater than the airflow demanded by the server fans and 共ii兲 pro-

073001-14 / Vol. 132, JULY 2010 Transactions of the ASME

 Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
Fig. 41 Comparison of measured and computed temperatures at the rack inlets

viding vents in the drop ceiling 共in addition to the ducted returns any hot-air streams in the data center and ensures proper cooling.
from the racks兲. The excess cold airflow will pass through these There are also overhead-cooling units that draw in hot air from the
vents and will help to reduce the pressure difference between the hot aisle, cool it, and blow cold air downwards into the cold aisle.
spaces above and below the drop ceiling.
The most serious concern is what happens when one or more 6.10 A Non-Raised-Floor Data Center. Nearly all the dis-
CRAC units fail. Then the above-mentioned matching of the flow cussion in this paper has focused on a raised-floor data center,
rates does not hold any more. Since the airflow exhausted by the which represents the most common design. However, there are
server fans cannot be handled by the remaining CRAC units, hot non-raised-floor data centers used for special applications. For
air will flow from the space above the drop ceiling into the room example, telecommunications equipment is so heavy that often it
below through the extra vents provided. This hot air can directly cannot be conveniently mounted on a raised floor. When there is
enter the server racks causing a catastrophic failure. no raised floor, the cooling air may come from upflow CRAC
units 共as opposed to the downflow units used in a raised-floor
6.9 Advanced Cooling Solutions. The increasing demands environment兲, from overhead ducts, and from the advanced cool-
on data centers have led to a number of unconventional cooling ing solutions mentioned above.
solutions. These include in-row coolers, rear-door heat exchang-
ers, and overhead-cooling units. An in-row cooler is similar to a
rack and is placed in the middle of a row of racks. It draws in hot 7 Validation of Above-Floor Simulation
air from its back side 共from the hot aisle兲, internally cools it, and As a sample of how computed results agree with measurements,
exhausts cold air into the cold aisle. This cold air then enters the a comparison with the measurements by Schmidt 关20兴 is shown
inlets of the surrounding server racks. A rear-door heat exchanger here. Figure 39 shows the layout of a data center at the National
is mounted literally as a rear door for a rack. The normal hot Center for Environmental Protection 共NCEP兲, Bethesda, MD. Fig-
exhaust from the rack is cooled in this heat exchanger so that the ure 40 gives a plot of computed and measured flow rates through
exhaust air is at an acceptably low temperature. This eliminates the perforated tiles in selected rows. Figure 41 presents the com-

Journal of Heat Transfer JULY 2010, Vol. 132 / 073001-15

 Acknowledgment
The author would like to thank his colleague Dr. Kailash C.
Karki for his significant help in the preparation of this paper.

Nomenclature
Atile ⫽ total area of the perforated tiles
Acs ⫽ cross-sectional area of the plenum
dh ⫽ hydraulic diameter
f ⫽ friction factor
F ⫽ fractional open area for the perforated tile
⫽

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
K pressure-loss factor
L ⫽ streamwise distance
Fig. 42 Comparison of measured and computed return tem- p ⫽ pressure
peratures for the CRAC units V ⫽ velocity approaching the perforated tile
␩ ⫽ area ratio
␳ ⫽ density of air
⌽ ⫽ frictional resistance parameter
parison of computed and measured temperatures at the rack inlets ⌿ ⫽ pressure variation parameter
at a height of 1.68 m 共5.5 ft兲 from the floor 共for selected rows兲.
Finally, Fig. 42 compares the computed and measured values of
the CRAC return temperatures for the seven CRAC units in the References
data center. In general, the agreement can be seen to be very good. 关1兴 Seymour, M., 1992, “Dynamic Design on Atria,” Building Services, Chartered
Institution of Building Services Engineers 共CIBSE兲, pp. 27–29.
关2兴 Awbi, H. B., and Gan, G., 1994, “Prediction of Airflow and Thermal Comfort
in Offices,” ASHRAE J., 36共2兲, pp. 17–21.
8 Work of the ASHRAE TC 9.9 Committee 关3兴 Kiff, P., 1995, “A Fresh Approach to Cooling Network Equipment,” British
Telecommunications Engineering, Institution of British Telecommunications
This paper has described various fundamental and practical Engineers, London.
challenges in cooling a data center. To make a concerted effort on 关4兴 Bullock, C. E., and Phillip, F., 1996, “The Sistine Chapel: HVAC Design for
resolving many technical issues and to provide leadership and Special-Use Buildings,” ASHRAE J., 38共4兲, pp. 49–58.
关5兴 Schmidt, R. R., 1997, “Thermal Management of Office Data Processing Cen-
standardization, the Technical Committee TC 9.9 共Mission Criti- ters,” InterPack ‘97, Hawaii.
cal Facilities, Technology Spaces, and Electronic Equipment兲 was 关6兴 Cinato, P., Bianco, C., Licciardi, L., Pizzuti, F., Antonetti, M., and Grossoni,
established in 2002 under the auspices of American Society of M., 1998, “An Innovative Approach to the Environmental System Design for
Heating, Refrigerating, and Air-Conditioning Engineers TLC Rooms in Telecom Italia,” Intelec 98.
关7兴 Quivey, B., and Bailey, A. M., 1999, “Cooling of Large Computer Rooms—
共ASHRAE兲. The committee has representatives from over 110 Design and Construction of ASCI 10 TeraOps,” InterPack 1999, Maui, HI.
organizations, which include manufacturers and users of data cen- 关8兴 Kang, S., Schmidt, R. R., Kelkar, K. M., Radmehr, A., and Patankar, S. V.,
ter equipment, consulting companies, and related agencies. The 2001, “A Methodology for the Design of Perforated Tiles in Raised Floor Data
committee has made important contributions to ASHRAE hand- Centers Using Computational Flow Analysis,” IEEE Trans. Compon. Packag.
Technol., 24共2兲, pp. 177–183.
books and has published seven books 关30–36兴 in the ASHRAE 关9兴 Schmidt, R. R., Karki, K. C., Kelkar, K. M., Radmehr, A., and Patankar, S. V.,
Datacom Series on topics directly related to data centers. These 2001, “Measurements and Predictions of the Flow Distribution Through Per-
books represent a valuable resource for cooling and related issues forated Tiles in Raised-Floor Data Centers,” ASME Paper No. IPACK2001-
in a data center. 15728.
关10兴 Schmidt, R. R., 2001, “Effect of Data Center Characteristics on Data Process-
ing Equipment Inlet Temperatures,” ASME Paper No. IPACK2001-15870.
关11兴 Schmidt, R. R., and Cruz, E., 2002, “Raised Floor Computer Data Center:
Effect on Rack Inlet Temperatures of Chilled Air Exiting Both the Hot and
9 Concluding Remarks Cold Aisles,” IEEE 2002 Inter Society Conference on Thermal Phenomena,
This paper has dealt with a number of issues pertaining to the pp. 580–594.
关12兴 Schmidt, R. R., and Cruz, E., 2004, “Cluster of High-Powered Racks Within a
airflow and cooling in data centers. For a raised-floor data center, Raised-Floor Computer Data Center: Effect of Perforated Tile Flow Distribu-
the flow field in the under-floor space holds the key to the distri- tion on Rack Inlet Temperatures,” ASME J. Electron. Packag., 126, pp. 510–
bution of airflow through the perforated tiles. If the airflow de- 518.
mand of each server rack is met by supplying the required airflow 关13兴 Patel, C. D., Bash, C. E., Belady, C., Stahl, L., and Sullivan, D., 2001, “Com-
putational Fluids Dynamics Modeling of High Compute Density Data Centers
at the foot of the rack, proper cooling is, in general, assured. The to Assure System Air Inlet Specifications,” ASME Paper No. IPACK2001-
airflow distribution through the perforated tiles is governed by the 15622.
pressure variation under the raised floor. This is affected by the 关14兴 Patel, C. D., Sharma, R., Bash, C. E., and Beitelmal, A., 2002, “Thermal
height of the raised floor, the locations of the CRAC units, the Considerations in Cooling Large Scale High Compute Data Centers,” ITHERM
2002, Proceedings of the Eighth Intersociety Conference on Thermal and
layout of the perforated tiles, their open area, and the presence of Thermomechanical Phenomena in Electronic Systems.
under-floor obstructions. Whereas some obstructions are present 关15兴 Karki, K. C., Patankar, S. V., and Radmehr, A., 2003, “Techniques for Con-
as a practical necessity, deliberate placement of obstructions 共such trolling Airflow Distribution in Raised-Floor Data Centers,” ASME Paper No.
as perforated partitions兲 can be used to influence the flow field in IPACK 2003-35282.
关16兴 Karki, K. C., Radmehr, A., and Patankar, S. V., 2003, “Use of Computational
a desirable way. Fluid Dynamics for Calculating Flow Rates Through Perforated Tiles in
In the above-floor space, the goal is to prevent any hot-air Raised-Floor Data Centers,” HVAC&R Res., 9共2兲, pp. 153–166.
stream from reaching the inlets of the server racks. This is prima- 关17兴 Patankar, S. V., and Karki, K. C., 2004, “Distribution of Cooling Airflow in a
rily achieved by supplying sufficient cold air at the perforated Raised-Floor Data Center,” ASHRAE Trans., 110共2兲, pp. 629–635.
关18兴 Guggari, S., Agonafer, D., Belady, C., and Stahl, L., 2003, “A Hybrid Meth-
tiles. In addition, air curtains, partitions, drop ceiling, and ducted odology for the Optimization of Data Center Room Layout,” Paper No.
racks can be used to achieve a separation between the hot and cold IPACK2003-35273.
air streams. In special circumstances, advanced cooling solutions 关19兴 Bash, C. E., Patel, C. D., and Sharma, R. K., 2003, “Efficient Thermal Man-
that provide localized cooling can be used. agement of Data Centers—Immediate and Long-Term Research Needs,”
HVAC&R Res., 9共2兲, pp. 137–152.
Cooling in a data center is a topic of enormous practical impor- 关20兴 Schmidt, R. R., 2004, “Thermal Profile of a High-Density Data Center—
tance. It also leads to many physical, mathematical, and compu- Methodology to Thermally Characterize a Data Center,” ASHRAE Trans.,
tational issues that are very fascinating. 110共2兲, pp. 635–642.

073001-16 / Vol. 132, JULY 2010 Transactions of the ASME

关21兴 Radmehr, A., Schmidt, R. R., Karki, K. C., and Patankar, S. V., 2005, “Dis- 关28兴 Bajura, R. A., and Jones, E. H., 1976, “Flow Distribution in Manifolds,”
tributed Leakage Flow in Raised-Floor Data Centers,” ASME Paper No. ASME J. Fluids Eng., 98, pp. 654–666.
IPACK2005-73273. 关29兴 Majumdar, A. K., 1980, “Mathematical Modelling of Flows in Dividing and
关22兴 Karki, K. C., Radmehr, A., and Patankar, S. V., 2007, “Prediction of Distrib- Combining Flow Manifolds,” Appl. Math. Model., 4, pp. 424–432.
uted Air Leakage in Raised-Floor Data Centers,” ASHRAE Trans., 113共1兲, pp.
关30兴 ASHRAE, 2004, Thermal Guidelines for Data Processing Environments
219–226.
共ASHRAE Datacom Series兲, ASHRAE, Atlanta, GA.
关23兴 Van Gilder, J., and Schmidt, R. R., 2005, “Airflow Uniformity Through Per-
forated Tiles in a Raised-Floor Data Center,” ASME Paper No. IPACK2005- 关31兴 ASHRAE, 2005, Datacom Equipment Power Trends and Cooling Applications
73375. 共ASHRAE Datacom Series兲, ASHRAE, Atlanta, GA.
关24兴 Karki, K. C., and Patankar, S. V., 2006, “Airflow Distribution Through Perfo- 关32兴 ASHRAE, 2005, Design Considerations for Datacom Equipment Centers
rated Tiles in Raised-Floor Data Centers,” Build. Environ., 41共6兲, pp. 734– 共ASHRAE Datacom Series兲, ASHRAE, Atlanta, GA.
744. 关33兴 ASHRAE, 2006, Liquid Cooling Guidelines for Datacom Equipment Centers
关25兴 Radmehr, A., Karki, K. C., and Patankar, S. V., 2007, “Analysis of Airflow 共ASHRAE Datacom Series兲, ASHRAE, Atlanta, GA.
Distribution Across a Front-to-Rear Server Rack,” ASME Paper No. 关34兴 ASHRAE, 2008, Structural and Vibration Guidelines for Datacom Equipment
InterPack2007-33574, pp. 8–12.
Centers 共ASHRAE Datacom Series兲, ASHRAE, Atlanta, GA.

Downloaded from http://asmedigitalcollection.asme.org/heattransfer/article-pdf/132/7/073001/5794918/073001_1.pdf by Indian Institute of Science - Bangalore user on 17 February 2025
关26兴 Sullivan, R. F., “Alternating Cold and Hot Aisles Provides More Reliable
Cooling for Server Farms,” a White Paper From the Uptime Institute, Incor- 关35兴 ASHRAE, 2008, Best Practices for Datacom Facility Energy Efficiency
porated, Santa Fe, NM. 共ASHRAE Datacom Series兲, ASHRAE, Atlanta, GA.
关27兴 Idelchik, I. E., 1994, Handbook of Hydraulic Resistance, CRC, Boca Raton, 关36兴 ASHRAE, 2008, High Density Data Centers—Case Studies and Best Practices
FL. 共ASHRAE Datacom Series兲, ASHRAE, Atlanta, GA.

Journal of Heat Transfer JULY 2010, Vol. 132 / 073001-17