46th International Conference on Environmental Systems ICES-2016-006

10-14 July, Vienna, Austria

Assessment of Thermal Balance Test Criteria Requirements

on Test Objectives and Thermal Design

John W. Welch1
The Aerospace Corporation, 2310 El Segundo Blvd., El Segundo, CA 90245

The primary purposes of a space vehicle thermal balance test are to gather thermal test
data to be used in thermal model correlation and to verify the design and performance of the
thermal control subsystem. During the vehicle thermal balance test, test phases simulate hot
and cold flight environmental conditions and the vehicle is allowed to reach equilibrium in an
energy balance with the environmental conditions. Following the test, data from each thermal
balance test phase are compared to thermal model predictions and adjustments are made to
the thermal model to result in more accurate agreement. Criteria for temperature stabilization
are used during the test to establish the equilibrium conditions and the test data are used for
thermal model correlation. Criteria for the thermal model correlation process are used to
establish when the thermal model has sufficient agreement with test data. Both of these criteria
set a “good enough” threshold, one for establishing equilibrium and the other for correlating
the thermal model. In this paper, these criteria values are assessed to determine their impact
on the thermal design and temperature margins used in the design.

Nomenclature
t = time
tstab = time at temperature stabilization
T = temperature
To = temperature at time zero
Tss = steady-state temperature
τ = thermal time constant

I. Introduction

P rior to launch, high priority United States Air Force space programs are required to demonstrate mission
capabilities in a vehicle thermal vacuum test. One part of that test applicable to qualification and protoqualification
vehicles is the thermal balance test. This test (1) gathers thermal test data to be used in post-test thermal model
correlation and (2) verifies the design and performance of the thermal control subsystem. During the test, hot and cold
flight environmental conditions are simulated in test phases and the vehicle reaches steady-state temperatures in an
energy balance with these environments. These test temperatures are compared to pre-test predictions of the test phase
conditions as a preliminary assessment of thermal control functionality and performance. Following the test, data from
each thermal balance test phase are compared to thermal model predictions of each phase and adjustments are made
to the thermal model to result in more accurate agreement. When the thermal model is correlated with data from each
test phase, a final set of mission temperature predictions are generated for a formal assessment of the thermal design
and a verification that temperature predictions are within allowable thermal limits.
Requirements for conducting the thermal balance test for U.S. military programs are specified in MIL-STD-1540E
(released as SMC-S-0016 [1] and TR-RS-2014-00016 [2]). The requirements include how many thermal balance test
phases will be conducted, how the vehicle should be thermally configured, and thermal equilibrium criteria to establish
when each thermal balance test phase is complete. The requirements also state criteria to establish the adequacy of the
thermal model correlation. Together these criteria set “good enough” thresholds, one for establishing equilibrium and
the other for establishing the thermal model accuracy. In this paper, these criteria values are assessed to determine
their impact on the thermal design and temperature margins used in the design. The paper will also discuss how these
criteria impact risk to the thermal design.

1
Associate Director, Spacecraft Thermal Department, Mail Stop: M4-916.

1
46th International Conference on Environmental Systems
10-14 July, Vienna, Austria

II. Thermal Stabilization Criteria

Thermal balance testing requires the test article to reach steady-state thermal conditions so that the equilibrium
temperatures can be compared to temperature predictions from the thermal model. If the thermal balance test phase is
not taken to a sufficient equilibrium state, the difference in the actual equilibrium temperature and the “assumed”
equilibrium temperature in the test adds to the uncertainty in the correlation process. However, an impractical test
duration would be required to establish actual equilibrium conditions, so it is necessary to define thermal stabilization
criteria for thermal balance testing that attempts to minimize this uncertainty without imposing unreasonable cost and
schedule constraints on the test program. The stabilization criteria are intended to establish an “approximate”
equilibrium condition adequate for thermal model correlation.
For thermal balance testing, MIL-STD-1540E states that thermal stability is achieved when the unit having the
largest thermal time constant has a temperature rate of change of less than 1 °C measured over 5 hours, that is, 0.2
°C/hr [1]. A comparison of thermal balance test stabilization criteria across industry reveals that some are more
restrictive than the MIL-STD-1540E requirement while others are less restrictive. Furthermore, some are general and
treat the criteria as more of a guideline for conducting thermal balance testing with latitude given to test planners and
thermal engineers. A summary of the values found in literature is provided in Table 1. The criteria differ in both the
rate of temperature change and in the test duration over which the rate of change is measured. The NASA Marshall
criterion is too restrictive to be useful in practical applications, and are meant to be a general statement of intent rather
than a verifiable test parameter. The criteria for NASA Goddard and the European Space Agency (ESA) are also more
restrictive than the MIL-STD-1540E criteria. Because some of these criteria are not rigorous requirements, actual
stabilization criteria used in thermal balance tests conducted by these organizations may be significantly different
from what shown in Table 1.

Table 1. Summary of Temperature Stabilization Criteria for Thermal Balance Testing

Organization Temperature Stabilization Criteria References and Notes

US Air Force < 0.2 °C/hr as measured over 5 hours [1], Stated as a requirement
NASA Goddard < 0.05 °C/hr as measured over no less than 6 hours [3], Note (1) below
NASA Langley < 0.5 °C/hr as measured over 1 hour [5]
NASA Marshall “Tenths or hundredths of a degree” [6]
Jet Propulsion Laboratory < 0.3 °C/hr as measured over 3 hours [7, 8], Note (2) below
European Space Agency < 0.1 °C/hr as measured over 5 hours [9, 10], Note (3) below
JAXA < 0.3 °C/hr [11], Note (4) below
Notes:
(1) Also exhibiting a decreasing temperature slope over that period. Alternatively, another stabilization
criterion which may be used is where the amount of energy represented by the time rate of temperature
change (and the thermal mass of the test article) is a small fraction (typically 2 to 5%) of the total energy
of the test article [3]. Actual programs are closer to < 0.1 °C/hr as measured over 1 hour [4].
(2) For hardware with time constants less than 4 hours, but inadequate for large thermal mass items [8]
(3) Concession is made of “relaxation for items with low thermal inertia”
(4) JAXA: Japanese Aerospace Exploration Agency

Most criteria shown in Table 1 specify a minimum rate of temperature change, dT/dt, and most also include what
the dt needs to be. Several sources permit extrapolating the test temperatures to an “assumed” steady-state value for
use in thermal model correlation, but do not state how that might be done. Some of the sources suggest using a more
rigorous approach of defining a temperature stabilization criterion by considering the thermal mass and heat transfer
paths of the test article. These methods are similar in that for an individual temperature sensor, a thermal time constant
is computed and temperature transition rates are extrapolated based upon preliminary test temperature data toward an
equilibrium temperature. These techniques recognize that for temperature sensors mounted to items of large mass or
thermally isolated from the environment, the stabilization criteria may be satisfied at a temperature significantly
different from its equilibrium value. A rigorous approach to defining the temperature stabilization criteria can be found
in a study by Colizzi [12]. A simpler method is outlined succinctly by Rickman and Ungar [13] whereby thermal time
constants and temperature rates of change are computed during the test and used to extrapolate to a target steady-state
temperature. It can be used to more accurately specify dT/dt values near steady-state temperatures.

2
46th International Conference on Environmental Systems
10-14 July, Vienna, Austria

While these extrapolation techniques are not difficult and can be automated with real-time test temperatures, there
is generally a reluctance among thermal test engineers to employ these methods of approximating steady-state test
temperatures. One reason for this caution is the opinion that the additional work does not achieve noticeable test time
savings. Another concern is that these methods prove very accurate for extremely simple configurations, but in
complex geometries, rarely will a temperature smoothly transition to its equilibrium value. In actual spacecraft
applications where heaters may be turning on and off, power dissipations are not uniform, and bus voltages and
environmental boundary temperatures will vary or fluctuate with time, calculating the temperature difference with
small time steps (dt) increases the uncertainty in the determination of the steady-state temperature. Thermal engineers
will likely agree that these extrapolation methods are technically more accurate estimates of equilibrium temperatures,
but in all practicality, temperature stabilization criteria similar to those shown in Table 1 are still needed to simplify
test procedures and operations.
A thermal balance test phase is complete when all temperature sensors (test thermocouples and flight thermistors)
are within the test’s equilibrium criteria. Sensors mounted to high mass items that have poor conductive and radiative
heat transfer paths will be last to meet the criteria. Although the number of temperature sensors can be in the hundreds,
there are typically only a few that are associated with high mass items or poor heat transfer paths (leading to large
thermal time constants), and these are usually known prior to the test and are associated with large thermal time
constants. Rickman and Ungar [13] showed that:

𝑑𝑇
𝑇𝑠𝑠 − 𝑇 = 𝜏 [1]
𝑑𝑡

where Tss is the steady-state temperature, T is the sensor temperature, τ is the thermal time constant, and dT/dt is the
average transition rate between time steps. The expression to the left of the equal sign is effectively the difference
between the equilibrium temperature and the test temperature. It can be viewed as the error in assuming that T is Tss
during the correlation process or as a goal temperature difference for defining a reasonable approximation of T for Tss
to determine when the test phase is complete. Values for τ can be approximated prior to the test by multiplying the
model node mass by its specific heat and dividing by the sum of the conductances (conduction and radiation) of each
heat transfer path from or to this node. These values of τ can be used to estimate whether the selected thermal
stabilization criteria are adequate to achieve the desired goal temperature difference. A simple example is shown in
Fig. 1. If a goal temperature difference of 1 °C is desired and the largest thermal time constant computed at the various
test temperature sensors is 5 hours, Fig. 1 indicates that a thermal equilibrium criteria of 0.2 °C/hr or less is required.
During the thermal test, refinements in τ will be necessary based upon actual test data, so computation of Tss becomes
an iterative process.
Equilibrium Criteria for Temperature
Rate of Change, dT/dt, (°C/hr)

Thermal Time Constant, τ (hr)

Figure 1. Thermal Equilibrium Criteria for Two Goal Temperature Differences

3
46th International Conference on Environmental Systems
10-14 July, Vienna, Austria

As to the appropriateness of the values listed in Table 1, the differences in actual and “assumed” steady-state test
temperatures (Tss – T) for data used in the correlation process need to be within the thermal model correlation criteria
for a credible correlation to be achieved. The thermal balance test stabilization criteria satisfy this goal when the
temperature rate of change has decreased below a specified value over a sufficient duration of test time. With a typical
correlation goal of ±3 °C or less for electronic boxes and critical vehicle units, the “assumed” equilibrium temperature
should be well within the correlation goal value. A value such as ±1 °C would therefore be reasonable for this
difference. The criteria parameters shown in Table 1 were selected by the different organizations to meet this intent.
To assess the validity of the selections, this report turned to actual thermal balance test data.
Figure 2 shows test data from a thermal balance test of a large military spacecraft approaching to a steady-state
warm test temperature. The data represent a typical temperature transition from a cold to a hot thermal balance
condition for a radiator equipment panel on a large spacecraft. The temperatures are from thermocouple readings on
three different equipment panels with electronic units mounted internally, honeycomb panels with embedded heat
pipes, and external radiator surfaces. These readings were chosen because they represent some of the larger thermal
time constants used in establishing thermal equilibrium for this test. The data from this test were selected because
thermal balance test phases were extremely long with final temperatures (at hour 38) computed (using [13]
methodology) to be within 0.1 °C of equilibrium values. With such long stabilization periods, the appropriateness of
different stabilization criteria can be measured and assessed.

F Panel
Test Temperature (°C)

L Panel

D Panel

Thermal Balance Test Time (hr)

Figure 2. Thermal Balance Test Data Transitioning from Cold into Hot Thermal Balance

Using the tabular temperature readings for the three thermocouples, the first time at which the various criteria are
satisfied were found. At these times, the panel temperatures were then compared to the computed equilibrium
temperatures (using [13]). These results are shown in Table 2. The final column computes the average temperature
difference from equilibrium for the three thermocouple readings, by simply averaging the three panel temperature
differences. To illustrate how the results in Table 2 were computed, applying the MIL-STD-1540E stabilization
criteria, the D Panel first recorded 5 hours of < 0.2 ºC/hr at 22 hours, and at this time, the temperature of the D Panel
is 3.8 ºC, 1.5 ºC colder than the actual equilibrium temperature of 5.3 ºC. Applying the ESA criteria, the D Panel first
recorded 5 hours of < 0.1 ºC/hr at 31 hours, and at this time, the D Panel temperature is 4.8 ºC, only 0.5 ºC colder than
5.3 ºC. The NASA Goddard criteria could not be satisfied for the D and L Panels, even after 38 hours of test time, so
for these calculations, the temperatures at 38 hours was used.

4
46th International Conference on Environmental Systems
10-14 July, Vienna, Austria

Table 2. First Times and Temperatures when Stabilization Criteria were satisfied

First Time Criteria Temperature when Criteria Temperature Difference

Organization
Satisfied (hr) Satisfied (°C) from Equilibrium (°C)
D Panel F Panel L Panel D Panel F Panel L Panel D Panel F Panel L Panel Avg.
U.S. Air Force 22 26 26 3.8 9.7 8.4 1.5 0.6 1.1 1.1
NASA Goddard > 38 35 > 38 5.3 10.1 9.5 0.0 0.2 0.0 0.1
NASA Langley 10 11 14 0.9 5.1 5.4 4.4 5.2 4.1 4.6
Jet Propulsion Lab 14 21 18 2.2 8.8 6.7 3.1 1.5 2.8 2.5
ESA 31 33 34 4.8 10.3 9.1 0.5 0.0 0.4 0.3
JAXA 13 13 14 1.9 5.8 5.4 3.4 4.5 4.1 4.0
Equilibrium - - - 5.2 10.3 9.4 - - - -

As expected, results from Table 2 show that the stabilization criteria with the largest temperature differences are
from the least restrictive NASA Langley and JAXA requirements. Their criteria are satisfied after 14 hours with an
average temperature difference from computed equilibrium to be 4.6 °C and 4.0 °C, respectively. The criteria for the
Jet Propulsion Laboratory (JPL) required 18 hours and had a temperature difference of 2.5 °C. The MIL-STD-1540E
criteria was met after 26 hours of testing with an average temperature difference of 1.1 °C. The most restrictive criteria
(ESA and NASA Goddard) required significantly more test time, 34 and more than 38 hours, respectively, but as
expected, their average temperature differences were extremely small.
Figure 3 plots the thermal balance test data from Fig. 2 along with the earliest time each criteria were satisfied.
The average temperature differences are also provided. The plot shows that the ESA and NASA Goddard criteria
provide test temperatures that are very nearly equal to steady-state conditions, but require test times to reach these
conditions significantly longer than the other organizations’ criteria.

NASA Langley
14 hours to stabilize
4.6°C to equilibrium

JAXA
14 hours to stabilize
4.0°C to equilibrium
Test Temperature (°C)

ESA
JPL 34 hours to stabilize
21 hours to stabilize 0.3°C to equilibrium
2.5°C to equilibrium

MIL-STD-1540E
26 hours to stabilize
1.1°C to equilibrium
NASA Goddard
>38 hours to stabilize
0.1°C to equilibrium

Thermal Balance Test Time (hr)

Figure 3. Thermal Balance Test Time with First Times when Various Equilibrium Criteria Satisfied

With a stabilization goal of about 1 °C or less, Table 2 shows that the MIL-STD-1540E criteria are near the
threshold of the goal, the NASA Goddard and ESA criteria are well within the goal and the other criteria fall short of
the goal. Several criteria are not adequate in that for hardware with relative large thermal mass, the criteria result in
“assumed” stabilization temperatures quite different from actual stabilization temperatures. While the NASA Goddard
criteria had the smallest temperature differences to the equilibrium values, a test time of more than 38 hours may be
viewed as excessive relative to a 1 ºC goal. In comparing the test times to the temperature profiles in Fig. 3, it is clear
that at 26 hours, the MIL-STD-1540E temperatures have not achieved steady-state conditions, but for thermal model

5
46th International Conference on Environmental Systems
10-14 July, Vienna, Austria

correlation purposes, the difference in “assumed” and actual equilibrium test temperatures is well within the thermal
model correlation goal. If the goal of a thermal balance test stabilization criteria is to achieve a reasonable equilibrium
temperature (e.g., within 1 °C of the actual equilibrium value) for all vehicle temperature sensors, then the MIL-STD-
1540E criteria achieve this goal.
The NASA Langley criteria specified the largest allowed temperature rate of change as measured over only one
hour. The JAXA criteria did not specify a duration to measure the temperature rate of change, so it was assumed to be
also over only one hour. Test data are rarely smooth and the tabulated data from Fig. 1 showed instances when the
criteria were satisfied at one time, but then fell out of compliance for several hours. The JPL criteria use a 3-hour
duration window, but at only 0.3 °C/hr, the criteria were met well before temperatures were near equilibrium values
(2.5°C average difference to equilibrium temperatures). The results presented indicate that a time duration of greater
than 3 hours is necessary to mitigate the effects of temperature fluctuations. A duration of 5 hours as per MIL-STD-
1540E seems quite appropriate.
These criteria comparisons are not intended to discredit any organization or their approach in establishing test
requirements. The stability criteria were judged using test data from a space vehicle with heavy equipment panels and
relatively large thermal time constants. The conclusions drawn in the previous paragraphs may not apply for small
spacecraft with simpler geometries that achieve equilibrium quicker. The NASA Langley, JPL and JAXA criteria may
be perfectly suited for the spacecraft developed by these groups. Each organization needs to assess their own criteria
to determine if their requirements are adequate in meeting test objectives. The primary purpose of this paper was to
assess the MIL-STD-1540E requirements using military test data, develop rationale for these requirements, and
determine how these requirements compare to those used by other organizations.

Extrapolating Out to an Equilibrium Temperature

There are some discussions [12] of improving thermal model correlation by extrapolating to an assumed
equilibrium temperature using thermal balance test data up to the point that thermal stabilization is declared. The
nearer the test data are to the actual equilibrium temperature, the more accurate the extrapolation will be. As a further
assessment of the thermal balance stabilization criteria, the temperature profiles were extrapolated using data points
up to the stabilization temperature indicated by the respective criteria. Assuming a temperature profile of the form:

𝑇 = 𝑇𝑜 + ∆𝑇(1 − 𝑒 −𝜏𝑡 ) [2]

where To is the temperature at time zero, and ΔT is the difference between the “assumed” equilibrium temperature and
To. Values for ΔT will vary with each stabilization criteria because each criteria establishes its own “assumed”
equilibrium temperature. Thermal time constants, τ, were computed for each panel using test temperature data up to
the time that the stabilization criteria was satisfied. Resulting profiles are shown in Figures 4a, 4b and 4c for
stabilization times of 14, 21, 26 and 34 hours. These stabilization times correspond to when stabilization criteria were
met and would indicate the end of the thermal balance test phase. Values for To, ΔT and τ are summarized in Table 3.

6 12
5 11
10
4
9
3 8
Test Temperature (°C)

Test Temperature (°C)

2 7
tstab = 14, 21, 26, 34, equilibrium
1 6
0 5 tstab = 14, 21, 26, 34, equilibrium
4
-1 tstab = 14 hr tstab = 14 hr
tstab = 21 hr 3 tstab = 21 hr
-2 tstab = 26 hr 2 tstab = 26 hr
tstab = 34 hr tstab = 34 hr
-3 tstab = equilibrium 1 tstab = equilibrium
-4 0
5 10 15 20 25 30 35 40 5 10 15 20 25 30 35 40

Time (hr) Time (hr)

Figure 4a. Temperature Profile Extrapolating Figure 4b. Temperature Profile Extrapolating
from Thermal Balance Stabilization (D Panel) from Thermal Balance Stabilization (F Panel)

6
46th International Conference on Environmental Systems
10-14 July, Vienna, Austria

10
9
8
Test Temperature (°C)

7
6 tstab = 14, 21, 26, 34, equilibrium
5
4
3
2 tstab = 14 hr
1 tstab = 21 hr
tstab = 26 hr
0 tstab = 34 hr
-1 tstab = equilibrium
-2
5 10 15 20 25 30 35 40
Time (hr)

Figure 4c. Temperature Profile Extrapolating

from Thermal Balance Stabilization (L Panel)

The figures indicate that the calculated temperature profiles (Eq. [2]) were reasonable representations of the
temperature data. The test data of Fig. 2 are also shown in the Fig. 4 profiles, but over most of the time, it is coincident
with the curve given by the stabilization time of 34 hours. As one would expect, using a stabilization time of 34 hours
resulted in an extrapolated temperature profile that best matched the actual test data, so much so that the two curves
are indistinguishable in Fig. 4. The extrapolated curves for a stabilization time of 14 hours were noticeably different
than the other curves with the greatest difference from the actual test data. Table 4 summarizes the differences in
temperatures between extrapolated values and test data at a time of 38 hours. The curves using stabilization times of
26 and 34 hours resulted in temperature differences of less than 0.5 °C, while the curve using a stabilization time of
21 hours resulted in temperature difference of less than 1 °C.

Table 3. Computed Values for To, ΔT and τ

Parameter D Panel F Panel L Panel

Stabilization
14 21 26 34 14 21 26 34 14 21 26 34
Time (hr)
To (ºC) -7.3 -7.3 -7.3 -7.3 -4.8 -4.8 -4.8 -4.8 -8.2 -8.2 -8.2 -8.2
ΔT (ºC) 11.0 11.8 12.1 12.4 12.7 15.4 15.1 15.2 15.8 16.9 17.3 17.5
τ (hr) 0.141 0.119 0.113 0.108 0.141 0.090 0.089 0.094 0.141 0.119 0.113 0.110

Table 4. Extrapolation Data from Thermal Balance Stabilization Temperatures

Difference between Extrapolated Temperature

Stabilization Extrapolated Temperatures (°C)
and Assumed Equilibrium Temperature (°C)
Time (hr)
D Panel F Panel L Panel D Panel F Panel L Panel
14 3.7 7.9 7.6 1.5 2.4 1.8
21 4.5 10.6 8.7 0.7 0.3 0.7
26 4.8 10.7 9.1 0.4 0.4 0.3
34 5.1 10.4 9.3 0.1 0.1 0.1
Equilibrium * 5.2 10.3 9.4 - - -
* Assumed equilibrium temperature based upon test data

It is clear from the extrapolated data that a stabilization criteria of 0.3 °C/hr measured over one hour (corresponding
to a 14-hour stabilization time for this test data) is inadequate for establishing or extrapolating an equilibrium thermal
balance test temperature meaningful for thermal model correlation to achieve the 1°C difference goal. This conclusion
is based upon temperature differences as high as 2.4 °C (Table 4). Such a difference would make thermal model
correlation difficult and would increase uncertainty in the correlation and subsequent temperature predictions. The
same criteria, but measured over longer time period (e.g., 3 hours), improves the confidence in the equilibrium

7
46th International Conference on Environmental Systems
10-14 July, Vienna, Austria

temperature, but may still be problematic for areas where thermal model correlation is difficult or where high
confidence is needed. The extrapolated test data indicates that a stabilization criteria of at least 0.2 °C/hr measured
over a time period greater than 3 hours (corresponding to a stabilization time of at least 26 hours) is necessary to
establish an equilibrium temperature within 0.5 °C of actual test data.

III. Thermal Model Correlation Criteria

Thermal model correlation is the process where adjustments are made to the thermal model to achieve better
agreement with thermal balance test data taken at the end of each thermal balance test phase. The purpose of the
correlation is to improve and quantify the accuracy of the thermal model using test data, thus increasing confidence
that the thermal control subsystem will meet thermal requirements over the mission life of the vehicle and that the
thermal model accurately predicts temperatures in flight environments. Test data consist of temperatures, heater
power, heater duty cycle, and any boundary conditions (bus voltage, environmental flux data, chamber wall, plate
temperatures, etc.) recorded when the thermal balance stability criteria were satisfied for each thermal balance test
phase.

Thermal Model Correlation Criteria Values

The thermal model correlation criteria establishes a requirement for a reasonable correlation of thermal model
temperature predictions with thermal balance test data, acknowledging that there are inherent uncertainties in the
thermal balance test process and in thermal modeling analysis assumptions. The correlation effort compares test
temperature data and model predictions for the same vehicle locations under the same thermal environments and
makes adjustments to the model thermal features to result in better temperature prediction agreement with the test
data. The goal of the criteria is to provide a reasonable allowance for these uncertainties such that there is still
confidence that the thermal model accurately predicts temperatures and heat transfer through the vehicle. Common
thermal model correlation criteria used in industry are provided in Table 5. Similar to the temperature stability criteria
discussed previously, several of the criteria are general recommendations (instead of requirements) and differences
within the same organization may be common.

Table 5. Thermal Model Correlation Criteria used in Industry

Organization Thermal Model Correlation Criteria References

U.S. Air Force Within ±3 °C (explanation for exceedances) [1]
Contractor A 90% within ±5 °C
NASA Goddard Compute ±1 °C and a standard deviation of ±2.5 °C (explanation for [4]
those outside 5 °C). Percentage errors computed for < 3 °C and <5 °C [5]
NASA Marshall Within ±5 °C [6]
Jet Propulsion Laboratory Within ±5 °C [14]
European Space Agency Typical: <5 °C for internal units and <10 °C for external units [9, 10]
Mean deviation within ±2°C, standard deviation <3 °C
JAXA Typically <5 °C, but not standard [11]

The MIL-STD-1540E correlation criteria associated with U.S. Air Force space programs are the most stringent of
the criteria shown in Table 5 in that the criteria applies to all test data points (The NASA Goddard criteria are also
stringent with different levels of acceptance). The MIL-STD-1540E criteria specify that comparison between all
temperatures be within ±3 °C of thermal model temperature predictions. When this cannot be satisfied, an explanation
is required stating why the criteria could not be achieved for that particular location and thermal condition. The most
common criteria in Table 5 are within ±5 °C and some of these criteria specify what percentage of measurements need
to be within correlation temperature ranges.

Assessment of the Thermal Model Correlation Criteria Values

The rationale for the MIL-STD-1540E requirement is based upon a goal of having the model correlation error well
within standard thermal uncertainty margins, which is ±11 ºC [1]. The thermal uncertainty margin is added to thermal
model temperature predictions, and accounts for inherent uncertainties in the thermal analysis properties, which

8
46th International Conference on Environmental Systems
10-14 July, Vienna, Austria

include interface resistances, thermal blanket performance, thermal model nodalization, radiation view factors,
material properties, surface degradation, etc. The model correlation error is an additional error in the ability to predict
vehicle temperatures and historically, as long as it was within the test tolerance (±3°C), the errors that resulted from
the thermal model correlation process were not added to the required thermal model uncertainty. The thermal
uncertainty margin used for U.S. military programs is specified in MIL-STD-1540E as ±11 °C, and the validity of this
margin for current spacecraft programs has been established [15]. A correlation error of 3 °C is slightly more than a
fourth of the 11 °C thermal uncertainty margin.
The most common thermal uncertainty margin used by the other sources in Table 5 is ±10 °C, so a correlation error
of ±5 °C represents one half of the thermal uncertainty margin. In other words, in some situations, half of the thermal
uncertainty margin will be used up by the thermal model correlation error. The implications of this can be realized by
better understanding the intent of the thermal uncertainty margin. The ±11 °C margin (MIL-STD-1540E) in intended
to provide, in general use, a 95% likelihood that flight temperatures will be within ±11 °C of pre-flight temperature
predictions from a thermal model correlated to thermal balance test data. In a worst-case situation, reducing this margin
by the maximum allowed thermal correlation error (±3 °C) to ±8 ºC reduces that comparison likelihood from 95% to
about 85% [16]. This is a three-fold increase in the likelihood that a flight temperature will be more than 11°C warmer
or colder than its pre-flight temperature prediction. However, because the maximum correlation error is equivalent to
the allowable tolerance for temperatures, the thermal design is nevertheless said to comply with ±11 °C uncertainty
requirement.
Using the same approach with a 10 °C thermal uncertainty margin and a maximum correlation error of 5 °C, the
likelihood of being within the temperature limits for any particular location is reduced from 93% to 64% [16]. This
suggests that more than one third of spacecraft telemetry locations may be more than 10°C warmer or colder than pre-
flight temperature predictions. For some customers, this may be an acceptable risk position, while for others, it may
be unacceptable. For military applications, unless mitigating explanations were provided, a third of locations being
more than 10 ºC outside of predictions would constitute a medium to high risk to the thermal design for those particular
locations. However, MIL-STD-1540E allows for explanations in situations when the ±3 °C correlation error cannot
be achieved, and this process needs to be utilized in reducing the thermal design risk and better understanding what
the actual thermal margin might be. Typical explanations include inadequate knowledge of the environmental
conditions, inadequate instrumentation to verify test data, and additional temperature margin to allowable limits.
For government military space programs, the rationale for having the thermal model correlation error well within
the thermal uncertainty margin centers on confidence that model predictions will accurately reflect flight temperatures.
When this does not occur and hardware is operating near allowable limits, mitigating steps, such as operational
constraints, are taken to keep flight hardware within acceptance temperature ranges. This may include changing the
operational configuration of the vehicle, re-orienting the vehicle to change environmental heating loads, modifying
heater set point values, and operating units in a different manner (including turning units off for a period of time). For
military spacecraft, such changes are strongly undesirable in that they can affect the user’s ability to support their
intended activities. For this reason, it is necessary to limit the thermal model correlation error to a small portion of the
thermal uncertainty margin.
A criteria listed in Table 5 (that provided by Contractor A) specify a percentage of temperature readings to be
within model predictions, specifically that 90% of all comparisons be within 5 °C. There are two primary concerns
with specifying a percentage as done in this example. First, the percentage typically applies to all temperature
measurements with no assessment as to the criticality of the location. Not all temperature sensor locations are of equal
importance. Temperature comparisons on electronic units should receive greater scrutiny in meeting the correlation
criteria than structural units or hardware with large thermal margins. As written, there is no emphasis on ensuring that
locations of high importance are correlated with sufficient focus. Second, the criteria make no provision as to what
should be done for areas that are outside this correlation goal. It appears that as long as 90% of measurements meet
this criteria, the correlation is deemed successful, and no further explanations are required. In other words, a correlation
error could be very large for an important location, but as long as the 90% goal is met across the vehicle, there may
be no motivation to reduce the large correlation error at this one location.

Additional Notes Regarding the Thermal Model Correlation Criteria

Correlation accuracy is typically displayed on a histogram plot such as the example shown in Figure 5. This method
of displaying correlation results is useful for illustrating how well the correlation is centered about zero and what the
standard deviation might be. The average correlation error is not the mean of the correlation accuracy, but rather is
the mean of the absolute values of the correlation accuracy data. The average correlation error provides insight into
the accuracy of the correlation.

9
46th International Conference on Environmental Systems
10-14 July, Vienna, Austria

In the example provided in Figure 5, the average correlation

accuracy is about -0.4 ºC, and the average correlation error is
1.7 °C. The average correlation error will always be equal to or

Error Distribution (%)

greater than the average correlation accuracy. This value is
within ±3 °C correlation criteria, so on average, the correlation
is compliant to the requirement. For a government space
program using the MIL-STD-1540E correlation criteria
requirement, an explanation would be required for the data
points that could not meet the criteria, namely the data points
with correlation errors of -6 °C, -5 °C, and -4 °C.
Even when the explanations are reasonable, the lack of Temperature (Deg. C)
compliant correlation will still result in an increased likelihood
of temperature limit violations in flight. Many times, the Figure 5. Correlation Accuracy Histogram
contractor will assist the customer in understanding the impact
of this risk, but it is ultimately the customer’s decision to either accept the increased risk (fly-as-is) or reject the
increased risk (redesign or improve correlation). Considerations used to assess this risk is described in the following
section.

Assessing Risk Associated with a Non-Compliant Thermal Model Correlation

When the thermal model correlation is completed with all temperature predictions within ±3 °C of the thermal
balance test data for all test phases (ideally), the thermal model is said to be correlated and is appropriate to predict
final flight thermal predictions. These final predictions verify thermal requirements related to temperature margins
and heater duty cycle control authority. In practice, nearly all thermal model correlations are completed with many
temperature comparisons greater than the ±3 °C criterion. A point is reached in the thermal model correlation effort
when further adjustments to the model may increase the overall uncertainty in the correlation, rather than reduce it,
especially when changes are made in model assumptions with low uncertainty. When this point is reached, the final
correlation errors should be tabulated and explanations provided for differences greater than the correlation criteria.
The impact of thermal model correlation uncertainty of the thermal design should be assessed. The capability of
predicting flight temperatures for model locations that were correlated to within ±3 °C are considered accurate to
within the thermal tolerance for temperature; no additional design margin is necessary for model locations correlated
to within ±3 °C. For locations with correlation errors greater than ±3 °C, the assessment needs to consider the
possibility that required thermal uncertainty margin may be reduced. It may be reduced in that the modeling
assumptions do not accurately (greater than 3 °C) reflect the actual thermal condition and in that the correlation error
may be a larger than desired percentage of the thermal uncertainty margin. For each comparison location with a
correlation error greater than ±3 °C, consideration should be given to:
 Whether the location is associated with critical or non-critical hardware. Critical hardware are units that have
relatively constraining allowable temperature ranges, such as electronic units, moving mechanical
assemblies, batteries, etc. Non-critical units will typically have very wide temperature ranges, such as
structural surfaces and radiators. Non-critical units may reasonably use a relaxed correlation error greater
than ±3 °C because these items typically have large thermal margins and are not instrumented well in thermal
balance tests.
 Whether the correlation error is biased conservatively or non-conservatively. A conservative correlation error
would be where the model prediction is hotter than the test data in a hot environment thermal balance test
phase or where the model prediction is colder than the test data in a cold environment thermal balance test
phase. A non-conservative correlation error would tend to reduce the thermal uncertainty margin. Two
correlation errors of the same value should be assessed very differently if one is biased conservatively and
the other non-conservatively with regard to increasing mission risk.
 Whether adjacent temperature readings confirm the trend seen in the non-compliant temperature comparison.
If the comparison is inconsistent with the adjacent readings and there is no obvious explanation for the large
difference in temperature between the test and predicted values, this may not raise as much concern as one
where adjacent sensors confirm the unexplained trend.
 Whether the trend is consistent in other thermal balance test phases. Similar to the previous consideration, if
a non-compliant comparison in one thermal environment shows good correlation agreement in other thermal
test environments, this single non-compliant comparison may not raise the same level of concern as compared

10
46th International Conference on Environmental Systems
10-14 July, Vienna, Austria

to a consistent trend in the data with other test phases. This consideration is only applicable when there are
sufficient thermal balance test phases over a reasonably wide range in thermal environments. For example,
it may not be relevant when the non-compliance is from a hot operational thermal balance test phase while
the only other test phases represent nominal or cold cases because heat loading and dissipation paths can vary
considerably between hot and cold cases.

These considerations can help reduce the number of non-compliant comparison points of concern in the assessment.
One method of judging the severity of the concern is by using larger correlation criteria for the purpose of assessing
risk. While an explanation should be provided for all temperature comparisons greater than ±3 °C, for assessing risk,
the ±3 °C criterion may be applied to only non-conservative errors on critical hardware. For assessment purposes,
consideration might be given to allowing less restrictive criteria for conservative errors on critical hardware and non-
conservative errors on non-critical hardware (perhaps ±5 °C) and for conservative errors on non-critical hardware
(perhaps ±10 °C).
The next step in the assessment process is to compare the correlation errors to the thermal margins for each non-
compliant location. If a correlation error is less than the excess margin (in addition to the required thermal uncertainty
margin) for the hardware at that location, then the risk to the thermal design at that location is not significant. If it is
more, then that difference is how much the required thermal uncertainty margin is reduced by. In both cases, the
thermal uncertainty margin is reduced, but in the latter situation, the thermal requirement of demonstrating a minimum
thermal uncertainty margin is not being satisfied. The difference should be more than 3 °C because that is the accepted
test tolerance. For example, consider a location with a correlation error of 7 °C from a hot environment thermal balance
phase. This is significantly greater than the ±3 °C correlation goal, so this is potentially a problem to the thermal
design for hardware at this location. If the unit associated with this location has a worst-case hot temperature prediction
of 45 °C and an acceptance hot limit of 61 °C, it has a 5 °C margin in addition to the 11 °C thermal uncertainty margin.
A comparison of the 7°C correlation error with the 5 °C analysis margin indicates that in flight, if the 7°C bias were
to occur, it would decrease the thermal uncertainty margin by 2 °C (reducing the 11 °C margin to 9 °C). A 2 ºC
difference is within the test tolerance of 3 °C, so this can be assessed as a low risk to the thermal design for this unit
in hot case environments. A similar process can be used for all other non-compliant correlation points.
The considerations and process summarized in this section are a simplistic assessment approach to quantifying
thermal design risk resulting from the thermal model correlation process. It is recognized that in practice, other
considerations, such as a program’s risk posture, schedule, and cost constraints can take precedence over the technical
assessments made. It is ultimately the customer’s decision to either accept or reject the thermal design based upon
how well thermal requirements are being satisfied.

IV. Thermal Balance Test Phase Placement in Test Profile

Thermal balance testing is conducted as part of the vehicle thermal vacuum test, either prior to the cycling portion
of the thermal vacuum test or during the first cycle of the thermal vacuum test. Conducting the test prior provides a
clear distinction between thermal balance test phases and thermal vacuum test phases, whereas conducting the test as
part of the first cycle results in some schedule savings because portions of the temperature transitions can be combined.
MIL-STD-1540E has no preference as to whether the thermal balance test phases are conducted prior to or as part of
the first cycle of the thermal vacuum test.
There have been some instances when a test profile is proposed with the thermal balance test as part of the last
cycle of the thermal vacuum test. In these instances, test planners have asserted that the final cycle is preferable
because thermal features, such as thermal interfaces, thermal blankets, etc., better represent the flight configuration
on the last cycle because they have been “worn-in” during the previous cycles of the thermal vacuum test. MIL-STD-
1540E does not require that thermal balance test phases occur in the first portion of the thermal vacuum test, but there
are advantages with doing so, including:
 Thermal balance testing on the first cycle detects problems with the thermal control subsystem early so that
if a chamber break is necessary for rework, there is less of an issue with repeating previously accomplished
test phases.
 Thermal balance testing on the first cycle provides an opportunity to modify the thermal design in the event
of a chamber break during the thermal vacuum portions of the test. Small changes to the thermal design can
be made during the chamber break and then verified when the thermal vacuum test resumes.
 Thermal balance testing on the first cycle places less schedule pressure on the thermal balance stabilization
process. Typically at the end of the thermal vacuum test, there is increased pressure to complete the test and

11
46th International Conference on Environmental Systems
10-14 July, Vienna, Austria

the required hours waiting for temperature stabilization may be scrutinized from a schedule perspective as
not absolutely necessary.
 Thermal balance testing on the first cycle gives thermal engineers more time to review and assess the thermal
balance test data collected. If concerns arise with the sufficiency of the data while the thermal vacuum test is
still in process, there may be an opportunity to repeat a portion of the thermal balance test.
 Thermal balance testing on the first cycle enables more thermal model correlation time. Because the vehicle
thermal vacuum test is last of the environment tests, there is increased schedule pressure to resolve thermal
vacuum test observations and complete the thermal model correlation before shipping the vehicle from the
factory. If model correlation can begin while the thermal vacuum test is still in progress, this can prove
beneficial for completing the correlation in a more reasonable time. Furthermore, if the thermal vacuum test
needs to be halted (planned or unplanned due to hardware fixes or modifications), thermal analysts can take
advantage of this time to begin the thermal model correlation process.

It is understood that many of these advantages may not be possible given test schedules and personnel availability,
but the advantages discussed are potentially significant and should be considered in developing a thermal vacuum test
profile. As to the position that thermal control features need to be “worn-in”, there is little evidence that thermal
control features respond differently from the first cycle to the last. In nearly identical thermal environments on the
first and last cycles of thermal vacuum testing, there appears to be no evidence of performance differences in thermal
control hardware significant enough to attribute to a “wearing-in” process.

V. Conclusions

Two key thermal balance test requirements were reviewed and assessed in this paper: the thermal balance
stabilization criteria and the thermal model correlation criteria. It was found that for typical large complex space
vehicles such as those developed for the U.S. Air Force, thermal balance stabilization criteria need to specify a
sufficiently low temperature rate of change and a time duration over that rate of change to ensure that temperatures
are reasonably stabilized for subsequent thermal model correlation. The criteria stated in MIL-STD-1540E requiring
a temperature rate of change of 0.2 °C/hr as measured over 5 hours is a reasonable requirement without incurring
unnecessary test time. A thermal model correlation of no higher than 3 °C for critical vehicle locations provides a
rational goal of effort without cutting into excessively into the thermal uncertainty margin. Finally, conducting the
thermal balance test phases either prior to or on the first cycle of the thermal vacuum test has advantages that should
outweigh any benefits of testing on later vacuum cycles.

VI. References

1. Test Requirement for Launch, Upper-Stage and Space Vehicles, Air Force Command, Space and Missile Center
Standard, SMC Standard SMC-S-016, 5 September 2014.

2. Perl, E., Peterson, A. J., Snyder J. M., Wright, C. P. and Welch, J. W., Test Requirements for Launch, Upper-
Stage, and Space Vehicles, Aerospace Report No. TR-RS-2014-00016, 25 June 2014.

3. General Environmental Verification Standard for GFSC Flight Programs and Projects, GFSC-STD-7000A,
(Revision A), NASA Goddard Space Flight Center, 22 April 2013.

4. Harrison, R., SEPT/SIT Thermal Balance Test Plan, STEREO-GFSC-002.C, NASA Goddard Space Flight
Center, Undated.

5. Killough, B. R., NASA LaRC Thermal Testing Philosophy, NASA Langley Research Center, Thermal Test Panel
Discussion presented at 14th Thermal/Fluids Analysis Workshop (TFAWS), 21 August 2003.

6. General Environmental Test Guidelines (GETG) for Protoflight Instruments and Experiments, MSFC-HDBK-
670, NASA Marshall Space Flight Center, 01 June 1991.

7. Tsuyuki, G., JPL’s Thermal Testing Philosophy, Thermal Test Panel Discussion, presented at the 14th
Thermal/Fluids Analysis Workshop (TFAWS), 21 August 2003.

12
46th International Conference on Environmental Systems
10-14 July, Vienna, Austria

8. Siebes, G., System Thermal Testing (D-22011), Jet Propulsion Laboratory, Document ID 58172, 15 March 2002.

9. Space Engineering – Mechanical – Part 1: Thermal Control, European Space Agency, ECSS-E-30 Part 1A, 25
April 2000.

10. Space Engineering – Thermal Control General Requirements, ECSS-E-ST021C, ESA-ESTEC, 15 November
2008.

11. Iwata, N., et. al., Solar Simulation Tests of the X-ray Astronomy Satellite ASTRO-H, 43rd International Conference
on Environmental Systems, AIAA 2013-3369, 14-18 July 2013.

12. Colizzi, E., Thermal Balance Testing: A Rigorous Theoretical Approach to Stabilisation Criteria Based on
Operative Re-Definition of Thermal Time Constant, AIAA 2012-3405, 42nd International Conference on
Environmental Systems, 15-19 July 2012.

13. Rickman, S. L. and Ungar, E. K., A Physics-Based Temperature Stabilization Criterion for Thermal Testing, 25th
Aerospace Testing Seminar, October 2009.

14. Novak, K. S., Kempennaar, J. E., Liu, Y., Bhandari, P. and Dudik, B. A., Mars Science Laboratory Rover System
Thermal Test, AIAA 2012-3516, 42nd International Conference on Environmental Systems, July 2012.

15. Welch, J. W., Comparison of Recent Satellite Flight Temperatures with Thermal Model Predictions, SAE ICES
2006-01-2278, Proceedings of the 36th International Conference on Environmental Systems, July 2006.

16. Wright, C. P., Test Effectiveness of SMC-S-016 Unit Thermal Acceptance Testing, 26th Aerospace Testing
Seminar, March 2011.

13