Modis LP Qa Tutorial-1b
Modis LP Qa Tutorial-1b
Part-2 of this document (in the near future) will provide a detailed delineation and examples
of the pixel-level QA structure in three MODIS land product suites: land surface reflectance,
vegetation indices, and BRDF and albedo.
Section-1: MODIS land products and their QA information
The MODIS Adaptive Processing System (MODAPS) facility at the Goddard Spaceflight
Center (GSFC) in Greenbelt, MD routinely produces MODIS land products from data
derived from twin MODIS instruments aboard the Terra and Aqua platforms that were
launched in December 1999 and May 2002 respectively. These data are archived at and
distributed from the Land Processes Distributed Active Archive Center (LP DAAC) at the
USGS EROS Center in Sioux Falls, SD. The MODIS Land Science Team (MODLAND)
is responsible for the MODIS land products in terms of their QA and validation. They help
evaluate and document the science quality of the products that are intended to constantly
inform the user community. The Land Data Operational Product Evaluation (LDOPE)
facility, collocated with MODAPS at GSFC, is responsible for the overall coordination of
the QA activities in support of the MODIS Science Team. A fairly complex and laborious
process, this includes the evaluation and documentation of the science quality of all MODIS
land products, which is finally incorporated in the operational production code and carried
within the products (at the pixel-level) and their metadata (at the file-level).
The MODIS land collections comprise over sixty-five products that include dailies to n-day
composites. Over the 14+ years since Terras launch, MODIS data at the LP DAAC
comprise over 31 million granules and ~650 Terabytes in volume (as of April 4, 2014).
Over this time, the NASA data discovery interfaces to search and procure data have also
continued to evolve, and the current discovery interface called Reverb is the third
incarnation, which replaces the Warehouse Inventory Search Tool (WIST). A number of
other MODIS data search, access, and procurement methods exist as well.
Given the large number of MODIS land products (they include 16 daily, 1 four-day, 29
eight-day, 8 sixteen-day, 7 monthly, and 5 yearly products), the dependencies that exist
between them, and the differences in the QA procedures that are applied to them, it is
difficult to provide a generic description and approach that applies to all. Within the userdata interaction process, MODIS QA-related information has the potential to manifest itself
at different times and locations. One of the primary goals of this document is to direct users
to the best combination of QA information sources, and methods to tap them to help drive
the data requirements for their research and applications.
NASAs Earth Observing System (EOS) manages one of the largest science data production
and applications enterprises in the world, of which MODIS datasets comprise a leading
component. QA has always been identified as very essential to the success of the real-world
applications that MODIS datasets help support but its complexity has discouraged
widespread use. The mechanisms to generate, publish, access, communicate, and interpret
QA for diverse suites of MODIS products are very elaborate. This document is intended to
expose users to basic information that successfully helps initiate their interaction with the
QA layers of MODIS land products.
Section-2: Why is it important for users to consult the QA information?
MODIS QA information provides vital clues regarding the usability and usefulness of the
data products for any particular science application. Usability is the capability of being used
for a particular purpose while usefulness refers to what extent something serves a purpose
towards meeting a practical objective. Usability and usefulness address any of the following
requirements that MODIS QA information provide, which are not mutually exclusive:
What proportion of data artifacts and anomalies present in the data are deemed to
exist within a satisfactory threshold to proceed with a particular science application?
Are there mitigating conditions under which we can rule certain science data layers
within a product as more or less useful than others?
Are the science data layers derived using the main algorithm deemed satisfactory
compared to the back-up algorithm or vice-versa?
Parts 2 through 4 of this document (that are available separately), provide details regarding
the QA structure and implementation within MODIS Land Surface Reflectance, Vegetation
Indices, and Bidirectional Reflectance Distribution Function and Albedo product suites.
Please note that the examples provided in these product suites demonstrate how the QA
characterization propagates to the higher-level products, and underscores the need to
understand the data fidelity at the very beginning stage when users contemplate use of a
particular MODIS data product.
Given the fact that any particular MODIS land product is the result of a fairly complex
process that involves a science algorithm, inputs that include MODIS level-1B data,
ancillary data, lookup tables, auxiliary inputs, and possibly, other derived MODIS data
products, users run a serious risk by not consulting the QA information. Some of the
known sources of error that impact data quality include data loss due to instrument contact
errors, striping in the land surface reflectance data, geolocation errors traceable to
instrument maneuvers, effects of solar eclipse on the data, and problems stemming from the
cloud mask, especially as a function of latitude. MODIS land data product users are
therefore strongly encouraged to consult the QA information before they decide to use their
data.
their data inputs remain consistently of reliable quality. Two kinds of pixel-level metadata
implementations exist in MODIS land products.
1. The first includes a QA SDS that contains multiple information sources
accomplished through binary encoding.
2. The second involves a QA SDS that contains a single information source, such as
pixel reliability in the Vegetation Indices products, or albedo quality in the
BRDF/Albedo products.
All MODIS land products contain one or more SDS devoted to QA among the multiple
HDF arrays. These SDSs are critical to understand, parse, and interpret pixel-level QA. As
users open the MODIS HDF dataset in any image processing software system, the one or
more QA-specific SDSs are identifiable through the inclusion of QA, QC, or Quality
in their name. Table 1 identifies the QA SDS arrays for each of the MODIS land products.
Radiation
Budget
Variables
Ecosystem Variables
Vegetation Indices
M*D09GA:
1 km Reflectance Data State QA
500 m Reflectance Band Quality
M*D09GQ:
250 m Reflectance Band Quality
M*D13A1:
500 m 16 days Pixel Reliability QA
500 m 16 days VI Quality
M*D13A2:
1 km 16 days Pixel Reliability QA
1 km 16 days VI Quality
M*D13A3:
1 km Monthly Pixel Reliability QA
1 km Monthly VI Quality
M*D13Q1:
250 m 16 days VI Quality
250 m 16 days Pixel Reliability QA
M*D13C1:
CMG 0.05 16 days Pixel Reliability QA
CMG 0.05 16 days VI Quality
M*D14:
Algorithm QA
M*D09A1:
500 m Reflectance Band Quality
500 m State Flags
M*D09Q1:
250 m Reflectance Band Quality
M*D09CMG:
Coarse Resolution QA
Coarse Resolution State QA
M*D13C3:
CMG 0.05 Monthly Pixel Reliability QA
CMG 0.05 Monthly VI Quality
M*D14A1:
QA
M*D14A2:
QA
Land
Cover
MCD12Q1:
Land Cover QC
MCD12Q2:
Dynamics_QC
MCD12C1:
Majority_Land_Cover_Type_1_QC
M*D17A2:
PSN_QC_1km
MOD44W:
Water Mask QA
M*D17A3:
NPP_QC_1km
Table 1: QA Science Data Sets within the HDF arrays for each MODIS land product suite
Pixel-level QA is generated by the production code to evaluate product quality at the finest
granularity. This information is useful to both users who need to decide the quality of
retrieval at any particular pixel, and for LDOPEs evaluation as well. LDOPE performs
detailed analyses before decisions regarding this assessment are integrated into the final
production code. Pixel-level QA provides information for each science parameter through
the following two methods:
Pixel-level QA code
Interpretation
00
01
10
11
Starting with Collection-5, some MODLAND products were implemented with a 1-bit
generic product assessment (Table 3) rather than a 2-bit summary (Table 2) in part to reflect
*
A bit (short for Binary Digit) is the smallest unit of information/memory in digital computing that can
represent two possible values, represented by 0 and 1.
**
The least-significant bit is the lowest bit in a series of numbers in binary notation, located at the far right of a
string; also referred to as the right-most bit.
algorithm evolution. Land surface reflectance, land surface temperature & emissivity, and
vegetation indices retained the 2-bit MODLAND QA as it continues to remain relevant to
those products. The second bit describes the condition in which a pixel was not produced
because of clouds or other effects. Table 3 describes the 1-bit codes implemented in
Collection-5.
Pixel-level QA code
Interpretation
(0, 1), while two bits represent four values, and three bits represent eight values, etc. as
depicted below.
Hence, the number of bits and the number of values (or quality attribute meanings) they
potentially represent, doubles with each step as shown below:
Number of Bits
1
2
3
4
5
6
7
8
Number of Values
2
4
8
16
32
64
128
256
Formula
21
22
23
24
25
26
27
28
The QA-specific SDS for each MODIS land product (listed in Table 1) generally breaks
down into four columns. The order of these columns as depicted under the Layers tab in
the product documentation on the LP DAAC Web page is described below. This order and
the terminology used may vary slightly in the File Specifications and/or the User Guides,
which are referenced in Section-5:
The third column identifies the bit values for each parameter
Users need to convert the pixel-level QA values from decimal to binary before they can
parse and interpret them. Several online converters are available to make such a
conversion, or users may convert the desired QA bits to their binary format using
LDOPEs unpack_sds_bits utility (further details regarding LDOPE tools are available in
Section-4). Once you have converted a pixel-level value from decimal to binary, you need
to correctly parse the bits before you can interpret them.
The following four examples demonstrate how you handle a particular pixel-level QA value
from four different products.
Example-1 (MOD09GQ: Terra/MODIS Surface Reflectance Daily L2G Global 250m)
A single pixels value of 7425 is derived from a 250 meters surface reflectance
(MOD09GQ) products QC_250m_1 SDS parameter. The 7425 decimal value converts
to a binary value of 1110100000001. (Users should check for the datatype, which provides
the number of bits in the QA word. The conversion should contain that many number of
binary positions as defined in the number of bits). We need to add three zeros to the mostsignificant bit to complete the 16-bit string (i.e., to the left). This value, as assigned to the
individual bit numbers, breaks down thus (based on the QA index specified in the
MOD09GQ products file specifications):
Users are reminded that all HDF-EOS products are written in the big-endian referencing scheme. The bits
are always numbered from right (least-significant bit) to left (most-significant bit).
10
Please bear in mind that the binary bit-string is parsed from right to left, and the individual
bits within a bit-word are read from left to right; the above string breaks down thus:
Bit Word
01
00
0000
1101
1
0
00
Description
Less than ideal quality some or all bands
Clear (Cloud state)
Highest quality (Band-1 data quality)
Correction out of bounds pixel constrained to extreme allowable value
Yes (Atmospheric correction performed)
No (Adjacency correction performed)
Spare (unused)
Clarification regarding cloud information source: Some confusion persists regarding the source
of cloud information that users should consider as part of their screening process for their
particular applications. Users should tap the State QA SDSs for cloud-specific
information. The reflectance band quality SDSs in the M*D09GQ and M*D09Q1
products carry a parameter called Cloud State that has not been populated since the V3
MODIS collection, and therefore not a reliable source of information. The MODLAND
QA bits (discussed earlier in this tutorial) are not a viable source of cloud status information
for a particular reason: In the case of the LSR products, these bits imply whether a particular
pixel was not processed due to cloud effects, but, since the V3 MODIS collection, the
algorithm does perform atmospheric correction over clouds. Given these idiosyncrasies,
please ensure that you consult the 1 km State QA SDSs for cloud-specific information in
either the daily (M*D09GA) or 8-day products (M*D09A1). Please remember that you
can also use this information to extrapolate to apply to other LSR products with
different spatial resolutions.
11
12
Bit Word
00
0000
0000
0000
0000
0111
0000
0000
1
0
Description
Correct product produced at ideal quality for all bands
Highest quality (Band-1 data quality)
Highest quality (Band-2 data quality)
Highest quality (Band-3 data quality)
Highest quality (Band-4 data quality)
Noisy detector (Band-5 data quality)
Highest quality (Band-6 data quality)
Highest quality (Band-7 data quality)
Yes (Atmospheric correction performed)
No (Adjacency correction performed)
13
The individual bit-words, delineated above, are parsed again to derive their decimal values.
The final interpretation yields the following information:
QA bit value
0011 = 3
0011 = 3
0011 = 3
0011 = 3
0011 = 3
0011 = 3
0100 = 4
000
0
Description
Magnitude inversion (numobs >= 3 & < 7) (Band-1 data quality)
Magnitude inversion (numobs >= 3 & < 7) (Band-2 data quality)
Magnitude inversion (numobs >= 3 & < 7) (Band-3 data quality)
Magnitude inversion (numobs >= 3 & < 7) (Band-4 data quality)
Magnitude inversion (numobs >= 3 & < 7) (Band-5 data quality)
Magnitude inversion (numobs >= 3 & < 7) (Band-6 data quality)
Fill value (Band-7 data quality)
Unassigned TBD
Not fill-value (QAFill)
011 | 01 | 0 | 1 | 1
The above bit-words are interpreted to reveal the following:
QA bit value
1
1
0
01
011
Description
Other quality (Back-up algorithm or fill-values) (MODLAND QA)
Aqua (Sensor)
Detectors apparently fine for up to 50% of channels 1, 2
Significant clouds were present (Cloud State)
Main algorithm failed due to problems other than geometry,
empirical algorithm used (Science Computing Facility QC)
14
executed either from the command-line or invoked via scripts. One of the most commonly
used routines (referred in the earlier section) is unpack_sds_bits which helps parse and
interpret the bit-packed QA attributes.
Interested users may register and download these LDOPE QA tools and related
documentation from the following site:
https://lpdaac.usgs.gov/lpdaac/tools/ldope_tools
Section-5: Links to QA-specific online information sources
This section provides links to all QA-related online information that users should find
useful.
LDOPE Web site: http://landweb.nascom.nasa.gov/cgi-bin/QA_WWW/newPage.cgi
LDOPE maintains the MODIS Land Quality Assessment Web site that provides a
plethora of QA-specific as well as other information. Under the Quality drop-down
menu, there are Terra C5, Aqua C5, and Terra+Aqua C5 hyperlinks that lead to
product-specific QA documentation. This includes Science Quality Flag values, their
explanation, and related comments for each products discrete acquisition time ranges.
Hyperlinks also exist for Terra Known Issues, Aqua Known Issues, and Terra+Aqua
Known Issues. They provide detailed descriptions of past problems and current issues
under investigation for each MODIS land product.
The Docs drop-down menu contains hyperlinks to the following resources for each
product suite:
User Guides
List of the Earth Science Data Types (ESDT) and Science Data Sets
A number of other links germane to MODIS land data quality assessment exists as well.
LP DAAC Web site: https://lpdaac.usgs.gov/products/modis_products_table
15
The above link to the LP DAACs Web site provides a table of all MODIS land products,
each of which leads the user to product-specific documentation. QA-specific attributes and
description information are provided under the Layers tab. This information is extracted
from the File Specifications for each product. The Links tab provides links to the
product-specific User Guide, ATBD, and the MODIS Validation Web site.
16
References
Roy, D.P., Borak, J.S., Devadiga, S., Wolfe, R.E., Zheng, M., Descloitres, J. (2002) The
MODIS Land Quality Assessment Approach. Remote Sensing of Environment, 83: 6276.
17