1/31/03

Appendices

FY 2002 Performance and Accountability Report

U.S. Department of Education
 1/31/03

Appendix A
ED Fiscal Year 2002 Percentage of Funding by Goal
(dollars in millions)

Other -- $81.9
0.1%
Goal 1 -- $2,797.1
Goal 6 -- $852.3
5.0%
1.5%

Goal 5 -- $22,660.2
40.5%

Goal 2 -- $28,104.0
50.2%

Goal 4 -- $433.8
0.8%
Goal 3 -- $1,050.1
1.9%

Goal 1: Create a culture of achievement

Goal 2: Improve student achievement
Goal 3: Develop safe schools and strong character
Goal 4: Transform education into an evidence-based field
Goal 5: Enhance the quality of and access to postsecondary
and adult education
Goal 6: Establish management excellence
Other: Civil rights activities

Source: Internal Department of Education estimates provided by Budget Service.

FY 2002 Performance and Accountability Report A-1

U.S. Department of Education
Appendix A: Department of Education Fiscal Year 2002 Appropriations
and Staffing Allocated by Goal and Objective
Total Program S&E Staffing
($ in millions) ($ in millions) ($ in millions) (FTE)
$55,979.4 $54,781.1 $1,198.4 4,710
Goal 1: Create a culture of achievement 2,797.1 2,738.7 58.4 358
Objective 1.1--Link Federal education funding to accountability for results 1,204.6 1,168.7 35.9 250
Objective 1.2--Increase flexibility and local control 67.4 60.0 7.4 54
Objective 1.3--Increase information and options for parents 874.9 859.8 15.1 54
Objective 1.4--Encourage the use of scientifically based methods within
Federal education programs 650.3 650.3 0.0 0
Goal 2: Improve student achievement 28,104.0 28,033.1 70.9 508
Improve student achievement 1,606.8 1,606.8 0.0 0
Objective 2.1--Ensure that all students read on grade level by the 3rd grade 9,898.9 9,873.5 25.4 179
Objective 2.2--Improve mathematics & science achievement for all students 6,897.0 6,879.8 17.1 125
Objective 2.3--Improve the performance of all high school students 4,204.6 4,183.1 21.5 154
Objective 2.4--Improve teacher and principal quality 5,496.7 5,489.9 6.8 50
Goal 3: Develop safe schools and strong character 1,050.1 1,039.1 10.9 79
Develop safe schools and strong character 15.5 15.5 0.0
Objective 3.1--Ensure that our Nation's schools are safe and drug free and
that students are free of alcohol, tobacco and other drugs 933.7 923.5 10.2 74
Objective 3.2--Promote strong character and citizenship among our
Nation's youth 100.8 100.1 0.7 5
Goal 4: Transform education into an evidence-based field 433.8 372.5 61.3 413
Objective 4.1--Raise the quality of research funded or conducted by
the Department 256.2 194.9 61.3 413
Objective 4.2--Increase the relevance of our research in order to meet
the needs of our customers 177.6 177.6 0.0 0
Goal 5: Enhance the quality of and access to
postsecondary and adult education 22,660.2 22,569.4 90.7 595
Enhance the quality of & access to postsecondary & adult educ. 169.8 169.8 0.0 0
Objective 5.1--Reduce the gaps in college access and completion among
student populations differing by race/ethnicity,
socioeconomic status, and disability while increasing the
educational attainment of all 2,063.1 2,040.8 22.3 158
Objective 5.2--Strengthen accountability of postsecondary institutions 98.8 77.6 21.2 134
Objective 5.3--Establish effective funding mechanisms for
postsecondary education 16,249.6 16,249.6 0.0 0
Objective 5.4--Strengthen HBCU, Hispanic serving institutions, and
tribal colleges and universities 534.5 532.5 2.0 14
Objective 5.5--Enhance the literacy & employment skills of American adults 3,544.4 3,499.1 45.3 289
Goal 6: Establish management excellence 852.3 23.8 828.6 2,039
Objective 6.1--Develop and maintain financial integrity and management
and internal controls 101.6 5.5 96.1 463
Objective 6.2--Improve strategic management of ED's human capital 175.0 0.0 175.0 211
Objective 6.3--Manage IT resources, using e-gov, to improve service
customers and partners 99.8 18.3 81.5 125
Objective 6.4--Modernize the SFA programs & reduce their high-risk status 466.5 0.0 466.5 1,190
Objectve 6.5--Achieve budget and performance integration to link funding
decisions to results 9.5 0.0 9.5 50
Objective 6.6--Leverage the contributions of community- and faith-based
organizations to increase the effectiveness of ED programs 0.0 0.0 0.0 0
Objective 6.7--By becoming a high performance, customer-focused
organization, earn the President's Quality Award 0.0 0.0 0.0 0
Other 81.9 4.4 77.5 718
Note--All funds under "Other" are attributable to civil rights activities.
Source: Internal Department of Education estimates provided by Budget Service.

A-2 FY 2002 Performance and Accountability Report

U.S. Department of Education
 Department of Education
Percentage of FY 2002 Appropriations Allocated by Goal or Objective

FY 2002 Performance and Accountability Report

U.S. Department of Education
1/31/03

Department of Education
Percentage of FY 2002 Appropriations Allocated by Goal or Objectives

FY 2002 Performance and Accountability Report

U.S. Department of Education
 Department of Education
Percentage of FY 2002 Appropriations Allocated by Goal or Objectives

FY 2002 Performance and Accountability Report

U.S. Department of Education
 Appendix B: President’s Management Agenda—Scorecard for Department of Education

Current Status Progress in Implementing the Comments

Appendix B
(As of September 30, 2002) President’s Management Agenda
Initiative ED’s human capital strategy (One-ED) Actions Taken Since September 30, 2001 The One-ED review process provides a
provides a framework for increasing the snap-shot (or baseline) of ED operations,
ED has begun implementation of the
efficiency of significant business work flows, and human resource
One-ED plan. ED has:
functions, improving management of allocations. When the baseline is complete,
Human Capital human resources, and optimizing the • issued a blanket purchase agreement ED will be able to set out a vision (i.e.
organizational structure to meet to obtain contractor assistance with performance targets and goals) for what the
Red Departmental goals and objectives. Green Phase 1 tasks re-engineered, delayered, and citizen-
Agency Lead: centered ED will look like.
As implementation efforts for One-ED • selected a contractor and begun work
are in the very early stages, ED does not on applying the One-ED model to The Department has established an
yet have in place a mature system for ED’s HR processes ambitious schedule to review every major
Bill Leidinger identifying and addressing human capital business function by 3/05. Prior to that
deficiencies. Specifically, the One-ED • convened a steering committee to time, however, One-ED will not have
plan needs to ensure that: manage/oversee implementation quantifiable goals, making it difficult to link
• mission critical skills • continued communication efforts the process to future budget requirements.
gaps/deficiencies are identified and with the Executive Management That said, the One-ED process puts ED in
addressed Team, Senior Executives, program the very favorable position of having an
offices accurate baseline for assessing human
• employee performance is tied to capital needs, including a global view of in-
strategic goals • begun to develop competencies for
mission critical business lines house skill deficiencies and intra and inter-
• human capital decisions are guided agency business function comparisons.
by a data driven, results-oriented • begun development of agency-wide
system linking employee Critical next steps include:
accountability system
performance to progress on strategic • continue implementation for HR and
“Current Status” upgrade possible in 1Q planning goals begin review of legal, audit, and
FY 2003 if implementation efforts student financial assistance policy
proceed on schedule and ED begins to • completed Union negotiations on
revised performance appraisal system processes.
show tangible evidence that the One-ED
process is leading to significant human and developed implementation • complete appraisal system reforms
capital improvements. schedule
• incorporate quantifiable performance
Planned Actions for Q1 FY 2003 measures into the One-ED process
During the first quarter of the upcoming
fiscal year, ED plans to continue re-
engineering analysis of HR, and expand
analyses to other ED processes, including
legal review, payment processing, audit
review, and student financial assistance
policy.
B-1

Definitions of Progress Evaluation

Green: Implementation is proceeding according to plans agreed upon with the agencies.
Yellow: Slippage in implementation schedule, quality of deliverables, or other issues requiring adjustments by agency in order to achieve initiative on a timely basis.
Red: Initiative in serious jeopardy. Unlikely to realize objectives absent significant management intervention.
 1/31/03
B-2

Current Status Progress in Implementing the Comments

Appendix B
(As of September 30, 2002) President’s Management Agenda
ED has developed its “One-ED” plan that Actions Taken Since September 30, 2001 • ED’s success in this initiative is
Competitive defines a process for simultaneously directly tied to its progress in
ED incorporated competitive sourcing
performing human capital restructuring implementing One-ED’s ambitious
Sourcing with competitive sourcing reviews.
into One-ED (described in Human
schedule. Any One-ED setbacks would
Capital), where ED will review personnel
Red Green delay progress in competitive sourcing.
deployment and competitive sourcing in
Agency Lead: At the current time, the plan is in the Still, One-ED, and ED’s willingness to
four phases, from June 2002 through
preliminary implementation stage. As a reexamine prior FTE/activity
March 2005. ED will conduct reviews of
Bill Leidinger result, ED has not achieved the short-term determinations in their FAIR Act
job-types across the Dept. rather than by
goal of competing 15 percent of the FAIR inventories, are both promising.
office.
Act inventory or the long-term goal of 50 • ED should ensure that routine
percent. interagency reimbursable activities are
In ED’s FY 2002 FAIR Act inventory,
commercial FTE increased from 1,826 in competed as part of One-ED.
ED and OMB have agreed in principle to Currently, it appears these activities
FY 2001 (37 percent of total FTE) to
the Department using a modified will be reviewed in phase IV,
3,062 (65 percent of total FTE). Per ED,
competitive sourcing process that relies beginning in Oct. 2004. ED should
this was due to One-ED, which redefined
on “Best Value” metrics in determining consider accelerating this review.
or eliminated previous activities, and
outsourcing. • One-ED phases I and II should provide
determined realigned FTE were no longer
inherently governmental. ED expects the basis for completing competitions
To maintain a green on progress in Q1 on 15 percent of its FAIR Act
further adjustments in FYs 2003–2004.
2003, ED must establish milestones and inventory by Sept. 2003. Current
show progress toward meeting the 15 status will likely be upgraded when this
ED completed One-ED briefings with all
percent goal by the end of FY 2003. happens.
affected Dept. organizations, and has
begun to assist employees with Critical next steps include:
developing competitive proposals. • Timely implementation One-ED,
beginning with the completion of phase
Planned Actions for Q1 FY 2003 I by Feb 2003.
• Continuing to assist employees with
Continue One-ED’s phase I, which developing competitive proposals.
focuses on human resources and other • Initiating competitions in order to meet
support services. Final phase I the 15 percent goal by Sept. 2003.
competitive sourcing decisions should be • Ensuring ED management coordinates
made by Feb. 2003. with key personnel to ensure success on
the “Best Value” approach for
competitive sourcing

Definitions of Progress Evaluation

Green: Implementation is proceeding according to plans agreed upon with the agencies.
Yellow: Slippage in implementation schedule, quality of deliverables, or other issues requiring adjustments by agency in order to achieve initiative on a timely basis.
Red: Initiative in serious jeopardy. Unlikely to realize objectives absent significant management intervention.
 Appendix B: President’s Management Agenda—Scorecard for Department of Education

Current Status Progress in Implementing the Comments

Appendix B
(As of September 30, 2002) President’s Management Agenda
ED received a qualified audit opinion on Actions Taken Since Sept. 30, 2001 In addition to the clean audit action plan,
Financial its most recent audited financial ED has made general improvements to
ED has implemented a detailed action
statements (FY 2001). The auditors financial management, including
Performance continued to cite weaknesses from prior
plan for achieving a clean FY 2002 audit
deployment of a new general ledger system
opinion. Specifically, ED has:
audits, including failures to reconcile Yellow and significant reductions in unreconciled
Red
Agency Lead: financial data, inadequate internal • Submitted 247 separate “provide by cash items.
controls, and lack of reliable opening client items” (which support
Jack Martin account balances. In addition, the statement balances and demonstrate Despite this progress, much work remains
auditors noted that the Department is not compliance with laws and to ensure a positive result for the FY 2002
substantially in compliance with the regulations) audit and beyond. The Department has not
Federal Financial Management • Provided auditors with interim resolved:
Improvement Act, particularly due to financial statements/trial balances for • problems with the data/template
information technology problems and period ending 6/30/02. between the FSA subledger and the
unsupported manual adjustments. • Accelerated the audit process by over General Ledger (continues to prevent
a month using mid-year credit data postings for FSA records, and has
ED’s current financial systems and subsidy estimates (which allow for a slowed critical reconciliation efforts)
processes do not support day-to-day more thorough audit of estimates) • need for increased coordination
operations. and by integrating trial balance and between FSA and ED accounting
financial statement relationship tests
“Current Status” upgrade possible 2Q FY into the Oracle Reporting module Critical next steps include:
2003, so long as: • Completed a mock year-end close • timely completion of FY 2002
• ED receives an unqualified audit financials
opinion in ‘02 Planned Actions for Q1 FY 2003 • implement a systemic process that will
ensure that accounting to support new
• Agency head provides unqualified • Successfully close FY 2002 in the feeder system feeds to and from the
assurance statement accounting system general ledger, or enhancements
• Successfully modify accounting thereto, are correct in all material
• There are no material noncompliance system to comply with FY 2003 SGL aspects
issues with laws and regulations account classifications and other
• There are no repeat material internal requirements
control issues • Prepare and support FY 2002
Financial Statements and deliver
• ED demonstrates use of accounting them to the auditors
data to support day-to- day
operations
B-3

Definitions of Progress Evaluation

Green: Implementation is proceeding according to plans agreed upon with the agencies.
Yellow: Slippage in implementation schedule, quality of deliverables, or other issues requiring adjustments by agency in order to achieve initiative on a timely basis.
Red: Initiative in serious jeopardy. Unlikely to realize objectives absent significant management intervention.
 1/31/03
B-4

Current Status Progress in Implementing the Comments

Appendix B
(As of September 30, 2002) President’s Management Agenda
The Department has improved its Actions Taken Since September 30, 2001 Initial analysis of 2004 Ex. 300’s show
E-Government investment review process by: progress in some areas and continued need
In addition to improvements to the for improvement in others.
• developing an agency-wide business investment review process, ED has:
Agency Lead: case template aligned with Ex. 300 Green While the 2004 business cases support ED’s
Red requirements • Begun implementation of
Performance-based Data strategic plan and business objectives, they
• expanding CPIC process to Management Initiative are not aligned with govt.-wide E-gov
Bill Leidinger incorporate all business units initiatives. Many systems do not have
• Linked management agenda to security plans and remediation actions, and
• establishing a dept.-wide IRB specific actions and milestones have not completed certification &
related to govt.-wide E-gov accreditation.
initiatives
ED has improved commitment to govt.- Although ED should continue to consolidate
wide E-gov initiatives: • Performed risk assessment of all EA and One-ED planning to provide an
major and significant systems; laying agency-wide perspective on how IT relates
• E-grants: detailing an FTE, assigning foundation to construct and
3 part-time FTE, and designating to business processes, One-ED timeframes
implement security plan for entire will delay completion of a consolidated EA.
CFO to exec. board Department Without a completed EA, ED is susceptible
• E-loans: solely funding Phase I and to redundant, incompatible investments.
the development of a joint business Planned Actions for Q1 FY 2003 For example, recent work between OMB
case; assigning 3 FTE and the CIO revealed redundant investments
• Complete e-Loans project plan and in web architecture to support various
• Funding contribution: $1.7 million in make final recommendations on the information collection efforts.
2002 (however, ED must work to parameters of the initiative.
finalize 2003 and 2004 funding • Continue implementation of system Critical next steps include:
commitments). risk assessments and associated • develop timeline for completed EA
To change status in Q1 2003, ED must: corrective action plans. • commit to 2003 and 2004 funding
• Provide OMB with a specific • complete security plans and
• complete security plans and timeline for completion of an remediation actions for all major
remediation actions for all major integrated EA systems (and prepare to complete
systems and demonstrate that
certification and accreditation)
certification and accreditation will
occur after end of Q1 • fully participate in the development and
deployment of E-gov initiatives
• commit to 2003 and 2004 funding
• demonstrate significant progress
toward the development of an
integrated EA

Definitions of Progress Evaluation

Green: Implementation is proceeding according to plans agreed upon with the agencies.
Yellow: Slippage in implementation schedule, quality of deliverables, or other issues requiring adjustments by agency in order to achieve initiative on a timely basis.
Red: Initiative in serious jeopardy. Unlikely to realize objectives absent significant management intervention.
 Appendix B: President’s Management Agenda—Scorecard for Department of Education

Current Status Progress in Implementing the Comments

Appendix B
(As of September 30, 2002) President’s Management Agenda
ED has not finalized a strategy for: Actions Taken Since September 30, 2001 ED worked with OMB to complete PARTs
for 21 programs (over 50 percent of disc.
Budget & • Integrating budget planning with budget). PARTs highlighted the lack of
ED’s FY 2004 OMB Budget request:
Performance strategic planning. outcome data in many programs.
Integration Green • Presents a breakdown of relevant
Red
• Obtaining performance data currently S&E costs attributed to each program NCLB established a linkage between
Agency Lead: lacking in many programs. and mission account. performance and Federal support for many
• Budget presentation aligned to el/sec education programs. Upcoming
• Charging full budgetary costs to strategic plan objectives and show reauthorizations should address data needs
William Hansen mission accounts and integrating resources for each goal. in other programs. Still, these efforts may
performance data into budget • Uses available performance data to not yield useful data for several years.
requests. guide budget decisions.
This lack of data remains a significant
ED continues to work on a multi-year ED reorganized the strategic planning and obstacle for integrating budget decisions
strategy to gather performance data. To budget staff under the Deputy Secretary. and performance. While ED’s budget
date, the Department: These staffs coordinate budget decisions, submission uses available outcome data, ED
strategic planning efforts, and track often could not link its budget to
• In Adult and Voc. ED, IDEA and progress on action steps. performance.
HEA reauthorizations, ED has
committed to increasing ED worked with OMB and other agencies ED plans to keep a centralized
accountability in programs by linking to develop Job Training Common administrative structure, while amending its
budget and performance through Measures (to be used for 10 ED budget planning process to include a
statutory changes. programs), and will first report disaggregation of admin costs for each
performance in the 2004 Budget. program, should provide needed flexibility
• Developed an evaluation plan for Planned actions for Q1 2003 while reflecting the full cost of programs.
programs over $100 million. • Prepare reauthorization proposals Critical next steps include:
and strategy for IDEA & OVAE to • Continue to develop measures and data
strengthen performance reporting. collection strategies for assessing
• Report on Job Training Common program performance, including
Measures, and address areas where student aid unit costs.
data are not available. • Refine S&E program allocations

Definitions of Progress Evaluation

B-5

Green: Implementation is proceeding according to plans agreed upon with the agencies.
Yellow: Slippage in implementation schedule, quality of deliverables, or other issues requiring adjustments by agency in order to achieve initiative on a timely basis.
Red: Initiative in serious jeopardy. Unlikely to realize objectives absent significant management intervention.
1/31/03

Program Current Status as of Progress in

Initiative September 30, 2002 Implementation
-------------------------------------------------------------------------------------
Elimination of Fraud
and Error in Student
Aid Programs and
Deficiencies in
Financial Management

Comments:

Despite ED’s continued progress on this initiative, student aid programs continue to be at high risk
for fraud and error. The Department has hired a new COO, CFO and FP manager for FSA this
quarter and the Secretary has made it clear to them that removal of the FSA programs from the High
Risk list is one of their top priorities. ED has made general improvements to its financial
management system, including deployment of a new general ledger system and significant
reductions in unreconciled cash items. However, the Department has not fully resolved ongoing
accounting problems, and the Department needs to implement a process to ensure that data and
accounting that support both FSA feeder system feeds to and from the general ledger and other
OCFO systems are correct in all material aspects. Problems in this area have delayed data postings
for several months, slowed critical reconciliation efforts, and is one issue that could influence a
positive result for the FY 2002 audit. The Department has made significant strides in the area of
default management and prevention, meeting its default collection targets for FY 2002 and
implementing performance-based contracts with private collection agencies that have yielded
increased collections for lower costs. However, Department management, the IG, and GAO all have
identified system weaknesses in ED’s school monitoring and student eligibility determination
process. The Department has taken several important actions to address these issues, including
acting on recommendations in GAO and IG reports, using data mining to help identify schools where
problems may exist and making systemic changes to match direct loan origination data with the
student eligibility system to ensure that borrower identifying information is accurate. On August 9,
2002, OMB and the Departments of Treasury and Education submitted a proposal for legislative
change that if implemented has the potential of eliminating hundreds of millions of dollars of
erroneous payments to students.

B-6 FY 2002 Performance and Accountability Report

U.S. Department of Education
 1/31/03

Appendix C: Findings From FY2002 Evaluations

1. State Title I Migrant Participation Information, 1998–99 School Year

States use Migrant Education Program (MEP) funds to ensure that migrant children are provided with
appropriate services that address the special needs caused by the effects of continual educational
disruption. MEP services are usually delivered by schools, districts, and/or other public or private
organizations and can be instructional (reading, mathematics, other language arts, etc.) or supporting
(social work, health, dental, etc.).
This report summarizes the participation information provided by state education agencies (SEAs) on
the MEP for the 1998–1999 school year, the 15th year that SEAs were required to submit information
using the State Performance Report.1 The report is organized into two sections: (1) an overall
descriptive summary of Title I MEP participation and staffing; and (2) individual state profiles.
Key findings included:
• Child Counts. In 1998–99, states reported 783,867 eligible students based on the 12-month
count of eligible students, ranging from 220,860 in California to 176 in Rhode Island.
States reported 311,914 summer term/intersession students, ranging from 121,788 in
California to 39 in Hawaii. MEP participation – that is, students actually receiving
program services, increased in both terms.
• Participation by Race/Ethnicity and Gender. The majority of migrant participants are
Hispanic (86 percent) compared to 17 percent of the general enrollment in the nation’s
public schools. Eight percent are white (not Hispanic), and less than 3 percent each are
black (not Hispanic), American Indian/Alaskan Native, and Asian/ Pacific Islander. The
percentage distribution of migrant participants among race/ethnicity categories remained
virtually unchanged between 1997–1998 and 1998–1999. The MEP serves slightly more
males (53 percent) than females (47 percent). Nationally, the school-aged population is 51
percent male and 49 percent female.
• Migrant Participants Receiving Special Services or Programs. Forty states reported
information on the number of migrant students receiving LEP services, and 37 reported on
the number receiving preschool services. Within the reporting states, 30 percent of migrant
students received LEP services and 9 percent preschool services.
• Participation by Grade. As noted above, regular term participation increased 9 percent
from 1997–1998 to 1998–1999. Forty-four percent of regular term participants were
served in the elementary grades (1 through 6), 30 percent in the secondary grades (7
through 12), and 20 percent in preschool. The remaining participants (6 percent) were
classified as ungraded or received services in an out-of-school setting. The distribution
among grade spans was about the same in 1997–98 and 1998–99.
• Participation by Service Area-Regular Term. In 1998–1999, in terms of instructional
services, the largest percentage of participants received reading services (31 percent)
followed by mathematics (22 percent), and English services for LEP students (15 percent).

1
In 1998–1999, State Performance Reports were received from 50 states, the District of Columbia, and Puerto Rico. Hawaii
implemented an MEP program during the 1998–1999 school year, but will not start serving participants until 1999–2000. The
District of Columbia and Puerto Rico are treated as states for the purpose of this analysis.

FY 2002 Performance and Accountability Report C-1

U.S. Department of Education
1/31/03

Less than 10 percent of regular term participants received MEP-funded instruction in

vocational/career, social studies, and science during the 1998–1999 school year. Thirty
percent of participants received undefined “other instructional services.” Information on
the types of other services (instructional and supporting) provided to migrant students is
provided in Appendix B. Over one-half of migrant participants received social
work/outreach services, making it the most common service—instructional or supporting—
provided to regular term participants. Eighteen percent of students received health-related
services, and 10 percent received transportation services. Fifty-nine percent of participants
received undefined “other support services” in 1998–1999—a figure that has more than
doubled since 1995–1996.
• Participation by Service Area-Summer Term. In the summer term, 61 percent of
participants received MEP-funded reading/language arts instruction; 42 percent received
mathematics instruction, and 18 percent received English services for LEP students. In the
supporting areas, the largest percentage of summer term participants (37 percent) received
social work/outreach services, followed by pupil transportation services (23 percent) and
health-related services (22 percent)
• Staffing. There was little change in the number of staff reported in both terms between
1997–1998 and 1998–1999. States funded a larger percentage of instructional staff (70
percent) in the summer term than in the regular term (54 percent). The ratio of participants
to instructional staff was much higher in the regular term (136:1) than in the summer term
(35:1).
• Projects and Project Sites. In 1998–1999, states operated 11,120 Title I MEP projects
across the nation—a 9 percent decrease from the previous year. Of the total number of
projects, 54 percent operated in the regular term only, 16 percent in the summer
term/intersession only, and 30 percent in multi-terms. Thirty-eight percent of the
schoolwide sites serving migrant participants blended migrant funds with regular Title I
funding. More than 163,000 migrant participants (or 24 percent of all migrant participants)
were enrolled in schoolwide programs that combined MEP funds with other forms of
federal assistance.

2. The Same High Standards for Migrant Students: Holding Title I Schools
Accountable
The three-volume study, The Same High Standards for Migrant Students: Holding Title I Schools
Accountable examines how the federal Migrant Education Program (MEP) is helping migrant
students succeed in school and meet academically challenging standards, and whether states and
districts are including migrant students in standards-based reforms.
National Longitudinal Study of Schools (NLSS) collected data over three school years from 1998–
1999 to 2000–2001. School-year 1998–1999 established a baseline of data on how these Title I
schools were implementing the accountability provisions of Title I of the ESEA, as amended, related
to high standards and assessments for all children. Data for Volume II were collected through
interviews with migrant program directors and data and assessment records specialists at state and
local levels during site visits between October 2000 and January 2001. Data for Volume III were
collected through case studies of district migrant education programs, chosen on the basis of
nominations from state directors.

C-2 FY 2002 Performance and Accountability Report

U.S. Department of Education
 1/31/03

The study found that principals and teachers in Title I schools serving migrant students have lower
expectations about how well their students can perform than teachers in other Title I schools. Fewer
seniors in Title I schools with migrant students have taken higher-level mathematics courses
compared with seniors in other schools. Title I schools with migrant students have higher
percentages of inexperienced teachers, teachers who are teaching in fields for which they are not
certified, as well as teachers who hold emergency or temporary certification. Many of the Title I
schools that served migrant students used different standards for their limited English proficient
students. A significant percentage of migrant students did participate in state or district assessments
in the 1997–1998 school year, but few of these schools received the results of the assessments
disaggregated by migrant status. Some states and school districts are implementing promising
practices to promote continuity of instructional services for migrant students to respond to the effects
caused by changing schools on students’ academic performance. Examples of other key findings
include:
• Expectations about student performance were low in Title I schools serving migrant
students. Some principals in Title I elementary schools with medium-to-high numbers of
migrant students reported that standards were too hard for a significant portion of their
students, and that a large percentage of these students were not prepared to do the work at
the next grade level. This finding is similar to National Longitudinal Survey of Schools
(NLSS) findings concerning differences between the highest-poverty and lower-poverty
Title I elementary schools.
• Many of the Title I schools that served migrant students used different standards for their
limited English proficient students. Many migrant students are also limited English
proficient. Over half of the Title I schools with medium-to-high numbers of migrant
students are classified as schools with 25 percent or more limited English proficient
students. Contrary to the requirements of the Title I program, principals in Title I
elementary schools reported that their schools used alternate state content standards and
different student performance standards for limited English proficient students.
• States’ knowledge of migrant student participation in assessments was weak. States
reported that they did not have a way to estimate the percentage of migrant students
participating in assessments because the number of students exempted from assessments is
typically a local decision that is often not reported to the state. Based on national survey
data, principals reported that 70 percent of migrant elementary students and 90 percent of
migrant secondary students participated in assessments in the 1998–1999 school year.
Mobility and language difficulties were the two leading barriers to migrant students’
participation in statewide assessments.
• Few schools with migrant students received disaggregated achievement scores. The single
greatest barrier to evaluating migrant student achievement data systems is the lack of
capacity that most state and local data systems currently have to link individual migrant
student records with state and district databases. Few states pursue data on graduation and
dropout rates because of the investment of resources involved, and because they are not
legislatively required to do so.
• Fewer students in Title I schools that serve migrant students were enrolled in higher-level
courses. Fewer seniors in Title I schools with medium-to-high numbers of migrant students
had taken higher-level mathematics courses compared with seniors in other Title I schools.

FY 2002 Performance and Accountability Report C-3

U.S. Department of Education
1/31/03

• Teachers in schools serving migrants were less experienced. Teachers in Title I elementary
schools with medium-to-high numbers of migrant students had fewer years of teaching
experience than teachers in other Title I elementary schools. These schools also reported
that more teachers were teaching in fields for which they were not certified and held
emergency or temporary certification. Many of these differences were similar to those
between highest- and lower-poverty schools.
• Title I schools with migrant students tend to be much poorer, and have high proportions of
students who are minorities and limited English proficient. Many of the differences
between Title I schools with no or few migrant students and schools with medium-to-high
numbers of migrant students parallel the differences between Title I schools that are
relatively lower in poverty and those that are highest-poverty.
• A few states and school districts are committed to aligning local instruction between
programs that share migrant students. In particular, these states and districts were
committed to aligning with the students’ home base schools for curricular content and
course requirements. Some examples of alignment policies included the following: LEP
students were placed in the same type of English acquisition program as their home base
school; trading partners compared their individual languages assessments scores to place
migrant students in the same types of course work; and trading partners agreed on common
grade placement policies.
• Technology is enabling states and districts to access other states’ and districts’ content and
performance standards. The use of technology provided solutions to the problems of
accessing information and providing instruction to difficult-to-reach students. Technology
was used to transfer information on students’ academic records between trading partners,
provide individualized instruction, and provide access to another state or district’s
assessments and standards.

3. State ESEA Title I Participation Information for 1999–2000: Summary Report

This report summarizes Title I information reported by states in their performance reports for school
year 1999–2000, as well as comparisons to 1998–1999 and previous years. It includes information on
both Part A (Grants to Local Educational Agencies Program) and Part D (the State Agency Program
for Neglected or Delinquent Children and Youth). Among the areas reported on for Part A are:
districts, schools, and students served, school improvement, services, and staffing. For the State
Neglected or Delinquent (N or D) program, information is provided on participation by institution
type and institution-wide programs. Key findings include:
Participation
• Almost 48,000 schools participated in the Title I program in 1999–2000. Of these, about
half reported operating schoolwide programs, a substantial 18 percent increase over 1998–
99. The remaining schools, numbering close to 25,000 reported operating targeted
assistance programs.
• The number of Title I participants increased to 14,855,155 in 1999–2000, a 10 percent
increase over the 1998–1999 level of 13,446,717. Participation rates by grade level stayed
the same, with the largest concentration of public and nonpublic school participants in the
elementary grades.

C-4 FY 2002 Performance and Accountability Report

U.S. Department of Education
 1/31/03

• Thirty-five percent of Title I participants were non-Hispanic whites, 27 percent were non-
Hispanic blacks, 31 percent were Hispanic, 4 percent were Asian/Pacific Islanders, 2
percent were Native American, and 1 percent were other. There were no significant
changes in these percentages since 1998–1999.
• In 1999–2000, students with disabilities represented 10 percent of participants; students
with limited English proficiency represented 17 percent; children of migratory workers
represented 2 percent; and homeless children represented 1 percent of participants. There
were no significant changes in these percentages since 1998–1999.
School Improvement
• In the 1999–2000 school year, 8,505 schools (19 percent of the total) were identified for
improvement, a slight decrease from the 20 percent identified in 1998–1999. Twenty-one
percent of schoolwide program schools were identified (a decrease from the 27 percent
identified in 1998–1999) and 16 percent of targeted assistance schools were identified,
compared to 15 percent identified in 1998–1999.
• Seventy-six percent of Title I schools met state criteria for adequate yearly progress (AYP),
an increase over the 74 percent reported by states in 1998–1999. The lowest poverty
schools (0-34 percent free and reduced price lunch) had higher percentages meeting AYP
criteria (83 percent), compared with the highest poverty (75-100 percent poverty) schools,
which had 71 percent meeting AYP criteria.
Services
• Eighty-nine percent of the Title I students in targeted assistance schools received Title I-
supported reading/language arts instruction, the same percentage as in 1998–1999. Fifty-
four percent received Title I-supported mathematics instruction, an increase from 43
percent in 1998–1999.
• Forty-four states and the District of Columbia reported that 2,628 districts (22 percent of
the total) provided Title I-supported family literacy services, an increase over the 15
percent reported in 1998–1999.
• Forty-seven states, the District of Columbia, and Puerto Rico reported that their Title I
schools operated 15,865 Title I extended time instructional programs, a substantial increase
of 59 percent since 1998–1999, but a number in line with the 15,736 extended time
instructional programs reported in 1997–1998.
Staffing
• Teachers and teacher aides constituted 85 percent of the total staff funded by Title I. Close
to half of Title I-supported staff (44 percent) were teachers, compared with 49 percent
reported in 1998–1999. Teacher aides represented 40 percent of staff in 1999–2000, a
small increase over the 37 percent reported for 1997–1998.
• Additional Title I-supported staff included: support staff (7 percent), administrators (3
percent) and other staff (6 percent).
State Neglected or Delinquent Program
• In 1999–2000, 10 percent of State Agency N or D participants were in facilities for
neglected children, a decrease from the 30 percent reported in 1998–1999; 66 percent were

FY 2002 Performance and Accountability Report C-5

U.S. Department of Education
1/31/03

in facilities for delinquent children, up from the 48 percent reported in 1998–99 and 24
percent were in adult correctional facilities, close to the 22 percent reported in 1998–1999.

4. Education for Homeless Children and Youth Program: Learning to Succeed

The Education for Homeless Children and Youth Program: Learning to Succeed is a two-volume
report on two studies that were conducted by Policy Studies Associates (PSA) under contracts with
the Planning and Evaluation Service of the U.S. Department of Education. Volume I, Reducing
Barriers for Homeless Children and Youth for Access and Achievement, reports on state and local
efforts to serve the educational needs of homeless children and youth and overcome barriers that
affect these students’ enrollment, attendance, and school success. Volume II, Educating Homeless
Children and Youth: A Resource Guide to Promising Practices, describes promising strategies and
practices that states, districts and schools can use to enroll homeless children and youth in school, and
to help them to achieve the same high standards expected of all children.
The studies found that homeless students are best served when promising practices are implemented
as part of a comprehensive and coordinated homeless education program. States have made the most
progress in eliminating immunization requirements as barriers to enrollment, but transportation,
guardianship, and frequent moves from school to school are still significant barriers to educating
homeless students. Regardless of whether or not they are awarded federal funds under the
McKinney-Vento Act, districts are finding ways to address at least some needs of their homeless
children and youth. However, districts with McKinney subgrants were able to provide a broader
range of educational and recreational services. Districts with no McKinney dollars relied on funds
from Title I and community organizations to support their efforts. Key findings include:
Homeless children continued to face a number of major barriers to enrolling and succeeding in
school. However, some states and school districts have come up with ways to overcome these
barriers.
• Districts with McKinney subgrants were able to provide a broader range of educational and
recreational services. Districts with no McKinney dollars relied on funds from Title I and
community organizations to support their efforts. Most districts that did not have
McKinney subgrants reported that they received no funds from outside sources to serve
homeless students. This was particularly true in smaller or more rural districts where
homelessness was not commonly recognized as a problem.
• Homeless students’ frequent moves from school to school were their most significant
barrier to academic success. Students who were unable to find stable shelter had difficulty
meeting state or district mandates regarding the number of days they must attend school to
stay enrolled. Some state coordinators indicated that the slow transfer of student records,
along with differing course requirements from school to school, complicated the accrual of
sufficient credits for homeless students to be promoted and receive a high school diploma.
However, many districts have developed or revised policies to ensure that homeless
students can enroll in school immediately. Some districts allow a grace period to track
down records or obtain oral confirmation that a child had attended another school or been
immunized.
• Transportation to and from school remained a major barrier to homeless children enrolling
and remaining in school. States and districts often have limited resources to address

C-6 FY 2002 Performance and Accountability Report

U.S. Department of Education
 1/31/03

transportation needs, particularly across district lines. However, some States and school
districts have found ways to meet the transportation needs of homeless children.
• Guardianship and immunization requirements were still significant barriers. Although
important for safety and health, these requirements were often at variance with efforts to
ensure that homeless children had access to school. However, some states have revised
their laws, regulations, and policies to remove obstacles to the education of homeless
children and youth.
• Lack of awareness and insensitivity to homeless students’ needs among school staff was
another barrier that could delay or prevent homeless children and youth from enrolling and
succeeding in school. However, many states are combating this lack of awareness and
sensitivity.
• Homeless students also had difficulty gaining access to special education, Head Start or
other publicly funded preschool programs, gifted and talented programs, Even Start or
other family literacy programs, and programs for students who are not proficient in
English. In many cases, this is because educators are unfamiliar with the requirements of
the law. However, some school districts went the extra mile to help homeless children gain
access to these programs.
• State coordinators collaborated and coordinated their work most often with local education
agencies, state government agencies, other offices within their state agency, and homeless
shelters to maximize funding and services.
• Many districts with McKinney subgrants sent information to schools about how to
recognize and meet the needs of homeless children and youth, and a few provided school
staff with professional development and training.
• Most districts with McKinney subgrants had set up regular lines of communication between
school officials and shelter providers, other service providers, and other educators to
identify and place homeless children in school.

5. Study on School Violence and Prevention

The Study on School Violence and Prevention examined the extent of problem behavior in schools
nationally and aspects of violence prevention programming, including activities supported by the Safe
and Drug-Free Schools (SDFS) program, a formula grant program. SDFS supports school-based
education to promote safe and orderly learning environments and prevent drug use and violence.
There are three reports presenting findings from the study:
• Wide Scope, Questionable Quality reports findings from surveys of a national sample of
elementary, middle, and high schools, including surveys of principals and activity
providers, and, in the middle and high schools, of teachers and students, along with district
SDFS program coordinators.
• A Closer Look presents case studies of 40 of the schools (20 middle schools and 20 high
schools) that were surveyed.
• School Crime Patterns provides the results of a new analysis of data from a previous
National Center for Education Statistics survey that asked principals about the number and

FY 2002 Performance and Accountability Report C-7

U.S. Department of Education
1/31/03

types of crimes they report to police. The new analysis focused on high schools to profile
schools with high and low levels of reported crime.
Results from the Study on School Violence and Prevention indicate that, while schools nationally
experienced relatively low levels of serious violent crime, some schools did experience serious
violence and disorder, and in many schools high levels of less serious violent crime and property
crime were common, particularly in middle schools. Although schools implemented prevention
programs, the quality of those activities was found wanting. Key findings are:

Disorder in Schools
• The vast majority of schools visited have relatively low levels of serious crime (e.g.,
aggravated assault, weapons violations, robbery). While fighting did occur and the
presence of weapons was not unheard of, the combination of the two was rarely seen in the
same school. Theft was much more common than robbery (i.e., stealing by force or threat
of force), and while teachers may have been verbally abused, they very rarely were
attacked or threatened with a weapon.
• Fear of disorder did not seem to interfere with the learning process. Site visitors at roughly
two-thirds of the schools described their schools as safe or very safe, with low or very low
levels of disorder.
Discipline Practices and Policies
• There was considerable overlap in discipline procedures across schools. Rules were
generally guided by the school district and involved varying levels of punishment as
offenses became more serious. Very few rewards were structured into the procedures.
• Schools seemed to have few problems communicating rules to all students. However, there
was some evidence that rules are inconsistently enforced across students (how common
inconsistent enforcement was across minor and serious infractions was unclear). Many
inconsistencies may stem from letting teachers handle many infractions within the
classroom.
Disciplinary Information
• Schools varied in how they recorded and used incident data. Collection forms vary widely
and information systems rarely include tracking of victims and offenders, although serious
incidents are usually reported to the district and state.
• Few schools had specific procedures in place to review incident data, and very few seemed
to follow guidelines recently recommended by a National Center for Education Statistics
task force on collecting and compiling incident data. Neither the level of detail collected
on particular incidents nor the unit of collection (incident, victim, and offender) seemed to
be in place in most schools to support the unit-based collection system recommended.
School Climate
• School staff expressed more satisfaction if the principal set clear expectations,
communicated regularly with staff, and gave them a voice in management issues.
• Schools had two basic approaches to rules and procedures. In one, teachers deal with all
but the most serious discipline problems. In the second approach, teachers deal with small
infractions and then refer students to an administrator for repeated or serious infractions.

C-8 FY 2002 Performance and Accountability Report

U.S. Department of Education
 1/31/03

Distinguishing Middle Schools with Different Levels of Disorder

• Low disorder schools were characterized by strong principals, school staff viewing
themselves as working as a team, active involvement of teachers in maintaining order, and
generally positive staff/student relationships. High disorder schools lacked a clear
approach to discipline, did not convey expectations to students well, and demonstrated poor
communication between teachers and administrators.
Key Findings of School Crime Patterns
• Violence (in terms of crimes reported by high schools to police) is clustered within a
relatively small percentage of locations, with about 60 percent of the violence occurring in
4 percent of the schools. This is about four times higher than would be expected based on
national rates of crime.
• The characteristics, such as size, location, and socio-economic make-up, of high-violence
schools differ markedly from the other schools. High schools with the highest levels of
violence tended to be located in urban areas and to have a high percentage of minority
students, compared to high schools that reported no crime to police. They also tended to be
located in areas with high levels of social disadvantage (such as unemployment) and
residential mobility. However, a relatively large minority (36 percent) of schools with high
levels of violent crime were located in rural areas.
• The types of violence prevention programs differed between high schools with different
levels of crime. The schools with a high level of serious violence also reported high use of
prevention measures and programs that were specifically aimed at controlling violence.
Schools in this group also appeared to put more emphasis on programs geared toward
changing individual behavior, compared to other schools, which tended to place a higher
priority on prevention instruction.
• High schools that experienced high levels of violent crime were more likely than other
schools to adopt a variety of security measures, particularly the random use of metal
detectors, used by about one-third of these schools compared to 10 percent or less of other
schools. Schools with high levels of violent crime were also more likely to use law
enforcement and security personnel as a measure to control disorder than schools with
lower levels of crime.

Key Findings of Wide Scope, Questionable Quality

Problem Behavior in Schools
• According to principals surveyed about the number of crimes they reported to police, 66
percent of schools experienced one or more incidents of less serious violent crime or
property crime (e.g., theft) and 10 percent experienced at least one serious violent crime
(e.g., fighting with a weapon). Compared with elementary and high schools, middle
schools had higher levels of many types of problem behavior.
• About 18 percent of students surveyed in middle and high schools were threatened with a
beating, and 13 percent were attacked without a weapon. About 11 percent of students
experienced at least one incident of robbery or being threatened with a weapon. About 62
percent of teachers experienced one or more incidents of being threatened in remarks by a
student, receiving obscene remarks or gestures from a student, damage to property, or theft

FY 2002 Performance and Accountability Report C-9

U.S. Department of Education
1/31/03

(40 percent received obscene remarks or gestures). Serious crime aimed at teachers was
relatively rare: only 3 percent were attacked and received minor injuries, while fewer (1
percent) were confronted with weapons or attacked and received injuries requiring a doctor.

6. Summary: First Final Report of the Longitudinal Study of the Vocational

Rehabilitation Services(VR) Program: How Consumer Characteristics Affect Access
to, Receipt of, and Outcomes of VR Services
The Longitudinal Study has followed 8,500 VR consumers at 37 locations for at least three years for
the purpose of identifying the types of individuals served, types of services provided, environments in
which the services were provided and short-term and long-term economic and non-economic
outcomes of the VR program
The study’s sample acquisition and data collection activities began in January 1995 and were
completed in January 2000, with sample acquisition occurring over a two-year period and each of the
study’s 8,500 participants tracked for three years.
1. Specific study questions that this report addresses are as follows:
2. What characteristics of individuals with disabilities affect their access to and receipt of VR
services and outcomes of those services?
3. What are the characteristics of consumers who apply for services, including their
• preservice earnings profiles;
• functional abilities;
• types and significance of disabilities;
• interests and motivations;
• demographic characteristics;
• membership in special populations;
• education; and
• work history?
4. Of the consumers designated by VR agencies as having a significant disability, what are their
characteristics?
To address these questions, the study compared the characteristics of groups of VR applicants,
consumers, and former consumers. For the issue of access, it compared applicants who were
accepted for VR services with those not accepted for services. To examine receipt of services, it
compared the characteristics of persons accepted for services who entered VR services with those of
persons accepted who decided to drop out of VR before initiating services under an Individualized
Plan for Employment (IPE). Below are selected study findings from the first final report.

Factors Associated With Access to VR Services

Individuals were more likely to be accepted for VR services if:
• their disability was significant or most significant;
• their disability was congenital rather than acquired;

C-10 FY 2002 Performance and Accountability Report

U.S. Department of Education
 1/31/03

• their disability was classified as either mental retardation or hearing impairment;

• they had higher self-esteem;
• if working at application, they were working at a job in clerical or sales occupations;
• if not working at application, they were a student, unpaid family worker, or volunteer.
Individuals were less likely to be accepted for VR services if:
• their disability was a nonorthopedic physical impairment;
• they were working at application in supported or extended employment;
• they were working at application at a job in a field other than clerical or sales occupations.

Factors Associated with Obtaining an Employment Outcome

Persons who received VR services were more likely to achieve an employment outcome if:
• their disability was a vision impairment, hearing impairment, mental retardation, or
orthopedic impairment, as opposed to another type of disability;
• they had higher gross motor function;
• they had greater self-esteem;
• they were working at application for VR services;
• their desire to obtain assistive technology devices or services was a motive for applying for
VR services;
• they had more dependents than did other consumers.
Persons who received VR services were less likely to achieve an employment outcome if:
• they were receiving SSI or SSDI;
• they were receiving other forms of financial assistance;
• their desire to obtain postsecondary education was a motive for applying for VR services;
• their race/ethnicity was other than white.

Factors Associated with Achievement of a Competitive Employment Outcome versus a Non-

Competitive Employment Outcome
Persons who received VR services were more likely to achieve a competitive employment outcome
if:
• they had higher gross motor function;
• they had higher cognitive function;
• they were working at application for VR services;
• they had higher earnings at their most recent job prior to VR application;
• they had greater knowledge of different jobs;
• they had greater knowledge of the nonmonetary benefits of jobs.
Persons who received VR services were less likely to achieve a competitive employment outcome if:
• their disability was vision impairment, mental illness, or mental retardation;
• their disability was significant or most significant;

FY 2002 Performance and Accountability Report C-11

U.S. Department of Education
1/31/03

• they were receiving SSI or SSDI;

• they were older.

7. Summary: Second Final Report of the Longitudinal Study of the Vocational

Rehabilitation (VR) Services Program: VR Services and Outcomes
The Longitudinal Study has followed 8,500 VR consumers at 37 locations for at least three years for
the purpose of identifying the types of individuals served, types of services provided, environments in
which the services were provided and short-term and long-term economic and non-economic
outcomes of the VR program. The second final report examines the number and type of services
provided to VR consumers, consumer involvement in the service planning process, the relationship
between services and disability, the relationship between services and vocational goal, and service
outcomes, both economic and noneconomic.
Specific study questions that this report addresses are as follows:
1. To what extent does receipt of specific VR services contribute to successful consumer outcomes?
2. What service inputs do state VR agency consumers receive, including:
• the amount, type, duration, and dollar value of purchased services;
• the amount, type, duration, and dollar value of comparable benefits;
• the amount, type, duration, and dollar value of agency-provided consumer services; and
• the amount of counselor time devoted to individual cases?
3. How long do most consumers retain their jobs, and are post-employment services utilized
adequately to maximize consumers’ job retention?
4. What short- and long-term economic and noneconomic outcomes do VR applicants and
consumers achieve as a result of their participation in VR?
5. How do consumers perceive the quality and utility of the services they receive and the
employment they ultimately obtain, as well as other outcomes (independence and community
integration)?
6. What are the long-term outcomes after VR closure, including:
• employment status at one, two, and three years following the consumer’s exit from VR
services;
• the extent that earnings, adjusted for inflation, changed over time;
• employment as it relates to employee benefits (e.g., health or life insurance, etc.) and
opportunities for advancement; and
• the extent that consumer independence and community integration are enhanced?
Below are selected study findings from the second final report.
Overview of VR Services
• Overall, consumers who received VR services averaged 12 services (median 9.0) during
their participation in VR.
• The service most frequently delivered to consumers was medical/physical function
evaluation, with 62 percent of consumers receiving this type of service. Other frequently
C-12 FY 2002 Performance and Accountability Report
U.S. Department of Education
 1/31/03

delivered services included cognitive/psychological assessment and services (35 percent of

consumers), employment-development services (33 percent), postsecondary education
(33 percent), and miscellaneous support services, such as transportation or maintenance
(38 percent).
The VR Process
• VR counselors spent about 14 percent of their time on eligibility determination activities
and an additional 14 percent on IPE development. Counseling/guidance took up
23 percent, and file management and documentation, about one-fourth of their available
time.
• Consumers reported active involvement in decision making and control over the process,
with 81 percent believing that they had sufficient choice in selection of vocational goal and
81 percent commenting that the counselor provided adequate information regarding
available services and service providers. Half reported being in charge of decisions to a
great extent, and an additional 41 percent to some extent.
VR Services and Primary Disability
• The most important factor that affected the specific services individuals received from VR
was their primary disability, both in terms of the types of services persons received and in
terms of the average number of services they received.
• Analysis of relationships between services and disability type reveals that service patterns
do differ by disability type, thus supporting the hypothesis that the VR program
individualizes services to meet consumers’ needs and preferences.
Economic Outcomes of VR Services
• On all measures of economic outcomes, VR consumers who achieved a competitive
employment outcome fared better than did those who achieved a noncompetitive
employment outcome and persons who exited without an employment outcome. For
example, at the third annual follow up, 78 percent of persons exiting into competitive
employment were still working, compared with 70 percent of those exiting into
noncompetitive employment, 37 percent of those who received services but exited VR
without an employment outcome, and 40 percent of those who were eligible but dropped
out before receiving VR services.
• In terms of earnings, by the end of the third year after exit, persons exiting into competitive
employment were much less likely than other consumers to have earnings below the federal
poverty level and much more likely to have earnings more than 200 percent of the poverty
level:
• Controlling for consumer characteristics (e.g., disability type and significance, receipt of
SSI/SSDI, etc.), RTI found that a number of services increased (or decreased) the
likelihood that consumers would achieve competitive employment. Those that increased
the likelihood of such employment were:
1. job development, job placement, on-the-job training;
2. business/vocational school, four-year college/university;
3. tools/uniforms/equipment; and

FY 2002 Performance and Accountability Report C-13

U.S. Department of Education
1/31/03

4. IPE amendment.
An important part of services leading to these outcomes was a relationship between the consumer and
counselor that the consumer believed was productive and helpful, with flexibility to amend the
consumer’s service plan as appropriate to facilitate achievement of the vocational goal. The quality
of the consumer-counselor relationship also contributed to higher earnings, both at closure and at
subsequent follow-up points.
Other Outcomes
• For up to three years following exit from VR services, persons who achieved competitive
employment consistently reported less frequent use of a variety of services than did persons
who exited into noncompetitive employment or those who exited VR without an
employment outcome following services. Comparable figures for the first year after exit
were 8 percent of those exiting into competitive employment, 25 percent for those with a
noncompetitive job, and 18 percent for those exiting services without an employment
outcome. By the end of year three, the percentages were 7, 24, and 16 percent,
respectively.
• In terms of community integration, at study entry fewer persons who later exited into
competitive employment reported that their disability restricted their ability to participate
fully in social and community activities than did those exiting into noncompetitive
employment or those exiting services without an employment outcome (32 percent versus
42 and 43 percent, respectively). Further, those exiting into competitive employment
reported that these restrictions continued to decline (i.e., become less of a problem) over
time; by the end of the third year, only 22 percent reported that their disability restricted
their participation in social or community activities.

8. The National Assessment of Vocational Education—Interim Report to Congress

Nearly half of all high school students and about one-third of college students are involved in
vocational programs as a major part of their studies. Perhaps as many as 40 million adults—one in
four—engage in short-term, postsecondary occupational training. Given the magnitude of the
vocational education enterprise, the ways in which students participate and the benefits they may
receive can have significant consequences for the nation’s workforce.
This report, the first in a series by the new National Assessment of Vocational Education (NAVE),
provides information to help policymakers shape future improvements in this particular component of
American education. Studies still underway will examine the effect of vocational education on
student outcomes, the quality of implementation, and the role of accountability provisions and other
aspects of federal policy; these results will be presented in a final report. Four key themes emerged.
A. Current education, labor market, and policy trends are likely to broaden Perkins reauthorization
debates.
• High schools increasingly emphasize academic reform and college preparation. The poor
performance of seniors on national and international tests, declining graduation rates, and
high rates of college remediation have raised concerns about academic achievement at the
high school level. Partly in response, nearly every state has set higher academic standards
for high school graduation, and many have begun to include exit exams. The challenge

C-14 FY 2002 Performance and Accountability Report

U.S. Department of Education
 1/31/03

many students, including those in vocational programs, face in meeting the new standards
has raised questions about the role of high school courses lacking clear academic focus.
• Good jobs require at least some postsecondary education. Both high- and low-paying
employment are available in the labor market, but a college credential of some kind is
needed for the better-paying jobs. Employment growth in occupations requiring a
vocational associate’s degree is projected to be higher (30 percent) than overall
employment growth (14 percent) through 2008 (Erard forthcoming). Thus, demand for
postsecondary vocational education is likely to remain strong.
• For the past 20 years Perkins has represented a declining share of federal education
budgets, but it is still the largest single source of Department funds spent on high schools.
In fiscal year 1980, funding for vocational education represented about 6 percent of total
ED appropriations; it is now less than 3 percent. Despite the relative declining share,
Perkins III remains the largest single source of federal education funds used to support high
schools. Comparing dollars spent at the high school level, vocational education appears to
be of equal federal priority as other programs focused on raising academic achievement
(Title I) and preparing students for college (TRIO).
B. Federal vocational policy attempts to achieve multiple goals and objectives.
Evolving priorities clearly have moved federal support for vocational education toward fulfilling a
broader set of objectives than training students for work in factories and on farms after high school,
the original aim of federal vocational legislation at the turn of the 20th century. For example, the
stated purpose of the 1998 Perkins III is to enhance not only the vocational and technical skills of
students who choose to participate in vocational education but also their academic skills. In addition,
other sections of the legislation suggest that vocational education is expected to contribute to high
school completion, entry into postsecondary education and training, postsecondary degree
completion, and employment.
C. Secondary vocational education remains a large component of the high school curriculum, but
the full effects of academic reform are not yet evident.
Although there has been little change in the amount of vocational course work taken by high school
students over the past decade, vocational education’s share of the overall high school curriculum has
declined as students earned more academic credits.
• Vocational participation rates have been relatively stable during the last decade. Across
most of the 1990s, almost 45 percent of all high school graduates earned three or more
occupational credits, the equivalent of three, year-long courses. Most of these students (25
percent of all graduates) “concentrated” their courses in a single program area (e.g., health
or business). Occupational “concentrators” are the closest proxy for vocational program
completers.
• Many types of students continue to be involved in vocational education, including those in
“special population” groups. For the most part, there has been little change in who
participates in vocational education over the last decade. Vocational education serves a
diverse set of students, with most coming from the middle range of academic and income
advantage. Still, some groups continue to participate more substantially than others:
students who enter high school with low academic achievement, have disabilities, are male,
English-language proficient, or from lower-income or rural schools. Gender differences
remain. Girls’ vocational course taking has been declining while that of boys has remained

FY 2002 Performance and Accountability Report C-15

U.S. Department of Education
1/31/03

consistent. Despite these trends, differences in the rates of participation in computer

technology courses, geared to a potentially high-paying field, virtually disappeared by
1998.
• Students who participate most in vocational education have increased their academic
course taking, but important gaps remain between them and other students. By 1998 the
gap in academic credits earned between occupational concentrators and other students had
grown smaller (from 1.6 in 1982 to 1.1 credits in 1998). However, vocational students still
take less rigorous academic courses than do other students: for example, substantially fewer
concentrators (26 percent) than non-concentrators (42 percent) completed a college
preparatory curriculum.
• Students take more vocational than math or science courses. Despite the emphasis placed
on academic reforms over the last decade, high school students earn more credits in
vocational education (4.0) than they do in math (3.4) or science (3.1).
These course-taking patterns may well change, as school reform continues and as rigorous state exit
exams become more common. By 1998–1999, nearly 20 states were already phasing in these
assessments, and 6 more were in the process.
D. Postsecondary vocational education serves a diverse set of students, many of whom will not
complete the course work needed to fulfill their objectives.
About one-third of all students in undergraduate postsecondary education are considered to be in
vocational programs. The Perkins Act defines vocational education as occupational programs
requiring less than a baccalaureate degree (P.L. 105-332, Section 3(29)), some of which are offered
at four-year postsecondary institutions. Not only do sub-baccalaureate students outnumber those in
baccalaureate programs, but twice as many sub-baccalaureate students choose a vocational over an
academic major.
These sub-baccalaureate vocational students vary in age and work experience, and they report
enrolling for different reasons—to get an associate’s degree or institutional certificate, to transfer and
pursue a bachelor’s degree, to enhance their job skills, or to engage in personal enrichment activities.
However, like their academic counterparts, many vocational participants leave sub-baccalaureate
institutions and programs having completed few courses. Just under half (47.5 percent) of the
younger students—those less than 24 years old—in vocational programs complete eight or fewer
months of postsecondary course work over a five-year time period. Nearly three-quarters (72.5
percent) of older vocational participants complete no more than eight months of course work. Eight
months of full-time equivalent course work corresponds to what might be considered a year of
postsecondary education and training.

GAO Reports
Student Financial Aid: Use of Middleware for Systems Integration Holds
Promise GAO-02-7 November 30, 2001
Although the Department of Education spent millions of dollars to modernize and integrate its
nonintegrated financial aid systems during the past 10 years, these efforts have met with limited
success. Recently, Education’s Office of Student Financial Assistance (SFA) began using a software
approach known as middleware to provide users with a more complete and integrated view of
information in its many databases. In selecting middleware, SFA has adopted a viable, industry-
C-16 FY 2002 Performance and Accountability Report
U.S. Department of Education
 1/31/03

accepted means for integrating and utilizing its existing data on student loans and grants. To meet its
human capital needs, SFA has solicited the help of a private sector “modernization partner” with
experience in implementing and managing middleware solutions—particularly in the financial
industry—and has also chosen to use a leading middleware software product.

Education Research: Education Should Improve Assessments of R&D Centers,

Regional Labs, and Comprehensive Centers GAO-02-190 January 24, 2002
Research and Development (R&D) Centers, Regional Labs, and Comprehensive Centers support the
Department of Education’s research agenda to various degrees. Because statutes define different
missions and activities for these programs, the amount and focus of the research and other research-
based activities they support varies. Education shapes the priorities that guide the research done by
the R&D Centers and targets the technical assistance provided by the Comprehensive Centers through
requirements in agreements with these entities. However, Education has little control over the
activities of the Regional Labs because, unlike most federal education programs, neither federal nor
state governments have oversight responsibility for their programs. The R&D Centers, Regional
Labs, and Comprehensive Centers reported collaborating and coordinating with each other and
Education and cited various factors that have either facilitated or hindered such activities. They said
that they were most likely to engage in these activities when they shared a common interest in a
specific student population, such as English language learners, or in a specific topic, such as
assessment. Current evaluation practices for assessing the R&D Centers, Regional Labs, and
Comprehensive Centers have provided only limited information about the performance of these
organizations and have not been useful for making future funding decisions.

School Dropouts: Education Could Play a Stronger Role in Identifying and

Disseminating Promising Prevention Strategies GAO-02-240 February 1, 2002
The National Center for Education Statistics (NCES) reports that the national status dropout rate—the
percentage of 16- through 24-year olds who are not enrolled in school and who lack a high school
diploma or a high school equivalency certificate—fluctuated between 10.9 and 12.5 percent between
1990 and 2000. However, dropout rates have varied considerably between regions of the country and
among ethnic groups. Research has shown that dropping out it is a long-term process of
disengagement that begins in the earliest grades. NCES and private research organizations have
identified two factors—an individual’s family and his or her experience in school—that are related to
dropping out. Various state, local, and private programs are available to assist youth at risk of
dropping out of school. These programs range in scope from small-scale supplementary services that
target a small group of students, such as mentoring or counseling services, to comprehensive school-
wide restructuring efforts that involve changing the entire school to improve educational
opportunities for all students. One federal program, the Dropout Prevention Demonstration Program,
is specifically targeted to dropouts, but the program is new and the Department of Education has yet
to evaluate its effectiveness. In September 2001, the program awarded grants to state and local
education agencies working to reduce the number of school dropouts. Other federal programs have
dropout prevention as one of their multiple objectives, and many more federal programs serve at-risk
youth but do not have dropout prevention as a stated program goal.

FY 2002 Performance and Accountability Report C-17

U.S. Department of Education
1/31/03

Federal Student Loans: Flexible Agreements with Guaranty Agencies Warrant Careful
Evaluation GAO-02-254 January 31, 2002
The relationship between the Department of Education and state-designated guaranty agencies that
run the largest federal student loan program is changing in order to achieve program and cost
efficiencies and improve delivery of student financial aid. These state or private not-for-profit
agencies guarantee payment if students fail to repay loans obtained through the Federal Family
Education Loan programs. The 1998 Amendments to the Higher Education Act authorize the
Secretary of Education to enter into “voluntary flexible agreements” (VFA) with individual guaranty
agencies. These agreements allow a guaranty agency to waive or modify some of the federal
requirements that apply to other guaranty agencies. GAO found that the process for developing the
agreements did not fully meet the needs of the guaranty agencies and other program participants. The
process frustrated guaranty agency officials GAO talked to, especially those who ultimately chose not
to apply for a VFA and those who were not granted a VFA. Agency officials said that Education’s
communication about the VFA development process was poor and that Education was unable to meet
its own timetable. The VFAs generally complied with most of the legislative requirements.
However, one of the four agreements does not conform to the requirement that projected federal
program costs not increase due to the agreements. The key changes implemented under the VFAs
include incentive pay structures for guaranty agencies and waivers of certain statutory and regulatory
requirements. Each VFA contains provisions for paying the guaranty agency incentive amounts on
the basis of specific performance measures, such as default rates. Education is not prepared to assess
the effects of VFAs because it lacks a way to adequately measure changes in guaranty agency
performance. The lack of uniform measures makes it difficult to distinguish the results of the VFAs
from the effects of other factors, such as the general condition of the economy. Although the
Department is required to report on the status of the VFA by September 2001, no reports have been
issued so far.

Federal Student Aid: Additional Management Improvements Would Clarify Strategic

Direction and Enhance Accountability GAO-02-255 April 30, 2002
The Department of Education’s Office of Federal Student Aid (FSA) administers more than $53
billion in financial aid for more than 8.1 million students. Since 1990, GAO has included student
financial aid on its high-risk list. To address these and other long-standing management weaknesses,
Congress established FSA as a performance-based organization (PBO) within Education in 1998. To
develop and implement a strategic direction, FSA set three strategic goals, created indicators to
measure progress toward these goals, and developed a tool to link employees’ day-to-day activities to
these goals. The goals are to (1) increase customer satisfaction, (2) increase employee satisfaction,
and (3) reduce unit cost. FSA’s efforts have generally improved customer and employee satisfaction
scores. FSA has begun to implement some human capital practices to better organize its services and
manage its employees. But gaps exist, and FSA has not yet implemented performance management
initiatives to develop and assess its employees. To better serve customers, FSA reorganized to reflect
its different customers—students, schools, and financial partners. To encourage accountability, FSA
is linking staff bonuses to FSA’s strategic goals. Education continues to clarify FSA’s level of
independence and is now reviewing FSA’s role and responsibilities as part of the department-wide
management planning effort.

C-18 FY 2002 Performance and Accountability Report

U.S. Department of Education
 1/31/03

Head Start and Even Start: Greater Collaboration Needed on Measures of Adult
Education and Literacy GAO-02-348 March 29, 2002
The Head Start and Even Start Family Literacy programs have sought to improve the educational and
economic outcomes for millions of disadvantaged children and their families. Because the two
programs seek similar outcomes for similar populations, GAO has pointed out that they need to work
together to avoid inefficiencies in program administrative and service delivery. Questions have also
arisen about the wisdom of having similar early childhood programs administered by different
departments. Head Start’s goal is to ensure that young children are ready for school, and program
eligibility is tied to specific income guidelines. In contrast, Even Start’s goal is to improve family
literacy and the educational opportunities of both the parents and their young children. Even Start
eligibility is tied to parents’ educational attainment. Despite these differences, both programs are
required to provide similar services. Both programs have some similar and some identical
performance measures and outcome expectations for children, but not for parents. Head Start and
Even Start grantees provided some similar services to young children and families, but how these
programs served adults reflect the variations in the need of the parents. No recent, definitive
information exists on the effectiveness of either program so it is difficult to determine which program
uses the more effective model to improve educational outcomes for disadvantaged children and their
parents. At the local level, differences in the needs of participants and the location of neighborhoods
served by the two programs may mean some Head Start and Even Start grantees find only limited
opportunities to work together. At the national level, the Departments of Health and Human Services
and of Education have begun to coordinate their efforts, including the funding of state-level
organizations to improve collaboration among groups serving poor children and their families.

Title I: Education Needs to Monitor States’ Scoring of Assessments GAO-02-393 April

1, 2002
Concerned that Title I of the Elementary and Secondary Education Act (ESEA) had not significantly
improving the educational achievements of children at risk, Congress mandated major changes in
1994. States were required to adopt or develop challenging curriculum content and performance
standards, assessments aligned with content standards, and accountability systems to measure
progress in raising student achievement. In return, states were given greater flexibility in the use of
Title I and other federal funds. The No Child Left Behind Act of 2001 augments the assessment and
accountability requirements that states must implement and increases the stakes for schools that fail
to make adequate progress. The 1994 legislation required states to comply with the requirements by
January 2001 but allowed the Department of Education to extend that deadline. Education has
granted waivers to 30 states to give them more time to meet all requirements. If states fail to meet the
extended timeliness, they are subject to the withholding of some Title I administrative funds. Title I
directors indicated that a state’s ability to meet the 1994 requirements improved when both state
leaders and state agency staff made compliance a priority and coordinated with one another. Most
directors said that inadequate funding hindered compliance. Many of the states reported taking action
to ensure that Title I assessments were scored accurately, that any exemptions for students with
limited English proficiency were justified, and students with disabilities were receiving appropriate
testing accommodations. As of March 2002, 17 states had complied with the 1994 assessment
requirements; 35 states had not.

FY 2002 Performance and Accountability Report C-19

U.S. Department of Education
1/31/03

Education Financial Management: Weak Internal Controls Led to Instances of Fraud

and Other Improper Payments GAO-02-406 March 28, 2002
Because of internal control weaknesses, the Department of Education’s student financial assistance
programs are at high risk for fraud or erroneous payments. GAO discovered fraud in the grant and
loan areas and pervasive control breakdowns and improper payments in other areas. Controls over
grant and loan disbursements lacked a key edit check or follow-up process that would identify
schools improperly disbursing Pell Grants. Significant internal control weaknesses over Education’s
third party drafts also increased the department’s vulnerability. GAO found that individual Education
employees could control the entire payment process for third party drafts. Education employees also
circumvented a key computerized control designed to prevent duplicate payments. Education
eliminated third party drafts in May 2001. Inconsistent and inadequate authorization and review
processes for purchase cards, combined with a lack of monitoring, meant that improper purchases
were unlikely to be detected. Inadequate control over these expenditures, combined with the inherent
risk of fraud and abuse associated with purchase cards, led to fraudulent, improper, and questionable
purchases totaling $686,000. Poor internal controls over computers acquired with purchase cards and
third party drafts led to 241 missing personal computers and other equipment valued at $261,500.
Although Education changed policies and procedures over disbursements to improve internal controls
and program integrity, many of these changes have not been effectively implemented.

Information Management: Selected Agencies’ Handling of Personal Information GAO-

02-1058 September 30, 2002
To obtain government services, members of the public must often provide agencies with personal
information, which includes both identifying information (such as name or Social Security number,
which can be used to locate to identify someone) and nonidentifying information (such as age or
gender). GAO was asked to review agencies’ handling of the personal information they collect and
whether this handling conforms to federal law, regulation, and agency guidance.
GAO reviewed the processes used in handling personal information collected from the public forms
at four different agencies—Agriculture, Education, Labor, and State. These four agencies were
chosen because their forms represent a range of characteristics, including the time needed to fill them
out (the total paperwork burden hours) and the purpose of the information they collect. In reviewing
these forms, GAO concentrated on four areas (information collection, privacy, security, and records
management). Handling of personal information varied among the agencies studied. Overall,
agencies collected a substantial amount of personal information of a wide variety of types, including
personal identifying information (names and Social Security numbers) and demographic, financial,
and legal data. Agency procedures for handling personal information collected were complex,
involving numerous processes and a wide range of personnel with access to the information. The
personal information collected was shared extensively with other federal agencies, other government
entities (state, local, tribal and foreign), and private individuals and organizations through authorized
procedures. The agencies generally complied with the key requirements and guidance pertaining to
information collection, privacy, security, and records management. However, GAO identified
isolated instances of forms that were not accurate or current; other forms did not contain the proper
privacy notices.
There are 17 GAO recommendations that remain open.

C-20 FY 2002 Performance and Accountability Report

U.S. Department of Education
 1/31/03

Appendix D: Program Assessment Rating Tool (PART)

In 2002, the Office of Management and Budget (OMB), in collaboration with the U.S. Department
of Education, conducted 18 reviews using OMB’s newly developed Program Assessment Rating
Tool (PART). The 18 reviews covered 17 grant programs and 1 administrative function. The PART
is a series of questions designed to provide a consistent approach to evaluating and rating the
effectiveness of federally funded programs. The goal of this process is to identify programs’
strengths and weaknesses and to take action to improve overall program performance, and to provide
information for budget justifications.
The PART is a diagnostic tool that assesses and evaluates programs across a range of issues related
to program performance. Each PART is divided into four sections:
(1) Purpose/Relevance/Federal Role examines the clarity of program purpose and related
program design and looks at all factors, including those the program may not directly control,
such as legislation.
(2) Strategic Planning focuses on program planning, priority setting, and the allocation of
resources and assesses whether the program has a number of goals that are ambitious, yet
achievable, to ensure that planning is strategic and focused.
(3) Program Management focuses on a variety of elements related to whether the program is
effectively managed to meet program goals and objectives and includes such as areas as
financial oversight, evaluation of program improvements, performance data collection, and
program manager accountability.
(4) Program Results considers whether a program is meeting its long term or annual goals and
assesses how well the program is doing compared to similar programs and its effectiveness
based on independent or external evaluations. Each of these sections includes a series of
questions designed to elicit specific information for program evaluation.
OMB identified seven categories of programs across the Federal government: competitive grant
programs; block and formula grant programs; regulatory based programs; capital assets and service
acquisition programs; credit programs; direct federal programs; and research and development
programs. There is a separate PART for each of the categories, though the majority of the questions
are alike on all forms. Questions concerning program purpose, and strategic planning, and program
results apply to all programs. Specialized questions are added to accommodate the differences in
types of programs.
The PART holds programs to a high level of evidence and expectation. It is not sufficient for a
program simply to comply with the letter of the law. Rather the program must show that it is
achieving its purpose and that it is managed efficiently and effectively. The performance of federal
programs should reflect the spirit of good government not merely compliance with statute. The
PART requires a high standard of evidence to receive a “yes” answer to a question.
Although the PART uses the Government Performance and Results Act (GPRA) performance
measures as a starting point, previously used measures may not meet the PART’s high standards.
The PART requires federal agencies to choose performance measures that meaningfully reflect the
mission of the program, not merely ones for which there are data. Performance measures should
reflect a sense of program priorities and, as a general rule, should reflect outcomes rather than

FY 2002 Performance and Accountability Report D-1

U.S. Department of Education
1/31/03

outputs. To score high on the PART, programs may need to revise their indicators to reflect PART’s
focus on outcomes.
To ensure fairness in interpreting PART questions, OMB and Department representatives have
partnered to create the Interagency Review Panel (IRP) to resolve issues of concern and ensure
consistency in the use of the PART instrument.
The Department’s intention is to improve the strategic planning process of its programs and we are
using the PART process to begin revising long-term and annual performance measures for our
programs. We are working to establish appropriate indicators and numeric targets that reflect high
performance and to compile available data in support of these measures. Where appropriate, we are
developing common measures for programs with similar desired outcomes.
The following Department of Education programs were reviewed under the PART process in
FY 2002:
• Adult Education State Grants
• America’s Career Resource Network
• Comprehensive School Reform
• Even Start
• Federal Pell Grants
• IDEA Grants for Infants and Families
• IDEA Grants to States
• IDEA Preschool Grants
• National Assessment (NAEP)
• National Center for Education Statistics (NCES)
• Safe and Drug Free Schools and State Grants
• Student Aid Administration
• Tech-prep Education State Grants
• Tribally Controlled Vocational & Technical Institutions
• TRIO Student Support Services
• TRIO Upward Bound
• Vocational Education State Grants
• Vocational Rehabilitation State Grants

D-2 FY 2002 Performance and Accountability Report

U.S. Department of Education
 1/31/03

Appendix E: The Department of Education’s Information

Quality Guidelines and Data Quality
Standards

Soon after the inception of the Government Performance and Results Act (GPRA), the Department
of Education identified data quality as a major challenge to our successful implementation of the
Act. In the early 1990’s the Department did not regularly have reliable information about its
programs available to allow managers to make program decisions. As a result of our commitment to
improve our data for GPRA reporting, the Department developed draft Data Quality Standards in
1998 to guide program managers in the collection and analysis of program data.
In 2002, in response in response to Section 515 of the Treasury and General Government
Appropriations Act and OMB guidance on its provisions, the Department developed Information
Quality Guidelines to guide program managers in preparing data for release to the public.
Through the development of these standards and guidelines, employee training, and other
mechanisms, the Department continues to make the improvement of data quality a top priority. The
Department’s Strategic Plan emphasizes the importance of information quality. Goal 4 of the six
strategic goals is to “Transform Education into an Evidence-based Field.” Under this goal, the
Department seeks to ensure that research funded or published by the Department is of the highest
quality and that that high-quality research—whether or not it is funded by the Department—is
synthesized, publicized, and disseminated widely. The Department also relies on high quality
information in the administration of its programs. For example, Objective 1.1 under the Strategic
Plan requires the Department to link federal education funding to accountability for results.
Consequently, programs that cannot demonstrate evidence of effectiveness will be candidates for
reform or elimination. High quality information is required to demonstrate evidence of effective
programs. To make sound decisions, the Department intends to accept and use only information that
is accurate and reliable.
The Department’s new Information Quality Guidelines, which apply to the release of information to
the public, complement the draft Data Quality Standards, which apply to the collection and analysis
of data. Both are presented below in abbreviated formats.

Information Quality Guidelines

The Department of Education’s Information Quality Guidelines (Guidelines) reflect the
Department’s policy and procedures for reviewing and substantiating the quality of information it
disseminates, (e.g., reports, studies, and summaries), as well as provide an administrative mechanism
allowing affected persons to seek and obtain, where appropriate, correction of information that does
not comply with the Guidelines. The Department’s Guidelines, along with those issued by OMB,
represent a performance goal for the Department and will improve data management within the
Department.

FY 2002 Performance and Accountability Report E-1

U.S. Department of Education
1/31/03

Our Guidelines require that Department staff treat information quality as integral to the creation,
collection, maintenance, and dissemination of information, and review products before they are
disseminated to ensure that they are consistent with these Guidelines.
Consistent with guidance from OMB, we consider three factors in the assessment of information
quality for any piece of information that is to be disseminated. These factors are utility, objectivity,
and integrity. These elements are intended to ensure that information the Department disseminates
is useful, accurate, reliable, unbiased, and secure.
The Department disseminates various kinds of information. All information that the Department
disseminates must be reviewed for utility, objectivity, and integrity, but the specific application of
these factors is targeted to the specific needs and appropriateness for various types of publications.
We have identified four most common types of information products:
• General Information
• Research and Evaluation Information
• Administrative and Program Data, and
• Statistical Data
While the guidelines for integrity are the same across all four types, guidelines for utility and
objectivity have been established for each of these four types of products. Exhibit 1 on the following
page shows a sample of the information quality standards for statistical data (other than NCES data,
which have separate published standards).
The Guidelines also provide that the level of quality assurance for information must be tied to its
level of importance. Influential information, that is information that will or does have a clear and
substantial impact on public policies or private sector decisions, must meet a higher level of quality.
Influential information must be reproducible according to commonly accepted scientific, financial,
or statistical standards for that type of data. It must be accompanied by supporting documentation
that allows an external user to reproduce it, taking into account any ethical and confidentiality
restraints. In the case of analytic results, the mathematical and statistical processes used to produce
the report must be explained in sufficient detail to allow an independent analyst to substantially
reproduce the findings using original data and identical methods. In situations where the public
cannot access the data due to compelling interests such confidentiality protections, the Department
will apply especially rigorous robustness checks and document these checks.
The Guidelines also detail a method for any person to request the correction of information the
Department disseminates that does not comply with Department and OMB information quality
guidelines.
A complete copy of the Department’s Information Quality Guidelines is available at
http://www.ed.gov/offices/OCIO/info_quality/.

E-2 FY 2002 Performance and Accountability Report

U.S. Department of Education
 1/31/03

Exhibit 1
Information Quality Guidelines
Sample Checklist Items for Statistical Data

Utility
Is the information grammatically correct and clearly written in plain English?
Is the audience identified?
Will the information help ensure equal access to education and promote educational
excellence throughout the nation?
Does the information fill needs that are identified through internal review, legislative
mandates or input from users outside the Department?
Objectivity
Statement of Purpose
Where does the product clearly describe the goals of the study?
Where does the report provide the reason the information is provided, its potential uses, and
cautions as to inappropriate extractions or conclusions?
Research Based
Where is the description of the data collection process with survey instruments that were
properly developed and tested?
Where is the description of how the response rates were monitored during data collection to
ensure the respondents are representative of the population?
Where does the product show findings and data collection properly documented with an
evaluation of the quality of the data including its known limitations (e.g., missing values,
amount of non-response)?
Sources
Where is the reliability of the data source(s) addressed?
Where does the report include the identification of other possible sources of potentially
corroborating or conflicting information?
Burden Reduction
Where does the report describe every effort made to minimize the amount of time required
for responses?
Reproducibility
Where is a description of the analytical work in sufficient detail to ensure that the findings
could be reproduced using the same data and methods of analysis?
Review and Release
Was all work conducted and released in a timely manner?
What are the procedures to correct any identified errors?
Integrity
Have we safeguarded personally identifiable information?
Have we ensured protection from unauthorized disclosure, alteration, loss, or destruction?

FY 2002 Performance and Accountability Report E-3

U.S. Department of Education
1/31/03

Draft Data Quality Standards

In 1998, the Department set out to develop data quality standards for program data collections. In
close consultation with the Office of Inspector General, the National Center for Education Statistics,
and several program offices, staff reviewed existing standards for data quality and modified them to
address recurring data issues in Departmental reporting. Many data quality concepts, such as
validity of measures, accuracy of definitions, and reporting for use are integral to the more general
process of developing high-quality performance measures.
The draft Data Quality Standards (see Exhibit 2) include eight standards for judging program data
and related operations. An important decision was made to write the standards in clear, nontechnical
language. For example, Standard #5 is “Calculation: The math is right.” This feature differentiated
the Education standards from typical documents on statistical standards. For example, the National
Center for Education Statistics (NCES) publishes its comprehensive set of standards in a lengthy
document that is more suited for statisticians than typical program managers. NCES’ standards can
be viewed at http://nces.ed.gov/statprog/2002/stdtoc.asp
To facilitate the use of the standards, a data quality checklist was created. This checklist provides
three to ten questions for each standard to assist program staff who are unfamiliar with data issues to
evaluate the extent to which their data meets the standards. For example, under the Calculation
standard, there is the question: “Are missing data procedures applied correctly?” For those who
might not understand the question and the importance of accounting for missing data, the checklist
has a section that explains each question and offers examples of meeting and not meeting the
criterion.
Classes teaching staff how to apply the Standards are popular and have been held regularly; and
some programs have adopted and modified the Standards to use with their data providers.

E-4 FY 2002 Performance and Accountability Report

U.S. Department of Education
 1/31/03

Exhibit 2
Data Quality Standards
Sample Checklist Items

1. Validity: Data adequately represent performance.

Have the objective, performance indicator and data been scrutinized to be sure that they
all describe the phenomena of interest?
2. Accurate Definitions: Definitions are correct.
Have clear, written definitions of key terms (including inclusions/exclusions) been
communicated to data providers?
3. Accurate Counts: Counts are correct.
Are counts accurate; e.g., is double counting avoided?
4. Editing: Data are clean.
Have you discussed large changes or unusual findings with the primary data providers
to see if they might be due to editing errors?
5. Calculation: The math is right.
Have the + or – confidence intervals been reported for sample data?
6. Timeliness: Data are recent.
Do data meet decision-making needs?
7. Reporting: Full disclosure is made.
Are data-quality problems at each level reported to the next level?
8. Burden Reduction: Data collected are used.
Are all data that are collected actually used?

FY 2002 Performance and Accountability Report E-5

U.S. Department of Education
1/31/03