Using Student Achievement Data to Support

Instructional Decision Making

Sharnell S. Jackson
Data-Driven Innovations Consulting
March 18, 2013

The Panelists and Co-Authors

• Laura Hamilton (Chair), RAND Corporation

• Richard Halverson, University of Wisconsin –
Madison
• Sharnell S. Jackson, Chicago Public Schools
• Ellen Mandinach, CNA Education
• Jonathan A. Supovitz, University of Pennsylvania
• Jeffery C. Wayman, University of Texas – Austin

2 2

1
 What is data-driven decision making?

The process by which an individual collects,

examines, and interprets empirical evidence

for the purpose of making a decision.

3 3

What are data?

• Pieces of information
• Data are meaningless by themselves and given
meaning through the context in which they occur in
instruction

• Context transforms data into information that is usable

to a decision-maker

• Educational data may be demographic, financial,

personnel, annual, interim, or classroom-level

4 4

2
 Fundamentals about data:
The Data Continuum

• Data – exist in a raw state without meaning

• Information – data given meaning in context
• Knowledge – collection of information deemed useful
to guide action

5 5

Why focus on data use?

• Technological advances, a proliferation of assessment

data, and human capacity issues
– Data and reports are increasingly accessible to
educators, students, and parents
– Need to promote appropriate use

• Growing recognition of the need to individualize

instruction to address achievement gaps and meet
accountability targets to help all children learn

• Policy requirements from the federal government

6 6

3
 Recent quotes to set the stage

Our best teachers today are using real time data in ways
that would have been unimaginable just five years ago.
They need to know how well their students are performing.
They want to know exactly what they need to do to teach
and how to teach it. (Duncan, 2009)

Data and data analyses are powerful tools that must be

used to improve schools. (Easton, 2009)

7 7

Continuous improvement process

• Target • Use data to

research to identify a
examine the problem
impact
•
Impact Problem

Monitor Solution

• Provide • Identify
continuous possible
monitoring solutions

Bryk, Sebring, Allensworth, Luppescu, and

Easton, (2010). Organizing Schools for
Improvement . The University of Chicago Press 8 8

4
 Practice guide structure

• Recommendations
• Action Steps
• Potential Roadblocks & Solutions
• Vetted References

9 9

Development process

• Input from expert panel of professors,

researchers in non-profit organizations, and
practitioners

• Research reviewed by What Works

Clearinghouse

• Examined hundreds of articles

(2,853 495 64 24)

• Recommendations
• Peer review
1010

5
 Notes on the guide (1)

• Scope of this guide is limited to typical assessment

data, but readers should consider how to integrate data
from multiple sources.

• Consider all five recommendations as a research-

based framework for effective data use that requires
coordinated, systemic mindset efforts across all levels
of the education system.

11 11

Notes on the guide (2)

• Recommendations have a low level of evidence by

WWC standards, but there is extensive support from
descriptive studies, case studies, and surveys.

• Focus is on the LEA level, but can be generalized to

the SEA level.

1212

6
 Recommendations

• Make data part of an ongoing cycle of instructional

1 improvement

• Teach students to examine their own data and set

2 learning goals

• Establish a clear vision for school-wide data use

3

• Provide supports that foster a data-driven culture

4 within the school

• Develop and maintain a district-wide data system

5
1313

Recommendation 1: Make data part of an

ongoing cycle of instructional improvement

Modify Collect and

instruction to prepare a
test hypotheses variety of data
and increase about student
student learning learning

Interpret data
and develop
hypotheses
about how to
improve student
learning

1414

7
 Characteristics of interim assessments

• Administered each semester, quarter, or month

routinely throughout the year
• Administered in a consistent manner across a
particular grade level and/or content area
• May be commercial or developed in-house
• May be administered on paper or online
• May be scored by a computer or a person

1515

Characteristics of testable hypotheses

• Identify a promising intervention or instructional

modification and an effect that you expect to see

• Ensure that the effect can be measured

• Identify the comparison data

1616

8
 Modify instructional changes to raise
student achievement by…
• Using formative assessment as a feedback loop to close
gaps
• Allocating more time for struggling students
• Reordering the curriculum to shore up essential skills
• Designating particular students to receive additional help
• Attempting new ways of teaching difficult or complex
concepts
• Aligning performance expectations among classrooms and
grades
• Aligning curricular emphasis among grade levels based on
analysis 1717

Self-Assessment: Can you complete the

cycle of instructional improvement?

• Quickly answer each question about your school with

a “yes,” “no,” or “sometimes”

• For each “no” or “sometimes,” jot down a few notes

about:
– What you might do to improve the situation; and
– What difficulties you see

18
18

9
 Recommendation 1:
Roadblocks and potential solutions

• Too much data

– Ask focused questions based on evidence
– “Triangulate” data (bring the sources together)
• Content areas don’t have readily available data
– Work across classes and content areas
– Use multiple sources of local data and information
– Develop school-wide interim or common
assessment data to monitor progress

1919

Recommendation 1:
Roadblocks and potential solutions

• A focus on the “bubble” kids

– Provide resources to all students according to
strengths and weaknesses
– Single test may produce errors in measurement

• Course assignment based solely on scores

– Use tests for valid purposes only
– Use multiple measures, not only one score

20 20

10
 Recommendation 2: Teach students to
examine their own data and set learning goals

Explain expectations and assessment criteria.

Provide feedback that is timely, specific,

well-formatted, and constructive.

Provide tools that help students learn from feedback

to increase individualization.

Use students’ data analyses to guide instructional

changes and learning options.
21121

Expectations and assessment criteria for

student self-assessment

• To interpret their own achievement data, students need

to understand how their performance fits within the
context of classroom-level or school-wide expectations.

• Teachers should articulate the content knowledge and

skills that they expect students to achieve throughout the
school year, conveying goals for individual lessons and
assignments, as well as goals for the unit and end-of-
year performance.

• Teachers should explicitly describe the criteria that will

be used to assess performance toward those goals.

2222

11
 Feedback for students

• Timely — Feedback should be rapid so that students

still remember the task and the skills on which they
were being assessed.

• Appropriately formatted — When providing

feedback, teachers should select a mode of delivery
(e.g., rubric based, handwritten, or typed) that best
meets students’ needs based on their grade level, the
subject area, and the assignment.

• Specific and constructive — Regardless of the

format, feedback should provide concrete information
and suggestions for improvement.
2323

Student worksheet for reflecting on

strengths and weaknesses
Areas of Strength and Areas of Growth

Topic: Writing a Five-Paragraph Essay

Based on: Rubric-based feedback from my last two essays
Name: John R. Student

Areas of Strength Areas of Growth

Organization and Content Organization and Content

• Stating the main idea in first • Need to state main idea of each body
paragraph paragraph
• Restating main idea in conclusion • Need to provide examples in each body
• Choosing a topic I know well paragraph

Grammar and Usage Grammar and Usage

• Indenting paragraphs • Using quotations correctly
• Correctly capitalizing sentences and • Avoiding sentence fragments (example:
proper nouns “Because he wanted to.”)
24
24

12
 Possible solutions

• Individualized learning plans

• Electronic portfolios
• Instructional management systems

2525

Example of a student’s worksheet for

learning from mistakes

Learning from Math Mistakes

Test: Unit 2. Single-Variable Equations
Name: Monroe J. Student
Problem My Answer Correct Steps for Reason Need to
Number Answer Solving Missed review this
(from post-test (fill in) concept?
review)

10 x = √21 x=3 Order of Yes

operations

18 x = 3/32 x = -3/2 Dividing Yes

by a
fraction
27 x=4 x = 4 or -4 Square No
roots
26

13
 Recommendation 2:
Roadblocks and potential solutions

• Students view feedback as reflection on their abilities

rather than opportunity for improvement
– Define learning goals that are focused and specific
– Don’t make statements linking performance to ability

• Different teachers, different approaches

– PD for teachers in providing useful feedback
– Collaborate using common assessments and
performance data to set measurable goals with
students

2727

Recommendation 2:
Roadblocks and potential solutions

• Teachers are concerned that they do not have

enough instructional time to explain rubrics or help
students analyze feedback
– Should be a regular part of teaching activities
– Integrate it into the curriculum, instruction, and
assessment cycle on a regular basis
– Time spent explaining tools and strategies for
analyzing feedback is essential to helping students
understand their own achievement

2828

14
 Recommendation 3: Establish a clear vision
for school-wide data use

Vision for Data Use

• Establish a school-wide
data team for ongoing
data use.
• Define critical teaching
and learning concepts.
• Develop a written plan
that articulates activities,
roles, and responsibilities.
• Provide ongoing data
leadership.

2929

Data team roles and responsibilities (1)

• Collecting and analyzing a variety of school data

• Developing or adapting common assessments
• Committing to norms of collaboration and examining
data from equity perspective

• Using processes to identify and monitor student

learning problems, verify causes, generate solutions,
monitor, and achieve results

• Consulting research to investigate problems, causes,

and effective best practices

• Developing data‐supported action planning 3030

15
 Data team roles and responsibilities (2)

• Communicating about the findings of action plans

• Overseeing implementation of action plan and
classroom instructional improvement

• Sharing successes and challenges

• Engaging stakeholders to gain input, involvement,
support, and commitment

• Coordinating with other school or district initiatives

• Developing knowledge and skill in data literacy,
collaborative inquiry, content knowledge, proficiency,
leadership, and facilitation 3131

Communication, collaboration,
and sharing are key

• What effective data practices have you been

using?

• What is the disconnect between data and results?

• What is needed to make the connection between
data and results in classrooms?

3232

16
 Ensure action plan goals are…

• Attainable, in that they are realistic given existing

performance levels

• Measurable, in that they clearly express the

parameters of achievement and can be supported by
data

• Relevant, in that they take into account the specific

culture and constraints of the school

3333

Ongoing data leadership

• Provide resources and support for data analysis and

interpretation, such as information about professional
development sessions and access to necessary
technologies.

• Encourage educators to use data in their daily work by

modeling data-use strategies.

• Create incentives to motivate staff to analyze data.

• Participate in grade- and subject-level meetings to
ensure structured collaboration time is used effectively.

3434

17
 Recommendation 3:
Roadblocks and potential solutions

• Staff do not have time to plan data use

– Integrate data use into the school improvement plan
and daily classroom instructional practices

• Lack of human capacity

– Look at staff strengths and leadership skills
– Help build capacity of a few (turnkey model)
– Encourage participation through incentives and
distributing leadership

3535

Recommendation 3:
Roadblocks and potential solutions

• Those who understand data are overwhelmed

– Define roles and responsibilities with job descriptions
– Turnkey model of training and structured
collaborative time
– Phase in data use throughout the school with support
• No research and development staff participation
– Consistent message from district and principal on
data use
– Build distributed leadership capacity to ensure data
use is accurately presented to data team members in
schools
3636

18
 Recommendation 4: Provide supports that
foster a data-driven culture within the school

Dedicate
structured
time for staff
collaboration
Provide
Designate a
regular
school-based
targeted
facilitator to
professional
discuss data
development

Essential
Supports

3737

Data facilitators

• Model data use and interpretation, tying examples to the

school’s vision for data use and its learning goals.

• Model how to transform daily classroom practices based

on data-driven diagnoses of student learning issues.

• Assist educators with data interpretation by preparing

data reports and related materials.

• Train and support educators on using data to improve

instructional practices and student achievement.

3838

19
 Collaborative team meetings

Preparation — Prior to meetings, the data team should set

an agenda that focuses on using the most updated data
relative to a specific and timely topic.

Analysis — During these meetings, teachers should follow

the cycle of inquiry, using data to state hypotheses about
their teaching and learning practices and then testing
those hypotheses.

Action agenda — At the end of each meeting, educators

should be prepared to enact a data-driven action plan that
examines and modifies their instruction to increase
student achievement in the area of focus for the meeting.
39

Suggested professional development and Principals Teachers Other Information

training opportunities Staff Technology Staff
Avoid common data analysis and interpretation X X X
mistakes
Data system use – avoid common mistakes X X X
Data system use – maintenance and X X
troubleshooting
Data system use – reporting capabilities X X X X
Data transparency and safety X X
Encourage staff leadership X
Foster a culture of data-based decision making X X
Identify needs for staff professional X X
development
Interpret data in an educational context X X X
Organize time for collaborative data X X X
discussions
Understand and use the cycle of instructional X X X
improvement
Use data to answer questions about student X X X
learning
Data system use – entering data X X
Use data to modify teaching and learning X X X
practices 40

20
 Recommendation 4:
Roadblocks and potential solutions

• Hard to find professional development tailored to

needs of the school
– Work with the PD providers so they understand
your needs and capacity
– Use the train-the-trainers model for sustainability
– Identify internal staff who can provide and support
PD

41

Recommendation 4:
Roadblocks and potential solutions

• Resources are shifted to other priorities

– Integrate interpretation and data use into all subjects
and grades with PD, materials, and support services
– Dedicate resources and structured time to improve
data literacy in order to support and enforce a culture
of effective data use

42

21
 Recommendation 5: Develop and
maintain a district-wide data system

Involve a Articulate
variety of system
stakeholders in requirements
selecting a data relative to user
system needs

Determine Plan and stage

whether to implementation
build or buy the of the data
data system systems

43

Staff Title Example of Uses of Data System

Administrators Compare rates of discipline referrals among different groups of students;

and principals discuss student progress and classroom pedagogy with faculty.
Counselors Place students into correct classes based on prior performance and
current schedule constraints; discuss student progress and needs with
other building educators.

Information Assess the interoperability of data systems, identify project scope; build
technology strong project plans; establish standards; manage differentiated access by
staff stakeholders; provide support, maintenance over time; identify challenges
that might prevent or hinder systems from working together for timely
access to information.

Support staff Use attendance and assessment data to identify students for targeted
interventions; work with faculty and administration on data use strategies
and changing practice.

Teachers Identify student and class strengths and weaknesses; interact with other
staff about student progress.
Parents Track immediate student outcomes and compare student performance
over time.
Students Review scores on recent assessments and track progress on outcomes.
44

22
 Stakeholder data system responsibilities

• Developing roles and structures to oversee the

district’s commitment to data quality and use

• Providing guidance about the requirements and design

of the data system

• Overseeing system development

• Serving as liaison between data-system advisory
council and its respective stakeholder groups

45

Recommendation 5:
Roadblocks and potential solutions

• Technology is hard to use

– Provide training and support for a variety of levels

– Ensure PD supports teachers’ use of data for

instruction

– Build knowledge of the technology, management,

support, capacity, and sustainability

– Provide continuous help focused on classroom

instructional needs—use it or lose it
46

23
 Recommendation 5:
Roadblocks and potential solutions

• No specifics on how to use technology in the

implementation plan
– Address teaching and learning goals for data system
requirements to better understand how it will be used
– Bring educational goals to the forefront

• Data systems are a financial luxury

– Not true! The use of student data to meet educational
improvement goals requires a data system that
supports teaching, learning, and continuous school
improvement
47

What do we need?
Train and support educators to use data

• Provide professional development courses for pre-

service and in-service educators for interpreting data to
improve instruction.

• Adopt a systematic process for using data and

designate a school-based facilitator to provide ongoing
professional learning, resources, and support for data
analysis and interpretations.

48

24
 What do we need?
Train and support educators to use data

• Recognize that professional learning about data-

driven decision making helps educators transform
data into actionable instructional processes through
research-based PEDAGOGICAL DATA LITERACY
SKILLS AND KNOWLEDGE!

• Recognize that a continuous school-wide

perspective on embedding data use into everyday
instructional practices is not a one-time event.

49

The payoff may be great!

• Help your students to meet their individual learning

needs by:
– Building the human capacity of teachers to use
data effectively to drive instructional decisions
– Using multiple data sources to identify strengths,
weaknesses, and interventions and inform instruction
– Customizing instruction to personalize students’
individual learning needs and monitor progress to
accelerate achievement and close gaps

50

25
 Think about…

• What are YOUR goals for using data?

• What are YOUR roadblocks?

• What are YOUR potential solutions?

• What help do you need and from whom?

51

Summing it up!

• We live in a time with unprecedented attention being

given to data-driven decision making.
– Funds for statewide longitudinal data systems
– Proliferation of technology-based tools
– Proliferation of data and assessments
– The need for human capacity to catch up

• Data are not going to go away, and educators must

learn to use them effectively.

52

26
 Summing it up!

• The research evidence needs to catch up, but our

experiences tell us that effective data use can make a
difference.

• The practice guide provides a data-use inquiry cycle

process as a framework for data analysis to guide
instruction which is fundamental as a starting point.

53

Q/A

?
54

27
 Strategic Planning Session (1)

• Look at Section B of the “Comprehensive Planning

Template” (pages 3-6)

• Review the 14 “Areas of School Responsibility” there

and note which are already in place in your
school/district

• Choose a problem area and plan how to improve it

– Record individual responses on the template
– Record team responses on easel paper

55

Strategic Planning Session (2)

For that Area of School Responsibility, discuss:

– What is your school doing now?
– What progress have you made recently?
– What next steps will help you get there?
– Who needs to be involved?
– What problems do you foresee?
– How can you work around those problems?
– When can you start this process?

56

28