Course Assessment and

Student Learning Outcomes

Danielle Mihram, Ph.D.

Distinguished Faculty Fellow
USC Center for Excellence in Teaching
[email protected]

 Course Assessment and
Student Learning Outcomes
Our Goals for this Session

Course and classroom assessment techniques range from simple to

complex strategies to motivate and engage students while
collecting feedback on their learning.

At the end of this workshop instructors should be prepared to:

State the relationship between course objectives and
assessment of student learning.
Identify and assemble a reliable and valid set of successful
classroom assessment tools and techniques, and know how to
use them effectively and appropriately.
Demonstrate their understanding of the relationship between
course assessment, classroom assessment, and evaluations of
teaching effectiveness.

 Overview

What is Assessment?
Our focus today: Course assessment and its intended
learning and achievement outcomes
Course assessment
Classroom assessment techniques
Learning assessment (Assessing students ability to
think critically and solve problems)
Using rubrics to provide feedback to students
Reflecting on ones teaching

 The Word Assess

From the Latin verb assidere = to sit by (e.g., as an assessor or

assistant-judge, originally in the context of taxes)
Hence in assessment of learning = to sit with the learner
Implies it is something that we do with and for students and not
to students

Assessment is the art and science of knowing what students know

It provides evidence of students knowledge, skills, and
abilities
Evidence supports instructors inferences of what students
know and can do (it guides and informs instruction)

 One Definition of Assessment in Education

Assessment is the process of Huba, M. E. & Freed, J. E. (2000).

Learner-Centered Assessment on
gathering and discussing
College Campuses - Shifting the Focus
information from multiple from Teaching to Learning. Boston:
and diverse sources in order Allyn and Bacon.
to develop a deep
understanding of what
students know, understand,
and can do with their
knowledge as a result of their
educational experiences; the
process culminates when
assessment results are used to
improve subsequent learning.
(p. 8)

Learner-centered Assessment in Higher Education

Three types, each with a very different focus:

Institutional assessment

Curricular and program assessment

Course and learner-centered assessments

Our focus today: Course and classroom assessments

 Overview

What is Assessment?
Our focus today: Course assessment and its
intended learning and achievement outcomes
Course assessment
Classroom assessment techniques
Learning assessment (Assessing students ability to
think critically and solve problems)
Classroom assessment techniques
Using rubrics to provide feedback to students
Reflecting on ones teaching

 As Discussed in Previous Workshop:
Developing Course Objectives

General objectives: A course objective is a simple statement of what you

expect your students to know.
Determining the objectives is the most important aspect of course
planning (Ask yourself, What do students need to know in order to
derive maximum benefit from this educational experience? What
educational outcomes do I want a graduate of this course to
display?).
Plan backwards from where you want students to end in terms of
their new knowledge, attitudes, and skills.
List these as learning objectives (student learning outcomes) [by
the end of the course you will be able to].
Design the course in a logical and scaffolded sequence of learning
activities (reading assignments, lectures, quizzes, technology-
mediated experiences, formative assessments)

 As Discussed in Previous Workshop:
Learning Outcomes

What your students will learn within the content of a body

of knowledge
Each course objective should lead to an actionable
learning outcome: A short statement, formulated
from the professors point of view, beginning with a
verb and providing actionable outcomes:
Introduce students to so that; help student
discover and then ; develop the ability to so
as to transfer to ; give students a theoretical
and practical overview to .

 Course Assessment:
A Review of the Terminology

Learning outcomes or learning intentions

What do we want students to know and be able to do as a result
of this learning experience
Achievement criteria or success criteria
What will students need to do in (or out of) class to achieve the
learning outcomes/learning intentions, and to what standard?
Context or task
What kind of learning experience will be appropriate to achieve
the learning outcomes/learning intentions?

N.B.: Students understanding of the task and their achievement will be maximized if both
the achievement criteria and the learning outcome(s) are shared with them prior to
the lesson
These criteria need to be the main focus of the feedback given to students

 Elements of the Course
Assessment Process

1. Formulate statements of intended learning outcomes

Formulate learning goals and learning outcomes

2. Develop or select assessment measures

Direct assessments of student learning: projects, products,
papers/theses, exhibitions, performances, case studies, clinical
evaluations, portfolios, interviews, oral exams
These assessment activities, assigned by the instructor,
yield comprehensive information for analyzing, discussing,
and judging a learners performance of desired abilities and
skills
Indirect assessment of student learning: surveys distributed to
students

 Elements of the Course
Assessment Process

3. Create experiences leading to outcomes

The question to ask: How will this experience (e.g., service
learning, field work, internship) help students achieve the
intended learning outcome(s) of the course?

4. Discuss and use assessment results to improve learning

Effective feedback (Discussions between instructor and
students)

 One Example of An Incomplete Assessment Process

Art History - Survey II

A. The student will identify vocabulary, media, and general theories related
to the history of art from the 14th century through present day. Evaluation:
written assignments, including research papers, and written exams.
B. The student will distinguish and classify works of art and architecture
within the context of the individual, society, time, place and circumstance
within the time frame covered in this course. Evaluation: written assignments,
including research papers, museum/gallery visits and written exams.
C. The student will describe the material, cultural and conceptual conditions
involved in making and using works of art and architecture. Evaluation:
written assignments, including research papers, museum/gallery visits and
written exams.
D. The student will interpret works of art and architecture by synthesizing
formal analysis with scholarly research. Evaluation: research papers, exhibit
and/or resource critique.

http://www.accd.edu/sac/vat/arthistory/arts1304/syllabus.htm

 Overview

What is Assessment?
Learner-centered assessment in higher education - Recent
developments
Our focus today: Course assessment and its intended learning and
achievement outcomes
Course assessment
Classroom assessment techniques
Learning assessment (Assessing students ability to think
critically and solve problems)
Using rubrics to provide feedback to students
Reflecting on ones teaching

 Classroom Assessment Techniques

What is classroom assessment? Angelo, Th. A. & K. P. Cross (1993) Classroom

Systematic collection and analysis of Assessment Techniques. San Francisco:
Jossey-Bass (2nd ed.)
information to improve educational
practice
Method for understanding student
learning
Based on the belief that the more
you know about what your students
know and how they learn, the better
you can plan your learning activities
and structure your teaching

 Benefits of Classroom Assessment

Serves as an ongoing communication process between you and your

students over the entire semester

Helps clarify your teaching goals and what you want your students to
learn, as you progress through the course content

Provides credible evidence regarding whether or not learning

objectives have been achieved

Provides specific feedback on what is working and what is not

working

Provides increased understanding about student learning in your

classroom, allow to adapt your teaching as the course progresses

 Three Examples of Very Simple Classroom

Assessment Techniques

1. One Minute paper: Provides a quick and extremely simple way to collect written
feedback on student learning.
The instructor stops class two or three minutes early and asks
students to respond briefly to some variation on the following two
questions: "What was the most important thing you learned during
this class?" and "What important question remains unanswered?"
Students write their responses on index cards or half-sheets of scrap
paper and hand them in.
Instructor cumulates answers and provides feedback at the start of
the next class

 Three Examples of Very Simple Classroom
Assessment Techniques

2. Muddiest Point: Remarkably efficient, since it provides a high

information return for a very low investment of time and energy.
The technique consists of asking students to jot down a quick response
to one question: "What was the muddiest point in ........?" The focus of
the Muddiest Point assessment might be a lecture, a discussion, a
homework assignment, a play, or a film. Instructor cumulates answers
and provides feedback during the next class

3. One sentence summary: This simple technique challenges students to

answer the questions "Who does what to whom, when, where, how, and
why?" about a given topic, and then to synthesize those answers into a
simple informative, grammatical, and long summary sentence.

(All three techniques provide useful cumulated information if you have a course wiki or
blog)

 Examples of Classroom Assessment Techniques (CAT)
(Angelo & Cross)

From the National Teaching and Learning Forum

http://www.ntlf.com/html/lib/bib/assess.htm

From Southern Illinois University Edwardsville

http://www.siue.edu/~deder/assess/cats/tchgoals.html

From Honolulu Community College.

http://honolulu.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip
/assess-2.htm

From the Field-Tested Learning Assessment Guide (Science, Technology, Engineering,

and Math)
http://www.flaguide.org/cat/cat.php

From Robert L. Harrold (Assessing Problem-Solving Skills)

http://www.ndsu.nodak.edu/ndsu/marmcdon/assessment/assessment_techniques/pr
oblem_solving_skills.htm

 Overview

What is Assessment?
Learner-centered assessment in higher education - Recent
developments
Our focus today: Course assessment and its intended learning and
achievement outcomes
Course assessment
Classroom assessment techniques
Learning assessment (Assessing students ability to
think critically and solve problems)
Using rubrics to provide feedback to students
Assessing students ability to think critically and solve problems
Reflecting on ones teaching

 In Assessment of Learning ,
What, Exactly, Do We Want to Assess ?

The answer: For most courses: Critical thinking and problem

solving

Ask yourself:

Are your students familiar with the current problems that experts in
your discipline are trying to solve?

How do you involve students in trying to solve them?

How do you help students develop skills in critical thinking?

 Essential Components of Critical Thinking and
Problem-solving

From the perspective of cognitive psychologists three types of knowledge

interact in the process of thinking critically and solving ill-defined
problems:
Declarative knowledge: knowing the facts and concepts in the discipline
Procedural knowledge: knowing how to reason, inquire, and present
knowledge in the discipline
Metacognition: cognitive control strategies such as setting goals,
determining when additional information is needed, and assessing the
fruitfulness of a line of inquiry (p. iv).

Kurfiss, J. G. (1988). Critical thinking: Theory, research, practice, and possibilities.

(ASHE-ERIC Higher Education Report No. 2). College Station, TX: Association for the
Study of Higher Education.

 How Do We Know What They Do Know?

Standardized tests (summative)

Alternative assessments (formative)

 Summative and Formative Assessment

Summative Assessment Formative Assessment

Is carried out at intervals when Informal: carried out frequently and
achievement has to be is planned at the same time as
summarized and reported teaching
Looks at past achievements Provides interactive and timely
feedback and response: which
Adds procedures or tests to leads to students recognizing the
existing work (learning) gap and closing it (it is
Involves only grading and forward-looking)
feedback of grades to students In addition to feedback, includes self-
Is separated from the act of monitoring
teaching Fosters life-long learning: It is
empirically argued that it has the
Certifies achievement greatest impact on learning and
achievement

 Summative Assessment:
Standardized Tests

Administered and scored in a standard Are not prescriptive

manner.
Give capsulated view of a students
Designed in such a way that the
questions, conditions for administering,
learning
scoring procedures, and interpretations Used in conjunction with
are consistent performance-based assessment
Examples:
Multiple-choice and true-false questions
(can be tested inexpensively and quickly
by scoring special answer sheets by
computer or via computer-adaptive Popham, J. (1999). Why standardized
testing. ) tests dont measure educational
quality. Educational Leadership,
Short-answer or essay writing components 56(6), 8-15.
that are assigned a score by
independent evaluators.
Can be graded by evaluators who use
rubrics [rules or guidelines] and anchor
papers [examples of papers for each
possible score] to determine the grade
to be given to a response.

 What Does the Research on
Formative Assessment Tell Us?

All students can succeed with appropriate guidance

Learners perceptions and beliefs about their capacity to learn
affects their achievement
Development of self-assessment is vital
Need to move from evaluation to assessment

Therefore
Consider separating feedback from grading
Focus on learning rather than just summative assessment
Encourage reflective assessment with peers

 Examples of Formative Assessment

A selective list
Paper/thesis; written composition
Project (including group projects [collaborative learning])
Experiment
Development of a product
Performance
Community-based experience (service learning)
Exhibition
Case study / Critical incident
Clinical evaluation
Oral exam or presentation
Interview
Comprehensive exam
Portfolio

 Questions to Ask When Developing an Effective
Assessment Task (Huba & Freed, Fig. 7-12)

1. What declarative knowledge [knowing the facts and concepts in

the discipline] do I expect students to draw upon in this task?

2. What procedural knowledge [knowing how to reason, inquire,

and present knowledge in the discipline] do I expect students to
use?

3. What metacognitive knowledge [e.g., setting goals, determining

when additional information is needed, and assessing the
fruitfulness of a line of inquiry] do I expect student to develop
and reveal?

 Questions to Ask When Developing an Effective
Assessment Task (Huba & Freed, Fig. 7-12)

4. In what real-life settings do individuals use the knowledge that I

identified and what ill-defined problems do they typically address?
5. For each ill-defined problem, what task(s) could I sketch out for students
to complete?
6. Which task best exemplifies the characteristics of an exemplary
assessment task (See previous slide)?
7. Which assessment format will work best for this task?
8. What criteria should my students and I use in shaping and critiquing
student work?
9. In view of 8, and if necessary, how can I improve the task so as to reflect
more clearly the characteristics of an exemplary assessment task?

 Characteristics of an Exemplary Assessment Task
(Huba & Freed, Fig. 7-11)

Valid Yields useful information to guide learning

Coherent Is structured so that activities lead to desired
performance product
Authentic Addresses ill-defined problems/issues that are
enduring or emerging
Rigorous Requires use of declarative, procedural, and
metacognitive knowledge
Engaging Provokes student interest and persistence
Challenging Provokes, as well as evaluates, student learning
Respectful Allows students to reveal their uniqueness as
learners
Responsive Provides feedback to students leading to improvement

 Teaching in the Context of Assessment
Depends on

Providing effective feedback to students

Encouraging students active involvement in their own learning
Adjusting teaching to take account of the results of assessment
Recognizing the profound influence of assessment on students
motivation and engagement (both are crucial in learning)
Ensuring that students assess themselves and understand how to
improve

 Learner-Centered Assessment
Implications for Classroom Practice

Clarifying learning outcomes at the course planning stage

Sharing learning goals with students throughout the semester
Asking appropriate and effective questions
Focusing oral and written feedback on the learning outcomes of
lessons and tasks

Encouraging students self-assessment against the learning

outcomes
Organizing individual student target-setting that builds on previous
achievement as well as aiming for the next level up

 Overview

What is Assessment?
Learner-centered assessment in higher education -
Recent developments
Our focus today: Course assessment and its intended
learning and achievement outcomes
Course assessment
Learning assessment
Classroom assessment techniques
Using rubrics to provide feedback to students
Reflecting on ones teaching

 Using Rubrics to Provide
Feedback to Students

Rubric defined:
an authoritative rule an explanation or introductory commentary.
(Webster)

As applied to assessment of student work:

[a rubric] explains to students the criteria against which their work will be
judged (the scoring rules).
It makes public key criteria that students can use in developing,
revising, and judging their own work

Elements of a good rubric

Levels of mastery
Dimensions of quality
Organizational groupings
Commentaries

 Developing Useful Rubrics for
Specific Assessments

Question Action
-What criteria or essential elements must -Include these as rows in your rubric
be present in the students work to
ensure that it is high in quality?
-How many levels of achievement do I -Include these as columns in your rubric and
wish to illustrate for students? label them
For each criterion or essential element of -Include descriptions in the appropriate cells of
quality, what is a clear description of the rubric
performance at each achievement
level?
-What are the consequences of performing -Add descriptions of consequences to the
at each level of quality? commentaries in the rubric
-What rating scheme will I use in the -Add this to the rubric in a way that fits in with
rubric? your grading philosophy
-When I use the rubric, what aspects work
well and what aspects need -Revise the rubric accordingly
improvement?

 Task-Related Rubrics for
Specific Assessments
In addition to rubrics for high achievement, the following need to be asked
as well:

Question Action
What content must students Develop criteria that reflect
master in order to complete the knowledge and/or use of
task well? content and add them to the
rubric
Are there any important aspects Identify skills and abilities that
of the task that are specific to are necessary in this context
the context in which the and add related criteria to the
assessment is set? rubric
In the task, is the process of Include and describe criteria
achieving the outcome as that reflect important aspects of
important as the outcome the process
itself?

 In Addition to Task-Related Rubrics: Teamwork
Rubric

Expectations of group members

Participation of group members

Level of involvement as team member

Quality of work as team member

 Example of Team Rubrics
(George Lucas Educational Foundation)
http://edutopia.org/teachingmodules/Assessment/rubrics.php

Team Rubrics 1 2 3 4

Cooperative Will not help -- Sometimes willing to Shares work when Willingly explains
ignores partner help partner asked and listens to things to partner and
partner will use partners
ideas
Creative Never thinks of other Occasionally has a Has new ideas but Develops new ideas
ideas to solve a new idea, but little will not share with or ways of doing
problem follow through others things. Products
exceed requirements
On Task Consistently talking Sometimes talks Usually follows the Always follows the
to others in room, about unrelated task and talks only to steps of the task and
rarely works on task subjects partner sometimes goes
beyond the concepts
Prepared Never has supplies or Looks through to Uses daily wrap-up to Arrives early for class
willing to find proper task to find place and find place in task and supplies are
place in task sometimes borrows ready
supplies
Skillful Makes no effort to Satisfies with Has general idea of Has clear idea of task
learn new skills answering questions, task. Able to answer and its relationship
but no real specific questions. to technology and
understanding education

 Examples and Information
about Rubrics
Samples of rubrics (Huba & Freed - Figures 6-1, 6-2, 6-3, 6-12):
Rubric for formal oral communication (graduate Program
Rubric for engine design project
Rubric for economic bill writing project
Problem-Solving rubric

Examples of rubrics: French 351: Early Modern Cultures (Danielle Mihram, Fall 2007) -
Handouts (2).

The Rubricator (free membership for faculty [but reads the terms])
http://www.rcampus.com/indexrubric.cfm
Rubrics by Subject
http://www.rcampus.com/rubricshellc.cfm?mode=gallery&sms=publicrub&

Kathy Schrocks guide for Educators: Assessment and rubric information

http://school.discovery.com/schrockguide/assess.html

Rubrics
http://www.shambles.net/pages/staff/rubrics/

 Ongoing Feedback:
Quality Feedback Should

Focus on the learning outcome of the task

Occur as the students are learning

Provide information on how and why the student understands or

misunderstands

Provides strategies to help the student to improve

Assist the student to understand the goals of the learning

 Effective Feedback Should

Be specific - both positive and constructively critical

Be descriptive rather than evaluative
Be offered as soon as possible after the event
Offer alternatives or ask the learner to do so
Look forward to the specific next steps to improve performance
Encourage and plan for opportunities for the feedback to be used as
soon as possible
Involve the learner wherever possible, to improve the chance of
feedback being understood and acted upon

 Training Students
for Self-Assessment

In addition to providing rubrics for assessment

Provide opportunities for self and peer assessment in each unit of

work
Create a supportive environment where students are willing to share
and discuss features on their work in pairs groups and with the
whole class

 Developing a Supportive
Classroom Environment

Share models of work before the students begin a learning task to

give them a clear idea of expectations

Develop success criteria with students

Reserve time periodically to discuss and reflect on the shared

learning intentions in relation to the learning success criteria

 Encouraging Active and Intentional Learning:
From a Teaching to a Learning Environment

For the student For the Instructor

Takes more responsibility for their Lets go of his/her total control
learning of the students learning
Becomes better at sharing
Works independently without learning goals and success
continually relying on instructors
criteria
direction
Focuses on providing feedback
Looks at success criteria and talk
to students and looking
about how and why they have met ahead techniques
them Spends less time recording
assessment data by taking into
account the students self and
peer assessments

 Overview

What is Assessment?
Learner-centered assessment in higher education - Recent
developments
Our focus today: Course assessment and its intended learning and
achievement outcomes
Course assessment
Classroom assessment techniques
Learning assessment
Using rubrics to provide feedback to students
Reflecting on ones teaching

 Reflecting on Ones Teaching

Students end of semester evaluation

Informal mid-term formative evaluation

A formative final evaluation

Administrative evaluation

 Students End of Semester Evaluations

Mostly summative, not a formative process.

In some cases, completed online (much too generic).
Too late for re-aligning the pace and the tasks.

Usually are student satisfaction responses rather than formative

assessment of ones learning and achievement outcomes.
Used for promotion and tenure decisions; do not really tap into the
spirit and soul of the course dynamics and student learning

 At the Very least - Informal Mid-term
Formative Evaluation

Usually takes place between the 4th and the 6th week.

Allows instructor to gather in writing information that can help in

making immediate adjustments or corrections

Benefits:
A perceptible improvement in the climate of your class
A strengthening of the ties between you and your students
An increase in the participation and motivation of the students
An improvement in the organization of the course (revised
timetable, clearer instructions, improved rubrics .)
An improvement in your teaching techniques

 For Personal Development and Improvement
A Formative Final Evaluation:
A Good Way to Reflect about Your Teaching

Prepared by instructor (not to be confused with teaching evaluations distributed at the

end of the semester for administrative, summative purposes)
Usually distributed a week before the end of the semester.
Aspects most often evaluated are:
Competence of the professor
Structure of the course
Professors communication skills
Professor-student relations
Means of assessing students
Teaching materials
Laboratory or discussion sessions
General characteristics of the professor and the class

Pregent, Richard (2000). Charting your course: How to teach more effectively. Madison, Wisc.:
Atwood (Fig. 9.2.1)

 A Formative Final Evaluation:
Another Option:
Distribute a Student End-of-Semester Self-evaluation

Ask each student to respond in writing to the following questions:

Has your approach to [course field/discipline] changed during this course or

compared to previous courses? If yes, how?
Have your attitudes or understanding about [course field/discipline]
changed? If yes, how?
How do think that you performed in this course?
What would you do differently if you had a chance to do this all over again?
Describe the aspects of the course that you found most beneficial to your
learning? Please be specific.
What specific suggestions do you have for improving the course so as to
facilitate or improve your learning? Please be specific.
What else would you like to add?

 Benefits of Student

End-of-semester Self-evaluation

An analysis of the students comments about their learning and expectations

provides:
An opportunity to compare such comments with the courses goals and
learning objectives
A clearer understanding of the diversity of learning styles and of student
expectations
An opportunity for improvement in the organization of the course
(review of courses pre-requisites, revised timetable to improve the
pacing of the contents, improved rubrics, review of assessment tasks .)
An opportunity for improvement in your teaching techniques
Together with what the students have done in the course (assessment
tasks) contributes to the culture of evidence of student learning

 Teaching Evaluations for
Administrative Purposes

Distributed at the end of the term

In some cases, completed online (generic in approach)

Are mostly summative, to be used for promotion and tenure decisions

Do not provide formative assessment of teaching and learning

Reflect students satisfaction with the course rather than assessment of
their learning and their achievements

 Resources for Assessment

Teaching and Learning Resources on the website of

the USC Center for Excellence in Teaching:
http://www.usc.edu/programs/cet/resources/

Assessment of Teaching & Learning

http://www.usc.edu/programs/cet/resources/assessment/

Bibliography on Assessment - See Handout

 Review

What is Assessment?
Our focus today: Course assessment and its intended
learning and achievement outcomes
Course assessment
Classroom assessment techniques
Learning assessment (Assessing students ability to
think critically and solve problems)
Using rubrics to provide feedback to students
Reflecting on ones teaching