EDUC.

105 – ASSESSMENT IN LEARNING 2

(BEED – 2A)
Reporter: LORENA L. CALUNSAG Topic: High Quality Assessment in Retrospect:
Adequate Sampling

Scoring Rubrics:
___________ Delivery/Presentation of the Report (15 pts)
___________ Report Materials (5 pts)
___________ Personality (5 pts)
___________ Handout (5 pts)
___________ Total (30 pts)
 EDUC. 105 – ASSESSMENT IN LEARNING 2
(BEED – 2A)

High Quality Assessment in Retrospect

 High quality assessment is balanced to provide instructors with ongoing feedback about
student progress.
 In particular, data gathered from assessments given throughout the learning process give
educators the information they need to adjust their instruction.
 Provides actionable information to inform curriculum and instruction decisions and allows
for a real- time change of course in meeting student’s needs.
 High Quality Assessment= it provides a result that demonstrates and improve targeted
students learning.
Census vs. Sampling Explanation
 For programs that are small, assessing the entire population may yield a more accurate
measure of student learning. Assessing the entire population is called a census whereas
assessing only part of the population is called a sample.
 Example of Using a Census: Whole populations should be considered for the assessment
and evaluation of an SLO when the population is rather small. For example, some degree
programs only have 5 or 10 students. Those programs could and should include all the
students in the assessment and evaluation of an SLO.
 Example of Using a Sample: It is difficult and time‐consuming to evaluate all students’
assignments for an SLO if there are over 100 students in a program, especially when the
artifacts are long and complicated such as a 20‐page research paper
Sampling for the Assessment of Student Learning Outcomes
 For Student Learning Outcomes (SLO’s), we must assess artifacts that reflect the course
desired outcomes.
 Sampling may be useful when assessing artifacts that take a long time to review.
 Each item should reflect a desired learning outcome and the test as a whole should also
cover all important outcomes.
 Relevance and adequate sampling will provide the basis for determining content validity.
Content Validity- refers to how the test covers a representative sample of the behavior
domain to be measured.
 Can the test perform its intended function?
 Is it aligned to the intended behavior domain to be measured?
 EDUC. 105 – ASSESSMENT IN LEARNING 2
(BEED – 2A)

Sampling in assessment: Why do we sample?

 Rather than assessing an entire group of students (a census), a sample assesses a subset
of a particular population. A sample is often used when assessing an entire group of
students is overly difficult or time consuming.
 Sampling facilitates the assessment process when programs have large numbers of
students and when programs have artifacts that take a long time to review.
 Sampling facilitates the assessment process when programs have large numbers of
students and it is not feasible to assess all students. Furthermore, sampling may be useful
when assessing artifacts that take a long time to review. For the Student Learning
Outcomes (SLO), we must assess artifacts that reflect the desired outcomes. You may use
a subjective artifact from a course (e.g. the research project in the capstone course or a
paper) for a SLO measure only if you have a faculty panel evaluate the artifacts
independent of the course instructor. There might be too many students or too many
artifacts for a panel to evaluate each semester or year; therefore, only some of the artifacts
would be effectively and efficiently evaluated each year. The portion evaluated is the
sample of the entire population.

Example:
 #1: In a program with roughly 60 seniors on each of two campuses, faculty assess student
poster presentations for a random sample of 36 seniors on each campus using a rubric for
two program SLOs.
 #2: It is difficult and time‐consuming to evaluate all students’ assignments for an SLO if
there are over 100 students in a program, especially when the artifacts are long and
complicated such as a 20‐page research paper.
 EDUC. 105 – ASSESSMENT IN LEARNING 2
(BEED – 2A)

Sampling Procedures
 Before evaluating artifacts or data for the SLO, you must:

1. Decide whether you will use a sample or the whole population.

2. Choose an appropriate sample size based on percentage, artifact size and

complexity, and faculty panels.

3. Choose an appropriate sampling method.

Determining Sample Size

 Whether or not to sample and the size of the sample depend on three factors, all of which
must be kept in mind when making sampling decisions:

1. The length and complexity of the assignments/artifacts.

2. The number of students enrolled in the program.

3. The number of faculty members serving on the faculty panel (the artifact

evaluators)

Choosing a Sample Based on a Percentage Explanation

 If you have a large program (over 100 students), you may not have the people and time to
evaluate 100 artifacts. Therefore, you would choose a specific percentage of students or
artifacts (e.g. 20%).
 Example: Programs with over 100 students and only 2 faculty evaluators may choose to
evaluate and report on a sample that represents 10% of the population (10 students) or
20% (20 students).
 Caution: Your percentage should be based on the three factors mentioned above. • The
length and complexity of the assignments o If the assignment or artifact is over 50 pages
(e.g. class action research project), then a smaller percentage of students should be
chosen for the sample. Since the university standard is 10% or 10 students, whichever is
greater, then you would sample 10% of your population (10% of 100 students = a sample
size of 10 students).
 EDUC. 105 – ASSESSMENT IN LEARNING 2
(BEED – 2A)

 The number of students enrolled in the program o If the program has less than 50 students,
then you should consider using a larger percentage or the entire population. According to
the university sampling policy, you would choose 10 students minimum.
 The number of faculty members serving on the faculty panel o If the program only has three
faculty members on the faculty panel, then a smaller sample size would be more
appropriate depending on the complexity of the assignment. However, programs with over
10 faculty members and short assignments could have a much larger sample size since
there are many more people available to evaluate the artifacts.

Choosing a Sample Based on Length and Complexity of the Assignment/Artifact

 Explanation: Programs that have a very long, complex artifact should use a smaller
sample size. Programs that have a short, simple artifact should use a larger sample size.
 Example #1: A large program with 200 students and a 50‐page artifact (a research paper)
for its SLO would have a sample size of 5% of the population (total number of students
enrolled in the program), which would yield ten 50‐page artifacts. It would be much more
feasible for a faculty panel of 2‐3 members to assess ten 50‐page artifacts than for them to
assess two‐hundred 50‐page artifacts.
 Example #2: A large program with 200 students and a 5‐page artifact (an analytical paper)
for its SLO should have a sample size of 10% of the population (total number of students
enrolled in the program), which would yield 20 5‐page artifacts. It would be feasible for a
faculty panel of 2‐3 members to assess 20 5‐page artifacts.

Sample Based on the Size of the Faculty Panel

 Explanation: The number of faculty raters on the faculty panel affects the number of
artifacts that can be evaluated each semester or year. Programs with a small faculty
panel (4 members or less) should evaluate a smaller number of artifacts. Programs with
a large faculty panel (5 or more faculty members) have the resources to evaluate a larger
number of artifacts.
 Example #1: A large program with 200 students and a 50‐page artifact (e.g. the research
project) for its SLO should have a small sample of 5% (10 students) if the faculty panel
only has 3 members.
 Example #2: A large program with 200 students and a short, oral artifact (e.g. 10‐minute
presentation) and 3 faculty panel members may have a large sample of 20% (20 students
 EDUC. 105 – ASSESSMENT IN LEARNING 2
(BEED – 2A)

Common Types of Sampling

 There are a variety of sampling methods. Simple random, stratified, systemic, and
cluster sampling are examples of four common and appropriate sampling methods for
institutional assessment activities.

1.Simple Random Sampling Explanation:

Selecting
from a
hat.

 Randomly select a certain number of students or artifacts.

 Example: You have 100 students in your program who have completed the mandatory
artifact. You want to sample 20% of your artifacts (20 artifacts). Therefore, you
randomly select the 20 students or artifacts without any order or plan. Random
sampling can be done with a random numbers table, by random number generators
(computerized), or by selecting from a hat.

2.Stratified Sampling Explanation:

 EDUC. 105 – ASSESSMENT IN LEARNING 2
(BEED – 2A)

 Students are sorted into homogenous groups and then a random sample is selected
from each group. This is useful when there are groups that may be underrepresented.
 Example: In a program that has few female students, it may be desirable to ensure they
are represented in the sample. Therefore, all students are sorted by gender and a
sample is selected from each group.

3.Systemic Sampling Explanation:

 You select the nth (e.g. 7th, 9th, 20th) student or artifact from a list.
 Example: You have an alphabetical listing of all 100 students who have just completed
your program. You want to sample 20% of your student population (20
students). Therefore, you go through the list of 100 students and pick every 5th student as
you move down the list.

4.Cluster Sampling Explanation:

 EDUC. 105 – ASSESSMENT IN LEARNING 2
(BEED – 2A)

 You randomly select clusters or groups (e.g. classes or sections), and you evaluate the
assignments of all the students in those randomly selected clusters or groups.
Example: The artifact that represents an SLO in your program is a paper that is produced
in the highest-level course of your program, and there are 15 sections offered a
semester. Each section has 30 ‐ 5 ‐ students. You would like your sample to be 20% (90
students) of your overall student population (450 students) for one semester. Therefore,
you randomly select 3 sections, and you evaluate the assignments of all 90 students
enrolled in those 3 sections.

Key Points

 High Quality Assessment= it provides a result that demonstrates and improve

targeted students learning.
 Relevance and adequate sampling will provide the basis for determining content
validity.
 Sample size considerations: number of students, artifacts and artifacts evaluator.
 Common Types of Sampling:

1. Simple Random Sampling -Randomly select a certain number of students

Or artifacts.

2. Stratified Sampling -Students are sorted into homogenous groups

and then a random sample is selected from each group.

3.Systemic Sampling: You select the nth (e.g. 7th, 9th, 20th) student

or artifact from a list.

4. Cluster Sampling: You randomly select clusters or groups (e.g. classes or

sections), and you evaluate the assignments of all the

students in those randomly selected clusters or groups.