UNIT-IV (205)

Planning, construction, implementation and reporting of assessment

Construction procedure of a test:

Constructing a test involves a systematic process to ensure that the test is valid, reliable, and
measures what it intends to measure. Here are the key steps involved in the construction
procedure of a test:
1. Define the Purpose and Objectives: Clearly define the purpose of the test and the specific
objectives it aims to assess. Determine what knowledge, skills, or abilities the test should
measure.
2. Select the Test Type and Format: Choose the appropriate test type and format based on
the objectives. Common test types include multiple-choice, true/false, essay, short answer,
and performance-based assessments.
3. Develop Test Specifications: Create test specifications that outline the content areas to be
covered, the number of questions in each section, the time limit, and any other relevant
details.
4. Write Test Items: Develop test items (questions) that align with the defined objectives. For
multiple-choice questions, ensure that each question has a clear stem and plausible
distracters. For essay questions, provide clear instructions and grading criteria.
5. Review and Revise Items: Have subject matter experts and other stakeholders review the
test items for clarity, relevance, and fairness. Revise items based on feedback to improve their
quality.
6. Pilot Testing: Administer the test to a small sample of the target population to identify any
issues with the test items, such as ambiguous questions or problems with instructions.
7. Item Analysis: Analyze the performance of the pilot test participants on each item to assess
item difficulty, discrimination, and reliability. Identify items that may need to be revised or
removed.
8. Establish Scoring Scheme: Determine how each item will be scored, whether it's a simple
right/wrong scoring or a more complex rubric-based evaluation for open-ended questions.
9. Administer the Test: Administer the final version of the test to the intended population
under standardized conditions to ensure fairness and consistency.
10. Score the Test: Apply the established scoring scheme to each participant's responses to
obtain test scores.
11. Conduct Statistical Analysis: Conduct various statistical analyses, such as item analysis,
reliability analysis, and validity analysis, to evaluate the quality of the test and its individual
items.
12. Evaluate Test Results: Interpret the test results and use them to make educational or
decision-making decisions, such as student placement, program evaluation, or employee
selection.
13. Continuous Improvement: Regularly review and update the test to ensure its relevance
and effectiveness. Gather feedback from test-takers and stakeholders to identify areas for
improvement.
Throughout the construction procedure, it is essential to consider ethical considerations, such as
ensuring the test is free from bias and discrimination. Additionally, the test should adhere to
relevant testing standards and guidelines to ensure its fairness and validity.

Guidelines for construction of test items:

Constructing test items that are valid, reliable, and fair requires careful consideration and
adherence to specific guidelines. The essential guidelines for constructing test items are:

1. Clearly Define Learning Objectives: Ensure that each test item aligns with specific
learning objectives. Clearly define what knowledge, skills, or abilities the item is intended to
assess.
2. Use Clear and Concise Language: Write test items in clear and straightforward language
that is appropriate for the target audience. Avoid ambiguous or confusing wording.
3. Avoid Negative Wording: When using multiple-choice questions, avoid double negatives
or negative wording that can lead to confusion.
4. Avoid Leading or Biasing Language: Ensure that test items do not include language that
leads test-takers to a particular answer or exhibits bias towards any group.
5. Write Plausible Distracters: For multiple-choice questions, include distractors that are
plausible and relevant to the question. This helps to differentiate between students who
understand the content and those who do not.
6. Ensure Mutually Exclusive Options: In multiple-choice questions, make sure that each
option is mutually exclusive, meaning that only one option can be correct.
7. Balance the Length of Options: In multiple-choice questions, ensure that the correct
answer is not always the longest or shortest option. Vary the lengths of the options to avoid
cues.
8. Avoid Tricky Questions: Construct items that assess genuine understanding rather than
trying to trick or confuse test-takers.
9. Provide Sufficient Context: For open-ended questions or essay questions, provide enough
context and instructions to guide the test-takers' responses.
10. Consider Appropriate Difficulty: Ensure that the difficulty level of the items matches the
ability level of the target population. Avoid items that are too easy or too difficult for the
intended audience.
11. Balance Content Coverage: Ensure that the test items represent a fair and balanced
coverage of the content areas being assessed.
12. Pilot Test Items: Before finalizing the test, pilot test the items with a small group to identify
any issues with clarity, difficulty, or bias.
13. Ensure Consistent Formatting: Maintain a consistent format throughout the test for ease
of readability and comprehension.
14. Use Real-Life Scenarios: Whenever possible, use real-life scenarios or authentic tasks in
test items to assess practical application of knowledge and skills.
15. Consider Time Constraints: Make sure that the test items can be completed within the
allocated time limit.
16. Avoid Guessing Clues: Eliminate clues that may unintentionally reveal the correct answer
to test-takers.
17. Ensure Test Security: Take measures to ensure the confidentiality and security of the test
items to prevent cheating or leakage.
18. Revise and Review: Review the test items multiple times to identify and correct any errors,
inconsistencies, or improvements.

By following these guidelines, test constructors can create assessment items that accurately
measure the desired learning outcomes and provide reliable and valid results. Regular review and
refinement of the test items based on feedback and data analysis can further enhance the quality
and effectiveness of the assessment.

Item analysis procedure:

Item analysis is a statistical procedure used to evaluate the quality of individual test items
(questions) in an assessment. It helps to identify items that are too easy, too difficult, or do not
effectively discriminate between high-performing and low-performing test-takers. The item
analysis procedure typically involves the following steps:

1. Administer the Test: Administer the test to the intended population under standardized
conditions, ensuring that all test-takers follow the same instructions and time constraints.
2. Collect Responses: Collect the responses from all test-takers for each individual test item.
3. Score the Test: Score the test according to the established scoring scheme for each item.
4. Create a Response Table: Construct a response table for each test item, showing the
number of test-takers who chose each response option (for multiple-choice questions) or the
distribution of scores for open-ended questions.
5. Calculate Item Difficulty: Calculate the item difficulty for each test item. Item difficulty is
the proportion of test-takers who answered the item correctly. It is calculated by dividing the
number of correct responses by the total number of responses for the item.

Item Difficulty = (Number of Correct Responses) / (Total Number of Responses)

6. Calculate Item Discrimination: Calculate the item discrimination for each test item. Item
discrimination measures how well the item distinguishes between high-performing and low-
performing test-takers. It is commonly computed using the point-biserial correlation
coefficient for dichotomous items (e.g., true/false or multiple-choice) and the correlation with
the total test score for other item types.
7. Identify Poorly Performing Items: Items with very high or very low item difficulty (close
to 1 or 0) may not effectively differentiate between test-takers and are considered poorly
performing. Similarly, items with low item discrimination values are also problematic and
may need to be reviewed or revised.
8. Review and Revise Items: Based on the item analysis results, review the poorly performing
items and consider revising or eliminating them. Items with low item discrimination or
difficulty outside the desired range should be carefully examined for potential flaws.
9. Retest or Retain Items: After making revisions, if necessary, consider retesting the items
in future assessments to assess their improved performance. Alternatively, if items have
strong psychometric properties, they can be retained for future assessments.
10. Interpret Results: Analyze the item analysis data to gain insights into the overall quality of
the test and the individual items. Use the results to improve the test's reliability and validity.
11. Continuous Improvement: Regularly conduct item analysis to identify opportunities for
test improvement and refinement. Continuous monitoring and evaluation of test items
contribute to the ongoing enhancement of the assessment's effectiveness.

Item analysis is an essential component of test construction and helps ensure that the assessment
accurately measures the intended learning outcomes and provides valid and reliable results. It
aids in the identification of problematic items and contributes to the overall improvement of the
test's quality and fairness.

Scoring procedures- manual and electronic:

Scoring procedures refer to the methods used to evaluate and assign scores to test responses.
Scoring can be done manually by human raters or electronically using automated scoring systems.
Each method has its advantages and considerations:

Manual Scoring:
1. Subjectivity and Bias: Manual scoring involves human judgment, which can introduce
subjectivity and bias. Raters may interpret responses differently, leading to variability in
scores.
2. Open-ended Questions: Manual scoring is commonly used for open-ended questions, such
as essays, where the responses require qualitative evaluation and detailed feedback.
3. Scoring Rubrics: To enhance consistency and reduce subjectivity, scoring rubrics are often
used in manual scoring. Rubrics provide clear criteria and guidelines for evaluating responses.
4. Time-Consuming: Manual scoring can be time-consuming, especially for large-scale
assessments or when numerous open-ended questions are involved.
5. Personalized Feedback: Manual scoring allows for personalized feedback, which can be
valuable for educational purposes and improving student performance.
6. Expertise Required: Skilled and trained raters are essential for accurate and reliable
manual scoring.

Electronic Scoring:
1. Efficiency: Electronic scoring is much faster and more efficient than manual scoring.
Automated systems can process large volumes of responses rapidly.
2. Objectivity: Electronic scoring eliminates human subjectivity and bias, ensuring consistent
and fair evaluation of responses.
3. Multiple-Choice and Objective Items: Electronic scoring is commonly used for multiple-
choice and objective items, as the responses are easily quantifiable.
4. Reliability: Automated scoring systems provide consistent and reliable results, reducing
variability in scores.
5. Scalability: Electronic scoring is highly scalable, making it suitable for large-scale
assessments, such as standardized tests.
6. Immediate Results: Electronic scoring allows for quick and immediate score reporting,
providing timely feedback to test-takers.
7. Data Analysis: Electronic scoring generates data that can be analyzed to assess the quality
of individual items and the overall test's performance.
Considerations:
 1. Test Type: The nature of the test and the type of questions (open-ended vs. objective)
influence the choice of scoring method.
2. Resources: Manual scoring requires trained human raters, while electronic scoring requires
access to appropriate technology and automated scoring systems.
3. Validity and Reliability: Both scoring methods should be designed to ensure the validity
and reliability of the assessment.
4. Combining Methods: In some cases, a combination of manual and electronic scoring may
be used, such as using electronic scoring for objective items and manual scoring for open-
ended questions.

Processing test performance- calculation of percentages, central

tendencies, graphical representation:
Processing test performance involves analyzing the data collected from the test to derive meaningful
insights about the test-takers' performance. Different statistical measures and graphical
representations can be used to summarize and interpret the data. Here are some common methods
used for processing test performance based on education:

1. Calculation of Percentages: Percentages are commonly used to represent the proportion of

test-takers who achieved a particular score or level of performance. To calculate percentages, follow
these steps:

• Count the number of test-takers who achieved a specific score or fell within a score range.

• Divide the count by the total number of test-takers.

• Multiply the result by 100 to get the percentage.

Percentages can be used to analyze how many test-takers performed at different levels, such as
passing rates or proficiency levels.

2. Central Tendencies: Central tendencies are measures that provide insights into the average or
typical performance of the test-takers. The three main measures of central tendency are:

• Mean: The mean is the arithmetic average of all the test scores. It is calculated by summing
all the scores and dividing by the total number of test-takers.

• Median: The median is the middle score when all the scores are arranged in ascending or
descending order. It is useful for identifying the typical performance when extreme scores or
outliers are present.

• Mode: The mode is the most frequently occurring score. It provides information about the
most common performance level.

Central tendencies help to understand the overall performance distribution and identify the typical
score achieved by the test-takers.
3. Graphical Representation: Graphical representation provides a visual way to present test
performance data. Common types of graphs used for this purpose include:

• Histograms: Histograms display the distribution of test scores in intervals (bins) and show
the frequency of scores within each interval. They provide a visual representation of the score
distribution.

• Bar Charts: Bar charts are used to compare the performance of different groups or
categories. For example, they can be used to compare the performance of students from
different educational levels.

• Line Graphs: Line graphs can show the trend in test performance over time or across
different test administrations.

• Box Plots: Box plots (box-and-whisker plots) provide a visual summary of the distribution
of scores, including median, quartiles, and outliers.

Graphical representation is useful for identifying patterns, trends, and outliers in the test
performance data, making it easier to communicate the results to stakeholders.

It's important to note that the specific methods used for processing test performance may vary
depending on the nature of the test, the data collected, and the research or educational objectives.
Valid and reliable interpretation of test performance data is essential for making informed decisions
about educational interventions, curriculum improvements, or individual student assessments.

