PYC2606/202/1/2015

Tutorial Letter 202/1/2015

Basic Measurement and Questionnaire Design

PYC2606

Semester 1

Department of Psychology
This tutorial letter contains feedback on Assignment 02 and important
information about the examination

Bar code

Open Rubric
 A bout the examination

How to complete this You may be aware that PYC2606 (Basic Measurement and
module successfully with Questionnaire Design) is one of the modules included in the Alternative
the best possible impact on Assessment Project. We will use an alternative assessment measure
your growth and for this module. Students will sit for a timed take-home assessment,
development as a student?
which means they will be required to complete the assessment online
on a particular date and time. This means that there is no venue based
written examination for this module. You will sit for an online
assessment. The online assessment means you will be required to log-
on online. If you cannot do so from home or from an internet café you
have to reserve a computer at a Unisa regional office. The exact
instructions will be posted on myUnisa two weeks before the
examination.

VERY IMPORTANT

If you experience problems during log-on procedures contact student

assessment administration for assistance and not the lecturers. Contact
details for the student assessment administration will be provided on
myUnisa. You will receive this information two weeks before the online
assessment.

How do I answer the online The duration of the online assessment is 2½ hours. It consists of 50
questions? multiple-choice questions. The paper total is 50 marks. Your mark out
of 50 is automatically converted to a percentage - that is, a mark out of
100. The assessment covers the entire syllabus. Assignment 01
together with Assignment 02 provide a good indication of the nature
and the format of the examination paper you will complete in online
assessment.

When is the examination? The examination date and time will be posted on myUnisa shortly. You
will be required to logon on the date and time of the examination.
Should you experience a problem that prohibits you from completing
the online examination you will be allowed a second opportunity. The
date of the second online examination will be posted on myUnisa
shortly. Please note that the second online examination opportunity is
only available to students who attempted the first opportunity but were
unable to complete the examination session. You cannot sit for both
examination sessions in the hope of obtaining the better of the two
marks.

When do I pass? You pass if you obtain an online assessment mark of at least 40% and

2
 PYC2606/201

a final mark (online assessment mark plus year mark) of at least 50%.
The year mark contributes 20% towards the final mark and the online
assessment mark contributes 80%.

What happens if I fail? If you obtain a final mark of at least 45% you may sit for a
supplementary online assessment. If you fail with a mark below 45%
you have to re-register, and redo the module.

When is the supplementary You have to write the supplementary online assessment at the next
examination? examination period.

3
 A bout the memorandum

In what format is the The memorandum provides feedback on Assignment 02. Answers to the
feedback presented? questions are provided. The key to each question is indicated, that is, the
option that provides the most correct answer.

Why should I study the Assignment 02 covers the second part of the syllabus. Working through
feedback? the discussions on these questions serves as revision of the work you
have studied. Your incorrect answers give an idea of the sections that
you need to spend additional time on when preparing for the examination.

What kind of multiple choice In addition to scenario embedded rank ordered multiple choice questions
questions are used in this (discussed in Tutorial Letter 201), this assignment also uses rating
assignment? questions. Rating questions are a form of multiple choice questions.
Multiple choice questions are questions that consist of a problem
statement and two or more potential responses. Respondents (in these
case students) are required to use the potential responses when they
respond to the problem stated in the question. In the case of rating
questions the potential responses are ratings. Respondents (i.e.
students) are required to select a particular rating. The problems stated in
the rating questions involve characteristics of the scenario, questionnaire
and mini-manual that need to be assessed. Particular criteria are used to
assess these characteristics. Respondents (i.e. students) are required to
apply the criteria to determine a rating. The determined rating is selected
as the answer to the question.

An example of a the kind of Here is the problem statement:

multiple choice question Judge whether the instructions of a questionnaire called AIQ are correct.
used in this assignment
Here are the criteria:
The instructions of the AIQ are correct if:
a the intention of the AIQ is indicated correctly
b the matter of confidentiality is addressed correctly
c general instructions for completing the AIQ are indicated correctly

Here are the potential ratings:

Rate 1 if: none of the following
Rate 2 if: a
Rate 3 if: a + b
Rate 4 if: a + b + c

Here is how to select a rating:

Have a look at the instructions of the AIQ. Change each criterion into a question.
Therefore ask: Is the intention of the AIQ indicated correctly in the instructions? If

4
 PYC2606/201

the answer is ‘yes’, criterion a has been met, and provisionally your rating is 2.
Then ask: Is the matter of confidentiality addressed correctly in the instructions?
If the answer is ‘yes’ criterion b has been met, and now your provisional rating
becomes 3.
Finally, ask: Are the instructions for completing the AIQ indicated correctly? If the
answer is ‘yes’ criterion c has been met and your provisional rating becomes 4,
which would be the rating you finally select.
Another example: If criterion a has been met, but criterion b has not been met
your final rating will be 2.

Why this kind of multiple This kind of multiple choice question tests more than knowledge. It tests
choice question? thinking skills and insight. It requires respondents (i.e. students) to judge
the quality of material they rate. For example, it requires students to
judge the quality of the instructions of a questionnaire.

5
M emorandum for Assignment 02

Question 1 Lebo: I think there is a printing error in the manual. Figure 1 is incomplete with
no proper explanation provided. Do you know what this figure is about?
Bertha: Looking at the numbers I think the figure is a graphical representation
Option 5 is the key of the information provided for Item 2.1 in Table 3
Piet: If your assumption is correct, Bertha, the figure expresses the item-total
Piet, Bertha, Keith correlation for Item 2.1
Keith: No, I think the figure expresses the fact that five people completed each
item in the questionnaire

You agree most with, second most with and least with:
1. Bertha, Keith, Piet
2. Piet, Keith, Bertha
3. Bertha, Piet, Keith
4. Keith, Bertha, Piet
5. Piet, Bertha, Keith

Question 2 Lebo: So, what do the numbers on the X-axis mean?

Piet: I think the numbers on the X-axis represent the possible responses to
Item 2.1
Option 3 is the key Bertha: If the numbers on the X-axis represent the possible responses to Item
2.1 it means five people completed the item
.Keith, Piet, Bertha Keith: If the X-axis represents Item 2.1 I would say it means the numbers 1 to
5 are associated with the scale categories that range from disagree strongly to
agree strongly

You agree most with, second most with and least with:
1. Piet, Keith, Bertha
2. Keith, Bertha, Piet
3. Keith, Piet, Bertha
4. Bertha, Piet, Keith
5. Piet, Bertha, Keith

Question 3 Lebo: And those numbers in the square brackets, [1], [2], etc. What do they
stand for?
Piet: The numbers in the square brackets are discrimination values. They
Option is the key reflect the proportion of individuals who completed the item correctly
Bertha: No, each point indicates the item score and the total score of a
Keith, Bertha, Piet particular individual
Keith: For example, [1] means that participant 1 obtained a total score of 44
and also disagreed strongly with the statement that drinking is relaxing.

You agree most with, second most with and least with:
1. Bertha, Piet, Keith
2. Bertha, Keith, Piet
3. Keith, Bertha, Piet
4. Piet, Bertha, Keith
5. Keith, Piet, Bertha

Question 4 You: Compare Item 2.9 as it is formulated in the questionnaire with the item’s
item-total correlation shown in Figure 2. Do you find this result surprising?
Lebo: The item-total correlation is weak, which is not surprising because it is
Option 2 is the key not clear whether the item reflects a positive or negative attitude towards
alcohol consumption

6
 PYC2606/201

Lebo, Keith, Piet Keith: The item-total correlation is positive and negative, which is not
surprising because the item reflects positive and negative attitudes towards
alcohol consumption
Piet: The item-total correlation is negative, which is not surprising because it is
not clear how the item relates to attitude towards alcohol

You agree most with, second most with and least with:
1. Piet, Lebo, Keith
2. Lebo, Piet, Keith
3. Lebo, Keith, Piet
4. Piet, Keith, Lebo
5. Keith, Piet, Lebo

Question 5 You: Compare Item 2.11 as it is formulated in the questionnaire with the item’s
item-total correlation shown in Figure 3. Do you find this result surprising?
Option 5 is the key Lebo: The item-total correlation is positive, which is not surprising because the
item reflects a clear stance in relation to alcohol consumption
Keith, Piet, Lebo Keith: The item-total correlation is negative, which is not surprising because
. the item requires reversed scoring
Piet: The item-total correlation is negative, which is surprising because the
item reflects a positive attitude against alcohol consumption

You agree most with, second most with and least with:
1. Keith, Lebo, Piet
2. Piet, Keith, Lebo
3. Piet, Lebo, Keith
4. Lebo, Piet, Keith
5. Keith, Piet. Lebo

Question 6 You: Compare Item 2.13 as it is formulated in the questionnaire with the
item’s item-total correlation shown in Figure 4. Do you find this result
surprising?
Option 2 is the key
Piet: The item-total correlation is positive which is not surprising because
Piet, Bertha, Lebo individuals with a high intention to use alcohol should get a high score on the
item
Bertha: The item-total correlation is positive which is surprising because the
item expresses an attitude that promotes alcohol consumption
Lebo: The item-total correlation is weak which is not surprising because the
item requires reversed scoring

You agree most with, second most with and least with:
1. Lebo, Bertha, Piet
2. Piet, Bertha, Lebo
3. Piet, Lebo, Bertha
4. Bertha, Piet, Lebo
5. Bertha, Lebo, Piet

Question 7 You: Compare Item 2.17 as it is formulated in the questionnaire with the item’s
item-total correlation shown in Figure 5. Do you find this result surprising?
Option 5 is the key Bertha: The correlation is negative, which is not surprising because the item
required reversed scoring
Keith, Bertha, Lebo Keith: The correlation is weak, which is not surprising because the item has a
very strong social response bias with low discrimination between individuals
Lebo: The correlation is positive, which is surprising because the item is
expresses a negative value against alcohol consumption

You agree most with, second most with and least with:
1. Bertha, Lebo, Keith
2. Keith, Lebo, Bertha
3. Lebo, Bertha, Keith
4. Bertha, Keith, Lebo
5. Keith, Bertha, Lebo

7
Question 8 You: Given your previous considerations, which of the four items would you
discard and why?
Option 3 is the key Piet: Item 2.9 because of low item discrimination and Item 2.11 due to
negative item discrimination
Keith, Piet, Lebo Lebo: Item 2.11 because of its negative item discrimination and 2.13 because
it has low item variance
Keith: Item 2.9 because of low item discrimination and Item 2.17 due to low
item variance

You agree most with, second most with and least with:
1. Keith, Lebo, Piet
2. Piet, Keith, Lebo
3. Keith, Piet, Lebo
4. Lebo, Piet, Keith
5. Piet, Lebo, Keith

Question 9 You: Given the way in which we want to use the AIQ, how would you
approach the reliability of the questionnaire?
Option 1 is the key Keith: It is important to make sure that the AIQ has high internal consistency
because we need a reliable measure of the ‘intention to use alcohol’ construct
Keith, Bertha, Lebo Bertha: I think test-retest reliability is more important than internal consistency,
given the way in which we want to use the AIQ
Lebo: Because we want to identify learners at risk of illegal drug use it is
important to consider criterion-related reliability as the most important
component of the AIQ’s reliability
You agree most with, second most with and least with:
1. Keith, Bertha, Lebo
2. Lebo, Bertha, Keith
3. Keith, Lebo, Bertha
4. Bertha, Keith, Lebo
5. Bertha, Lebo, Keith

Question 10 Bertha: I presume the same goes for validity. We have to consider our
approach concerning the validity of the AIQ
Lebo: Because our goal is to identify learners who could benefit from drug use
Option 4 is the key prevention programmes I think it is important to consider the concurrent
validity of the AIQ
Keith, Lebo, Piet Keith: I agree, Piet, but we have to make sure the AIQ has construct validity
otherwise the AIQ’s concurrent validity may be suspect
Piet: Face validity is the most important aspect of validity to consider. If the
learners cannot see that the AIQ is about intention to use alcohol they may
think the questionnaire has no value

You agree most with, second most with and least with:
1. Lebo, Keith, Piet
2. Piet, Lebo, Keith
3. Keith, Piet, Lebo
4. Keith, Lebo, Piet
5. Lebo, Piet, Keith

Question 11 You: Considering the information provided in Table 4, what do you think of the
construct validity of the AIQ?
Option 1 is the key Bertha: The high correlation between the Attitude and Norm factors supports
the construct validity of the AIQ
Piet, Bertha, Keith Keith: The low correlation between the Recreation factor and the Attitude and
Norm factors shows low construct validity of the AIQ
Piet: The information in Table 4 supports the construct validity of the AIQ, but
the AIQ does not fully represent the content domain of the construct it purports
to assess

You agree most with, second most with and least with:
1. Piet, Bertha, Keith
2. Bertha, Keith, Piet
3. Piet, Keith, Bertha
4. Keith, Bertha, Piet
5. Bertha, Piet, Keith

8
 PYC2606/201

Question 12 Lebo: I think Table 5 provides further information about the construct validity of
the AIQ?
Keith: Yes, the correlations among the questionnaires show convergent and
Option 4 is the key discriminent validities that support construct validity for the AIQ
Piet: In addition to supporting construct validity for the AIQ the correlation
Piet, Keith, Bertha between the AIQ and the treatment suitability ratings also indicates support for
concurrent validity for the AIQ
Bertha: The table shows convergent validity but it does not support
discriminant validity for the AIQ

You agree most with, second most with and least with:
1. Keith, Piet, Bertha
2. Bertha, Keith, Piet
3. Piet, Bertha, Keith
4. Piet, Keith, Bertha
5. Keith, Bertha, Piet

The following questions are rating questions

IMPORTANT:
Please note that in Assignment 02 the rating scale indicates ‘none of a, b, c’ for Rate 1. It is more correct to
indicate Rate 1 as ‘none of the following’. This was corrected in the scale ratings below. It now says ‘none
of the following’ instead of ‘none of a, b, c’.

Q Rating Notes Criteria Scale

13 1 AIQ instructions only The instructions of the AIQ are correct if: Rate 1 if: none of the following
refer to criterion c. a the intention of the AIQ is indicated Rate 2 if: a
correctly Rate 3 if: a+b
b the matter of confidentiality is Rate 4 if: a+b+c
addressed correctly
c general instructions for completing
the AIQ are indicated correctly
14 2 Not clear – contains Item 2.9 is formulated correctly if: Rate 1 if: none of the following
two inherent issues. a Item 2.9 has content and construct Rate 2 if: a
relevance Rate 3 if: a + b + c
b Item 2.9 is clear and unambiguous Rate 4 if: a + b + c + d + e
c Item 2.9 is configured correctly
d Item 2.9 is not leading
e Item 2.9 is not likely to elicit a socially
desirable response

9
15 4 All criteria met. Item 2.9 is suitable if: Rate 1 if: none of the following
Language level a the respondents are likely to find the Rate 2 if: a
appropriate. language level appropriate Rate 3 if: a + b
Item relevant in b the respondents are likely to find the Rate 4 if: a + b + c
context. item relevant in their context
Respondents have c the respondents are likely to possess
knowledge to the knowledge required to respond
distinguish between correctly
the terms ‘beer’ and
‘wine’.
16 2 Item unclear due to Item 3.3 is formulated correctly if: Rate 1 if: none of the following
double negative. a Item 3.3 has content and construct Rate 2 if: a
relevance Rate 3 if: a + b + c
b Item 3.3 is clear and unambiguous Rate 4 if: a + b + c + d + e
c Item 3.3 is configured correctly
d Item 3.3 is not leading
e Item 3.3 is not likely to elicit a socially
desirable response
17 4 All criteria met. Item 3.3 is suitable if: Rate 1 if: none of the following
a the respondents are likely to find the Rate 2 if: a
language level appropriate Rate 3 if: a + b
b the respondents are likely to find the Rate 4 if: a + b + c
item relevant in their context
c the respondents are likely to possess
the knowledge required to respond
correctly
18 4 All criteria met. Item 3.5 is formulated correctly if: Rate 1 if: none of the following
a Item 3.5 has content and construct Rate 2 if: a
relevance Rate 3 if: a + b + c
b Item 3.5 is clear and unambiguous Rate 4 if: a + b + c + d + e
c Item 3.5 is configured correctly
d Item 3.5 is not leading
e Item 3.5 is not likely to elicit a socially
desirable response
19 3 Respondents not likely Item 3.5 is suitable if: Rate 1 if: none of the following
to possess required a the respondents are likely to find the Rate 2 if: a
knowledge. language level appropriate Rate 3 if: a + b
b the respondents are likely to find the Rate 4 if: a + b + c
item relevant in their context
c the respondents are likely to possess
the knowledge required to respond
correctly
20 1 The AIQ does not The AIQ is functional if: Rate 1 if: none of the following
cover ‘positive control’ a the information gathered by the AIQ Rate 2 if: a
– therefore content concerns the intended content Rate 3 if: a + b
validity compromised. domain Rate 4 if: a + b + c
b the AIQ is suitable for the intended
target population
c the AIQ produces information in a
format that facilitates data processing
21 2 Criterion a met: Aim The purpose of the AIQ is described if: Rate 1 if: none of the following
described in manual. a the aim of the AIQ is stated Rate 2 if: a
Criterion b not met: b the target population is defined Rate 3 if: a + b
Target population not c the design of the AIQ is described Rate 4 if: a + b + c
defined.
Criterion c met: Design
process described.

10
 PYC2606/201

22 2 Sampled is described. The properties of the AIQ are discussed Rate 1 if: none of the following
Item information is if: Rate 2 if: a
summarised but no a the sample used to test the AIQ is Rate 3 if: a + b
analysis is described described correctly Rate 4 if: a + b + c or a + b + d
and no indication is b the item analysis and selection Rate 5 if: a + b + c + d
given as to how items procedures are described correctly
were selected or c the reliability of the AIQ is discussed
discarded. correctly
Insufficient information d the validity of the AIQ is discussed
is provided about correctly
reliability and validity.
Information was
collected and tabled,
but reliability and
validity were not
discussed.
23 4 All criteria met. The procedures for utilising the AIQ are Rate 1 if: none of the following
Instructions for described if: Rate 2 if: a
administration are a instructions are provided for Rate 3 if: a + b
provided administering the AIQ Rate 4 if: a + b + c
Scoring instructions b instruction are provided for scoring
are provided the AIQ
Interpretation c instructions are provided for
instructions are interpreting results
provided
24 4 All criteria met. The manual is structured properly if: Rate 1 if: none of the following
a the text is clustered into topical Rate 2 if: a
sections Rate 3 if: a + b
b topics are presented in logical order Rate 4 if: a + b + c
c the information about each topic is
relevant and presented coherently
25 3 The text is factual and The manual is written properly if: Rate 1 if: none of the following
objective and is not a the text is factual and objective Rate 2 if: one of a, b, c
inundated with b the text is clear and to the point, with Rate 3 if: two of a, b, c
spelling, grammar and no ambiguous and/or conflicting Rate 4 if: a + b + c
punctuation errors. statements
However, the text does c the text is not inundated with
not refer correctly to numerous spelling, grammar and
items and tables punctuation errors
(numbers differ).
Therefore the text
contains conflicting
statements.

Wishing you success in your studies!

11