SPSS Training Program & introduction to statistical testing

May y 11, 2009 Part 1

Variance and Variables

It has been said that all research is conducted for the purpose of explaining variance i variance, i.e. e explaining why individuals are different from each other. (Wiersma & Jurs, 2005, p.84)
SPSS Statistical Package for the Social Sciences

A more likely scenario is that some individuals in group 1 & group 2 measured characteristic. All individuals in group 1 are different are the same with respect to a certain measured characteristic. And yet, from all decide individuals in group This is notwithstanding that we might statistically that2. the overlap is small enough to be there able issay variability between individuals in group 1 and samples in group 2. the to that these two groups are unlikely to represent from In this scenarioi.e. no that need for statistical assessment, they are same population, the alternative hypothesis clearly different (but very situation in research). (they represent samples from rare two distinct groups) is more plausible.

No overlap between the individuals of both groups for a particular

What is a Variable?
Simply a characteristic that varies within a population or group grade level Test T t scores Gender, age, grade attitudes Self efficacy

Another outcome is that we find the two groups overlap a lot on a particular characteristic that we decide on probability that these Group 1 Group 1 two groups are not sufficiently different from each other to warrant consideration of an alternative hypothesis to explain the data.

Group 2

Kinds of data
Four kinds of variables This is important when choosing appropriate kind of analysis (particularly statistical tests)

When investigating Causal relationships

Dependent variables Independent variables E.g. g Are test scores dependent p on g gender? (test scores dependent variable; independent variable gender) Are working relationships between colleagues dependent on leadership style of senior management? (working relationships dependent variable; independent variable leadership style of senior management)

Kind Ratio Interval Ordinal Categorical

Description Constant interval, True zero (e.g. test scores) Constant interval, arbitrary or non-zero.
e.g. grade, survey response scales

Intervals not equal but inferred directionality.

e.g. size of school (small, medium, large).

Nominal Qualitative
e.g. gender, Ethnicity, Language background

Generalisability
In the simplest sense, when we collect quantitative data about some characteristics of a sample of participants, we commonly seek to infer that our results can tell us something meaningful about the larger population from which the sample was drawn external validity. Commonly we compare groups within our sample on one or more measured variables to determine whether there are any meaningful differences between them (e.g. are there gender differences in academic performance?)

How do we decide what statistical tests to use? example questions

What kind of variables am I comparing? If dependent variables are ratio or interval data is my data normally distributed (or near enough to)? Should I use a Non Non-parametric parametric or Parametric statistical test?
http://en.wikipedia.org/wiki/Parametric_statistics

How many groups am I comparing? Are these groups independent, or have I measured the same people twice or more for a given characteristic? Am I inferring causal relationships?

Appropriate statistical tests independent groups

For comparing measures of central tendency (i.e. ratio, interval or ordinal data) 2 groups compared with respect to one dependent variable > 2 groups compared with respect to one dependent variable > 2 groups compared with respect to two independent variables Comparing groups with respect to more than one independent variable Parametric test (ratio, interval data) compares means T-test Non-parametric test (ordinal or non-normally distributed data) compares medians Mann-Whitney U

Appropriate statistical tests for repeated measures (same group measured like pre-post)
For comparing measures of central tendency Parametric test (ratio, interval data) compares means Non-parametric test (ordinal or non-normally distributed data) compares medians Wilcoxon test

1 group measured twice Paired T-test & compared with respect to 1 dependent variable 1g group p measured twice ANOVA repeated p or more with respect to 1 measures independent variable 1 group measured twice or more with respect to >1 independent variable 1 group measured twice or more with respect to an independent variable 2-way ANOVA repeated measures ANCOVA also relevant when controlling for predata or another covariate

1-way 1 way ANOVA

Kruskal-Wallace Kruskal Wallace

Friedman

2-way ANOVA

Kruskal-Wallace need to collect interval or ratio data. No (see text Orequivalent. Kruskal-Wallace in blue box)by lots of followed Mann-Whitney U tests but not considered appropriate

MANOVA (like an ANOVA)

Appropriate statistical tests for categorical data

When the dependent variable is categorical (e.g. yes/no responses, or preference for a particular program) the Chi square test is used Tests for difference in proportion by comparing expected and observed frequencies for two or more groups. If the expected e pected and obser observed ed freq frequencies encies are comparable across groups you would report no significant difference. If the expected and observed frequencies are very different across groups you would report that an alternative hypothesis is more likely to explain the data (that the results suggest the groups are different).

correlation
When you are interested in determining the degree of relationship between two non-categorical variables g E.g.
height and weight; self-confidence and academic outcomes; number of professional development workshops attended and leadership ambitions
For ratio or interval data Pearson correlation (parametric) For ordinal data Spearman correlation (non-parametric)

Correlations
Each point represents a persons score for both scales. v1 = variable 1 (e.g. level of boredom learning grammar) v2 = variable 2 (e.g. test scores)

For TESOL teachers the more experience applying Communicative Language Teaching methods, the less likely they were to report problems with implementation (a strong negative relationship)
Years of experience applying CLT and perceived problems with implementation
problem s applying C L LT

v2

v4

v6

v1
negative

v3

v5
No correlation, no relationship

Caution: Not necessarily causal relationship

6 5 4 3 2 1 0 0 2 4 6 years of experience with CLT

positive

What could v3,v4 represent?

Correlation
When you conduct a correlation analysis you obtain a correlation coefficient r between -1 and 1. Zero = no relationship (variables do not co-vary) +1 = as one variable increases by one scale point the other variable increases by y one scale point p -1 = as one variable increases by one scale point the other variable decreases by one scale point Values in between -1 and + 1 are indicative of the degree of relationship

correlation
Correlation coefficients can be statistically significant and still be low. r <|0.3| weak Correlations between |0.4-0.6| moderate and |0.7-0.9| strong A correlation of 0.4 means that an estimated 16% of the variance is explained by this relationship. A correlation l ti of f 0.5 0 5 means that th t an estimated ti t d 25% of f th the variance i i is explained by this relationship 0.6 - 36%; 0.7 49% ; 0.8 64% 0.9 = 81% So if one found a correlation of 0.40 between level of boredom with learning grammar and grammar test scores, (100 -16=84% of the variation in scores is explained by other factors (e.g. ability; background of the students; languages spoken; which may or may not have been measured).

Kinds of Distributions in a population - variability

The bell shape curve has perfect symmetry, where the mean, median and mode are identical.
M Mean, mode d and d median di

Why is this important?

parametric statistical tests assume for the variables measured, the distribution of scores approximates a normal curve. Parametric statistical findings can be invalid if data is not approximately normally distributed, particularly when statistical significance is claimed.

frequency However, real data do not fit the bell shape perfectly

Measures to describe deviation from normal distribution

What are the relative positions of the mean, median and mode?
mode median mean

Skewness measure of symmetry Kurtosis measure of flatness or

peakness
Negative skew -ve

What are the relative positions of the mean, median and mode?
mode median mean +ve

Kurtosis
Kurtosis is a measure of whether the data are peaked or flat relative to a normal distribution. A positive kurtosis indicates a peaked peaked distribution. A negative kurtosis indicates a flattened distribution

Positive skew

General forms of kurtosis

Standard Deviation

A measure of the average difference of the scores from the mean, but determined using the squared difference see next slide

Example of SD computation for set of scores

SCORES 77 78 79 80 81 82 83 Sum 560 Mean 560/7 = 80 Variance = Sum of the differences squared divided by the number of scores 28/7 = 4 Standard deviation is the square root of the variance = 4 =2 difference from mean -3 -2 -1 0 +1 1 +2 +3 0 (??) Square of the difference 9 4 1 0 1 4 9 28

Example of SD computation for set B scores

SCORES 50 60 70 80 90 100 110 Sum 560 Mean 560/7 = 80 Variance = Sum of the differences squared divided by the number of scores 2800/7 = 400 Standard deviation is the square root of the variance = 400 =20 difference from mean -30 -20 -10 0 +10 10 +20 +30 0 (??) Square of the difference 900 400 100 0 100 400 900 2800

How does the standard deviation really help us understand the spread in the data?

Units of kurtosis and skewness are in Standard deviations. But B t more importantly i t tl

Along with the mean, the standard deviation allows you to describe a large data set very effectively (particularly if normally distributed)

99.7%

For normal distribution of scores (bell shape curve)

Approximately 68% of the scores are within one standard deviation of the mean Approximately 95% of scores are within t two standard t d d deviations d i ti of f th the mean Approximately 100% of scores are within 3 standard deviations of the mean
.1359 .0215
-3 SD -2 SD -1 SD

95.4% 68.3%

.3413

.3413

.0013

.1359 .0215 mean

1 SD 2 SD 3 SD

.0013

Check this with our data

Write down the mean and Standard deviation for our data Mean = ? SD = ? If normally ll distributed, di t ib t d expect t about b t 68% of our sample to have provided a value between ? and ? Use frequency table to assist you.

But what is the use of this?

because many populations are not perfectly normally distributed with respect to a variable. However large collections of random However, samples do pattern themselves into a normal distribution. This is the basis of parametric statistical tests.

Standard Error of the mean

The standard deviation of a sampling distribution of means is called the standard error. = STANDARD DEVIATION divided by the SQUARE ROOT OF THE sample SIZE your sample p mean is to the It is an estimate of how close y population of interest. So a sample mean of 3.5 with a standard error of 0.5 indicates that you can be confident that 68.3% of the population of interest (not the sample) would provide a value for the variable between 3 and 4. And that 95.4% of the population have a value between 2.5 and 4.5 And that 99.7% of the population have a value between 2 and 5

Significance testing
When you read a finding in a research article that statistical significance was obtained when comparing two group sample means (for example), it refers to the probability of these two means having come from different populations. Probability y of 0.05 represents p a 5/100 chance, , and this indicates that the estimated population means approximately overlap at the 2 SE range. Sample group A mean = 2.6 SE = 0.1 Sample group B mean = 3.0 SE = 0.1 So while the two samples may have been drawn from the same population the chance is 5/100. This probability value is commonly considered unlikely to have occurred by chance.

How girls often bullying happen at anddo boys yousee would expect to report From a you cursory glance what that bullying occurs at school assuming there is no gender difference, school? conclusions are you drawing?
(i.e. that the results are due to sampling error)

The Chi-square test involves determining how many

Never Girls Boys Total 4 9 13

Sometimes Often 21 23 44 6 5 11

total 31 37 68

For this sample there is little difference between the How often do you see bullying happen at observed and actual frequencies of responses for the boys school?observations & girls with respect to their reported concerning bullying.

Never

Sometimes Often
Of the 2113 students 6 p they y have who report 20.1 observed 5.0 never bullying, how 23 5 many would you expect to 23.9 be girls if there6.0 was no 44 difference?11 gender

total 31 37 68

Girls Boys Total

4
31/68 5.9 X 13

9
7.1

13

Chi-square formula

Chi squared test

There are two ways to do this test Select analyze Select Descriptive statistics Select cross tabs Select one variable into row The other into column Select statistics Select chi-square Select cell Select expected values as well as observed (or select non-parametric tests and chi-squared)

Practical significance
Effect size

Go to: http://web.uccs.edu/lbecker/Psy590/es.htm#I,%20Overview for more information

Effect size (ES) is a name given to a family of indices that measure the magnitude of a treatment effect. Unlike significance tests, these indices are independent of sample size. ES measures are the common currency of metameta analysis studies that summarize the findings from a specific area of research. (Lipsey and Wilson,1993). But useful for single studies as well to measure practical significance of findings

10

What do I mean by effect size and practical significance? Imagine that you were reporting on whether an educational intervention in primary science teaching had a positive impact. One of your measures was test scores. While you might discover a difference between as usual science classrooms and your intervention program in the direction you were seeking, this difference needs a context. i.e. was the average difference half a grade? or perhaps a quarter of a grade? Or perhaps only 0.1 of a grade? How do you make a decision as to whether any of these findings are important enough to pursue your intervention on a wider scale? Statistically speaking, an effect size in this kind of scenario can report on the proportion of students in the intervention program who obtained scores greater than those in the as usual group. Statistically speaking, we can also report on the estimated proportion of variance in the test scores that can be attributed to the intervention.

There is a wide array of formulas used to measure ES F For the th occasional i l reader d thi this di diversity it can be confusing

Effect Size Measures for Two Independent Groups

Cohen's d (Cohen (1988) defined d as the difference between the means of two independent groups, (M1 - M2), divided b th by the standard t d dd deviation i ti ( (sd), d) of f either ith group. Cohen argued that the standard deviation of either group could be used when the variances of the two groups are homogeneous (i.e. similar or the same) d = (M1 - M2)/ sd

In practice, the pooled standard deviation, spooled, is commonly used (Rosnow and Rosenthal, 1996). d = M1 - M2 / sdpooled sdpooled = [(sd1+ sd2) / 2]

11

 d can also be computed from the value of the t test of the differences between the two groups (Rosenthal and Rosnow, 1991). 1991) d = 2t / df df is the degrees of freedom for the t test

You can obtain an effect size that measures the percent of variance explained by the independent variable in SPSS. When conducting an ANOVA, select options and select effect size This eta squared value is a measure of the proportion of variance explained by the independent variable

Practical significance is just as important as statistical significance More journals are requiring this to be reported with all quantitative data analysis

12

Help resources
On the menu select Help. Case studies takes you through examples of particular tests, very useful. Statistical Coach helps you determine what kind of test you need to do for your data data. Fairly useful. useful Online SPSS support with examples Web address http://www.visualstatistics.net/SPSS%20workbook/spss workbook.htm (Still incomplete)

13