Test of significance

Dr. Imran Zaheer
JRII
Dept. of Pharmacology

outline
 Types of data
 Basic terms – Sampling Variation, Null hypothesis, P value
 Steps in hypothesis testing
 Tests of significance and type
 SEDP
 Chi Square test
 Student t test
 ANOVA

Types of Data
Qualitative Data:
• Also called as enumeration data.
• Qualitative are those which can be answered as YES or NO,
Male or Female, tall or short.
• Can only measure their numbers with out unit of measurement
eg: Number of males, Number of MDR-TB cases among TB
patients, etc.

Quantitative Data (measurement data)
• Quantitative are those which can be measured in numbers with
measurement, like Blood pressure (mm of Hg), haemoglobin
(gm%), height (cm) etc.
• A set of quantitative date can be expressed in mean and its
standard deviation.
• Mean is the average of all variables in the data.
• Standard deviation is a measure of the distribution of variables
around the mean

Sampling Variation
• Research done on samples and not on populations.
• Variability of observations occur among different samples.
• This complicates whether the observed difference is due to
biological or sampling variation from true variation.
• To conclude actual difference, we use tests of significance

Null Hypothesis (HO)
• 1st step in testing any hypothesis.
• Set up such that it conveys a meaning that there exists no
difference between the different samples.
• Eg: Null Hypothesis – The mean pulse rate among the two
groups are same (or) there is no significant difference between
their pulse rates.

• By using various tests of significance we either:
–Reject the Null Hypothesis
(or)
–Accept the Null Hypothesis
• Rejecting null hypothesis → difference is significant.
• Accepting null hypothesis → difference is not significant.

Level of Significance – “P” Value
• p-value is a function of the observed sample results
(a statistic) that is used for testing a statistical hypothesis.
• It is the probability of null hypothesis being true. It can accept
or reject the null hypothesis based on P value.
• Practically, P < 0.05 (5%) is considered significant.

• P = 0.05 implies,
– We may go wrong 5 out of 100 times by rejecting null
hypothesis.
– Or, We can attribute significance with 95% confidence.

Steps in Testing a Hypothesis
• General procedure in testing a hypothesis
1. Set up a null hypothesis (HO).
2. Define alternative hypothesis (HA).
3. Calculate the test statistic (t, X2 Z, etc).
4. Determine degrees of freedom.
5. Find out the corresponding probability level (P Value) for
the calculated test statistic and its degree of freedom.
6. This can be read from relevant tables.
7. Accept or reject the Null hypothesis depending on P value

Test of significance
• The test which is done for testing the research hypothesis
against the null hypothesis.
Why it is done?
-To assist administrations and clinicians in making decision.
• The difference is real ?
• Has it happen by chance ?

Classification of tests of significance
For Qualitative data:-
1. Standard error of difference between 2 proportions (SEp1-p2)
2. Chi-square test or X2
For Quantitative data:-
1. Unpaired (student) ‘t’ test
2. Paired ‘t’ test
3. ANOVA

Standard error of difference between
proportions (SEp1-p2)
• For comparing qualitative data between 2 groups.
• Usable for large samples only. (> 30 in each group).
• Calculate by (SEp1-p2) using the formula

• And then calculate the test statistic ‘Z Score’ by using the
formula:
• If Z > 1.96 P < 0.05 SIGNIFICANT
• If Z < 1.96 P > 0.05 INSIGNIFICANT

Example
• Consider a hypothetical study where cure rate of Typhoid fever
after treatment with Ciprofloxacin and Ceftriaxone were
recorded to be 90% and 80% among 100 patients treated with
each of the drug.
• How can we determine whether cure rate of Ciprofloxacin is
better than Ceftriaxone?

Solution
Step – 1: Set up a null hypothesis – H0:
– “There is no significant difference between cure rates of
Ciprofloxacin and Ceftriaxone.”
Step – 2: Define alternative hypothesis – Ha:
– “Ciprofloxacin is 1.125 times better in curing typhoid fever
than Ceftriaxone.”
Step – 3: Calculate the test statistic – ‘Z Score’

Step – 3: Calculating Z score
– Here, P1 = 90, P2 = 80
– SEP1-P2 will be given by the formula:
So, Z = 10/5 = 2

Step – 4: Find out the corresponding P Value
– Since Z = 2 i.e., > 1.96, hence, P < 0.05
Step – 5: Accept or reject the Null hypothesis
– Since P < 0.05, So we reject the null hypothesis(H0)
There is no significant difference between cure rates of
Ciprofloxacin and Ceftriaxone
– And we accept the alternate hypothesis (Ha)
that, Ciprofloxacin is 1.125 times better in curing typhoid
fever than Ceftriaxone

Chi square (X2) test
• This test too is for testing qualitative data.
• Its advantages over SEDP are:
– Can be applied for smaller samples as well as for large
samples.
• Prerequisites for Chi square (X2) test to be applied:
– The sample must be a random sample
– None of the observed values must be zero.
– Adequate cell size

Steps in Calculating (X2) value
1. Make a contingency table mentioning the frequencies in
all cells.
2. Determine the expected value (E) in each cell.
3. Calculate the difference between observed and expected
values in each cell (O-E).

• Calculate X2 value for each cell
• Sum up X2 value of each cell to get X2 value of the table.

Example
• Consider a study done in a hospital where cases of breast
cancer were compared against controls from normal
population against possession of a family history of Ca Breast.
• 100 in each group were studied for presence of family history.
• 25 of cases and 15 among controls had a positive family
history.
• Comment on the significance of family history in breast
cancer.

Solution
• From the numbers, it suggests that family history is 1.66
(25/15) times more common in Ca breast.
• So is it a risk factor in population
• We need to test for the significance of this difference.
• We shall apply X2 test.

Solution
Step – 1: Set up a null hypothesis
– H0: “There is no significant difference between incidence of
family history among cases and controls.”
Step – 2: Define alternative hypothesis
– Ha: “Family history is 1.66 times more common in Ca
breast”
Step – 3: Calculate the test statistic – X2

Step – 3: Calculating X2
1. Make a contingency table mentioning the frequencies in all
cells

Step – 3: Cont..
2. Determine the expected value (E) in each cell.

Step – 4: Determine degrees of freedom.
• DoF is given by the formula:
DoF = (r-1) x (c-1)
where r and c are the number of rows and columns
respectively
• Here, r = c = 2.
• Hence, DoF = (2-1) x (2-1) = 1

Step – 5: Find out the corresponding P Value
– P values can be calculated by using the X2 distribution
tables

Step – 6: Accept or reject the Null hypothesis
• In given scenario,
X2 = 3.125
• This is less than 3.84 (for P = 0.05 at dof =1)
• Hence Null hypothesis is Accepted, i.e.,
“There is no significant difference between incidence of
family history among cases and controls”

Student’s ‘t’ test
• Very common test used in biomedical research.
• Applied to test the significance of difference between two
means.
• It has the advantage that it can be used for small samples.
Types
—Unpaired ‘t’ test
—Paired ‘t’ test.

Unpaired ‘t’ test
• This is used to test the difference between two independent
samples
• Test statistic is
Standard error of difference between two means.

where S1 and S2 are Standard deviations and n1 and n2 are the
sample sizes of 1st and 2nd samples respectively.

Example
 The chest circumference (in cm) of 10 normal and
malnourished children of the same age group is given below:
Normal : 42, 46, 50, 48, 50, 52, 41, 49, 51, 56
Malnourished : 38, 41, 36, 35, 30, 42, 31, 29, 31, 35
Find out whether the mean chest circumference is significantly
different in the two groups (table ‘t’ value at 18 df at 0.05 level
of significance is 2.02)

Solution
• In the given example, following are the values of mean, SD
and sample size
Mean SD N
First sample 48.5 4.53 10
Second sample 34.8 4.57 10
By substituting the values, we get
(a)

Solution contd.
(b) Thus S (x1 – x2) = 4.55 1 + 1
10 10
= 4.55 0.2
= 4.55 X 0.44 = 2.03 cm

Solution contd.
(c) Calculation of t value
t = X1 – X2
S (x1 – x2)
= 48.5 – 34.8
2.03
= 6.75

Solution contd.
(d) Degrees of freedom (df) = n1 + n2 -2
= 10 + 10 – 2 = 20 -2 = 18.
 The maximum ‘t’ value for 18 df for 0.05 level of significance
is 2.02 (from table)

Solution contd.
(e) Conclusion: As the calculated ‘t’ value is higher than the
table value for 18 df and 0.05 level of significance,
— reject the null hypothesis and state that the difference in the
mean chest circumference values between normal and
malnourished groups is statistically significant.
— The difference might not have occurred due to chance
(P< 0.05)

Paired ‘t’ test
• This test is used where each person gives two observations –
before and after values
• For example, if the same person gives two observations before
and after giving a new drug – in such cases, this test is used.

Example
• In a clinical trial to evaluate a new drug in reducing anxiety
scores of 10 patients,
• the following data are recorded.
• Find out whether the drug is significantly effective in reducing
anxiety score
• (Table ‘t’ value at 9 df at 0.05 level of significance is 2.21)

Before
treat.
(X1)
After
treat.
(X2)
Differ.
(X1 –X2)
= x
x (x - x) (x – x)2
22 19 + 3 1.3 + 1.7 2.89
18 11 +7 1.3 + 5.7 32.49
17 14 +3 1.3 + 1.7 2.89
19 17 + 2 1.3 + 0.7 0.49
22 23 - 1 1.3 - 2.3 5.29
12 11 + 1 1.3 - 0.3 0.09
14 15 - 1 1.3 - 2.3 5.29
11 19 -8 1.3 -9.3 86.49
19 11 + 8 1.3 + 6.7 44.89
7 8 -1 1.3 - 2.3 5.29
161 148 + 13 +13 --- 186.1

Solution contd.
Calculation of Mean of difference and SD
Mean = Σ x = +13 / 10 = +1.3
n
SD = Σ (x – x) 2
n-1
= 186.1 / 9 = 4.54

Calculation of Standard error of mean
(c) SE = SD / n
= 4.54 / 10
= 4.54 / 3.16 = 1.43
(d) Calculation of t value
t = x / SE = 1.3 / 1.43 = 0.90
(e) Calculation of df = n-1 = 10 -1 = 9

Solution contd.
Conclusion: As the calculated ‘t’ value is lower than the table
value for df -9 at 5% level (0.05 level) of significance,
 we accept the null hypothesis and conclude that there is no
statistically significant difference in the anxiety scores before
and after treatment.
 The difference might have occurred because of chance
(P>0.05; NS)

How ANOVA works
ANOVA measures two sources of variation in the data and
compares their relative sizes
• variation BETWEEN groups
• for each data value look at the difference between its
group mean and the overall mean
• variation WITHIN groups
• for each data value look at the difference between that
value and the mean of its group
 2
iij xx 
 2
xxi 

The ANOVA F-statistic is a ratio of the Between Group
Variaton divided by the Within Group Variation:
MSE
MSG
Within
Between
F 
A large F is evidence against H0, since it indicates that
there is more difference between groups than within
groups.

Test of significance

More Related Content

What's hot (20)

Similar to Test of significance (20)

Recently uploaded (20)

Test of significance