2/4/2025

Estimation
Statistical Inference
Point & Interval Estimation
Estimation
Sample Size Determination

Estimation Point Estimates

 Statistical estimation procedures use sample data to
Point estimates use sample data to calculate a
obtain the “best” possible estimates of the
single “best” value which estimates a population
corresponding population parameters.
parameter. Point estimates alone give no idea of
the magnitude of the error involved in the
There are two types of estimates: estimation process.
 Point estimates and
 Interval estimates.
Shahed B Sadique

Shahed B Sadique

Sampling Fundamentals Estimation 3 Sampling Fundamentals Estimation 4

1
 2/4/2025

Interval Estimates Criteria for choosing a good Estimator

An interval estimate describes a range of values with in The usefulness of a point estimate depends on the
criteria by which it is judged. The properties of
which a population is likely to lie. Suppose T is an
𝜎 estimators which are used to help us pick that
estimator of 𝜃 , such that E(T) = 𝜃 and is small. estimator that is most useful for our purpose are
 unbiasedness,
 efficiency,
 Consistency, and
 sufficiency
Shahed B Sadique

Shahed B Sadique
Sampling Fundamentals Estimation 5 Sampling Fundamentals Estimation 6

 (i) An estimator should on the average be equal to the value  (iii) An estimator should use as much as possible the
of the parameter being estimated. This is popularly known as information available from the sample. This property is
the property of unbiasedness. An estimator is said to be known as the property of sufficiency.
unbiased if the expected value of the estimator is equal to  (iv) An estimator should approach the value of population
the parameter being estimated. The sample mean 𝑥̅ is the parameter as the sample size becomes larger and larger. This
most widely used estimator because of the fact that it property is referred to as the property of consistency.
provides an unbiased estimate of the population mean µ. Keeping in view the above stated properties, the researcher
must select appropriate estimator(s) for his study.
 (ii) An estimator should have a relatively small variance. This
means that the most efficient estimator, among a group of
unbiased estimators, is one which has the smallest variance.
Shahed B Sadique

This property is technically described as the property of

Shahed B Sadique

efficiency.

Sampling Fundamentals Statistics II 7 Sampling Fundamentals Statistics II 8

2
 2/4/2025

A. Unbiased Estimator
An estimator T = 𝜃 is an unbiased estimator of the Thus the sample mean is
parameter 𝜃 , if its expected value is equal to that of ∑ 𝑥
. 𝑥̄ =
𝑛
the parameter itself, that is E(T) = 𝜃 .
𝑥
For example, the sample mean 𝑥̄ = is an unbiased
𝑛 ∑ 𝑥 ∑ 𝐸(𝑥 )
estimator of the population mean 𝜇 , since 𝐸(𝑥̄ ) = 𝜇 Now , 𝐸 𝑥̄ = =
𝑛 𝑛
Suppose, 𝑥 , 𝑥 , 𝑥 , ⋯ ⋯ ⋯ , 𝑥 is a random sample of size
n from some population with mean .
∑ 𝜇 𝑛𝜇
= = =𝜇
𝑛 𝑛
Shahed B Sadique

Shahed B Sadique
Sampling Fundamentals Estimation 9 Sampling Fundamentals Estimation 10

B. Efficient Estimator C. Consistent Estimator

An estimator T will be said to be efficient for 𝜃 if it yields An estimator is consistent for 𝜃 , if it yields values
estimates which are “close” to the value of 𝜃 .
which get closer and closer to the true value of the
The way of measuring the closeness of the value of T about its
parameter 𝜃 as the size of the sample is increased.
excepted value 𝜃 , that is, by considering the variance 𝜎 of T. The
estimator that has the smallest variance is said to be the most
Suppose, 𝑥 , 𝑥 , 𝑥 , ⋯ ⋯ ⋯ , 𝑥 is a random sample of
efficient estimator about the parameter 𝜃 . That is, the estimator size n from a normal population with mean  and
T is the most efficient estimator about 𝜃 for if E(T) = 𝜃 and variance 2. Thus the sample mean is 𝑥̄ = 𝑥 /𝑛
𝜎 < 𝜎, where T1 is any other unbiased estimator for 𝜃 .
As the size of the sample increases, the sample mean 𝑥̄
approaches to population mean . Therefore, we can
Shahed B Sadique

Shahed B Sadique

say that 𝑥̄ = 𝜇̂ is a consistent estimator for .

Sampling Fundamentals Estimation 11 Sampling Fundamentals Estimation 12

3
 2/4/2025

D. Sufficient Estimator
Let 𝑓(𝑥|𝜃) be the density function of a random variable The necessary and sufficient condition for t to be
x where  is unknown fixed parameter. Let sufficient for  is that the likelihood function L may
be factorized as follows
𝑥 ,𝑥 ,𝑥 ,⋯⋯⋯,𝑥 be a random sample from this 𝐿(𝜃 𝑥) = 𝑔(𝑡|𝜃). ℎ(𝑥 , 𝑥 ,𝑥 , ⋯ ⋯ , 𝑥 )
density. Let t and 𝑡 are two statistics such that 𝑡 is
not a function of t. If the conditional distribution of 𝑡
given t be independent of , then t is called a
sufficient statistic for . If further E(t) = , then t is
called a sufficient estimator of .
Shahed B Sadique

Shahed B Sadique
Sampling Fundamentals Estimation 13 Sampling Fundamentals Estimation 14

Point and Interval Estimation of Mean

Suppose 𝑥 , 𝑥 , 𝑥 , ⋯ ⋯ ⋯ , 𝑥 is a random sample of The best point estimate of  is 𝑥̄ . The expected value
size n from a normal population with mean µ and the variance of 𝑥̄ are E( 𝑥̄ ) =  and ,
𝜎
and known variance 2. The most efficient estimator 𝜎̄ = respectively.
𝑛
of the population mean µ is given by the sample
mean The statistic 𝑡 has a normal distribution with mean 
𝑥
𝜇̂ = 𝑥̄ =
𝑛 and variance (2/n). Thus the random variable
𝑥̄ − 𝜇
𝑧= has a standard normal Distribution, that is
𝜎/ 𝑛
z= E(z) = 0 and variance 𝜎 = 1.
Shahed B Sadique

Shahed B Sadique

Sampling Fundamentals Estimation 15 Sampling Fundamentals Estimation 16

4
 2/4/2025

𝑧
Thus 𝑃(−𝑧 <𝑧<𝑧 )=1−𝛼
, where 𝜎 𝜎
⇒ 𝑃(𝑥̄ − 𝑧 < 𝜇 < 𝑥̄ + 𝑧 ) =1−𝛼
𝑛 𝑛
is that value in the standard normal distribution that
𝛼
has area𝛼 to the left, that is
1−
2 The interval (tL, tU) = (𝑥̄ − 𝑧 , 𝑥̄ + 𝑧 )
𝑃(𝑧 < −𝑧 )= , 𝑎𝑛𝑑𝑧 = −𝑧 . (1)
2 .
is a 100(1-)% confidence interval for .

Substituting for z in (1) gives

𝑥̄ − 𝜇
𝑃(−𝑧 < 𝜎 <𝑧 )=1−𝛼
𝑛
Shahed B Sadique

Shahed B Sadique
Sampling Fundamentals Estimation 17 Sampling Fundamentals Estimation 18

Different Methods of Estimation A. Method of Maximum Likelihood Estimation

R. A. Fisher has not only propounded the criteria of Suppose, 𝑥 ,𝑥 ,𝑥 ,⋯⋯⋯,𝑥
be a random sample
consistency, efficiency and sufficiency which are desirable
properties of an estimator but has also developed (1921) a from the density 𝑓(𝑥|𝜃) . The likelihood function is given
method of estimating parameter. This method namely the by
method of maximum likelihood is the most important 𝐿(𝑥 , 𝑥 , ⋯ ⋯ , 𝑥 𝜃) = 𝑓(𝑥 |𝜃)
and powerful method so far known. There are some other
methods such as Karl Pearson’s method of moments,
Now our target is to choose a value of  denoted
Baye’s method, method of least square, method of
minimum chi-square. Method of maximum likelihood and by 𝜃 for which maximum likelihood function will be
method of moments are discussed below. maximize. Thus 𝜃 is the maximum likelihood estimator
of  .
Shahed B Sadique

Shahed B Sadique

Sampling Fundamentals Estimation 19 Sampling Fundamentals Estimation 20

5
 2/4/2025

EXAMPLE:

The maximum likelihood estimator is the solution of Let 𝑥 , 𝑥 , 𝑥 , ⋯ ⋯ ⋯ , 𝑥 be a random sample from
distribution 𝑁(𝜇, 𝜎 ). Find an ML estimator for .
𝛿𝐿 𝛿 log 𝐿
𝛿𝜃
= 0𝑜𝑟
𝛿𝜃
=0
Solution: The maximum likelihood function is
1 ∑ ( )
𝐿(𝜃 𝑥) = 𝑒
𝜎 ( 2𝜋)
1
log 𝐿 = − 𝑛 log( 𝜎. 2𝜋) − (𝑥 − 𝜇)
2𝜎
𝛿 log 𝐿 1
⇒ =− 2(𝑥 − 𝜇)(−1) = 0
Shahed B Sadique

Shahed B Sadique
𝛿𝜇 2𝜎
⇒ 𝜇 = 𝑥̄
Sampling Fundamentals Estimation 21 Sampling Fundamentals Estimation 22

B. Method of Moments Estimation Interval Estimation

It is one of the oldest methods of estimation. Let Estimating Population Mean:
𝑥 , 𝑥 , 𝑥 , ⋯ ⋯ ⋯ , 𝑥 be a random sample from i) When  is known & N infinite : 𝑥̅ ± 𝑧 ×
distribution 𝑓(𝑥|𝜃). Let 𝝁𝒓 , the r-th raw moment of the
distribution and 𝑚 , the r-th sample raw moments ii) When  is known & N is Finite: 𝑥̅ ± 𝑧 ×
exists. The method of moments consists in equating 𝝁𝒓
with 𝑚 (r = 1, 2, …..) and solving for . iii) When  is unknown & N is infinite : 𝑥̅ ± 𝑧 ×
iv) When  is unknown & N is infinite :
𝑠 𝑁−𝑛
𝑥̅ ± 𝑧 ×
𝑛 𝑁−1
Shahed B Sadique

Shahed B Sadique

Sampling Fundamentals Estimation 23 Sampling Fundamentals Statistics II 24

6
 2/4/2025

v) When  is unknown & N is infinite : Estimating Population Proportion:

𝑥̅ ± 𝑡 × For infinite population:
𝑝. 𝑞
𝑝±𝑧×
vi) When  is unknown & N is infinite : 𝑛
𝑠 𝑁−𝑛
𝑥̅ ± 𝑡 ×
𝑛 𝑁−1 For finite population:
𝑝. 𝑞 𝑁−𝑛
𝑝±𝑧× ×
𝑛 𝑁−1
Shahed B Sadique

Shahed B Sadique
Sampling Fundamentals Statistics II 25 Sampling Fundamentals Statistics II 26

Example: Interval Estimation

Example: From a random sample of 36 New Delhi civil The 95% Confidence interval for mean is
service personnel, the mean age and the sample σ
x±z×
standard deviation were found to be 40 years and 4.5 n
years respectively. Construct a 95 percent confidence .
= 40 ± 1.96 ×
interval for the mean age of civil servants in New Delhi.
Solution: The given information can be written as = 40 ± 1.47 Years
under:
 n = 36, 𝑥̄ = 40 years, and s = 4.5 years and the
standard normal variate, z, for 95 per cent confidence
is 1.96 (as per the normal curve area table).
Shahed B Sadique

Shahed B Sadique

27 28

7
 2/4/2025

Shahed B Sadique Estimating Population Proportion

Shahed B Sadique
Sampling Fundamentals Statistics II 29 Sampling Fundamentals Statistics II 30

CONCEPT OF STANDARD ERROR

 The standard deviation of sampling distribution of a statistic 2. The standard error gives an idea about the reliability and
is known as its standard error (S.E.) and is considered the key precision of a sample. The smaller the S.E., the greater the
to sampling theory. The utility of the concept of standard uniformity of sampling distribution and hence, greater is the
error in statistical induction arises on account of the following reliability of sample.
reasons: Conversely, the greater the S.E., the greater the difference
 1. The standard error helps in testing whether the difference between observed and expected frequencies. In such a situation
between observed and expected frequencies could arise due to the unreliability of the sample is greater. The size of S.E.,
chance. The criterion usually adopted is that if a difference is depends upon the sample size to a great extent and it varies
less than 3 times the S.E., the difference is supposed to exist inversely with the size of the sample.
as a matter of chance and if the difference is equal to or more
than 3 times the S.E., chance fails to account for it, and we
conclude the difference as significant difference.
Shahed B Sadique

Shahed B Sadique

Sampling Fundamentals Statistics II 31 Sampling Fundamentals Statistics II 32

8
 2/4/2025

 3. The standard error enables us to specify the limits within Range Percent Values
which the parameters of the population are expected to lie
with a specified degree of confidence. Such an interval is µ ± 1 S. E. 68.27
usually known as confidence interval. The following table µ ± 2 S. E. 95.45
gives the percentage of samples having their mean values
µ ± 3 S. E. 99.73
within a range of population mean µ ± S.E.
µ ± 1.96 S. E. 95.00
µ ± 2.5758 S. E. 99.00
Shahed B Sadique

Shahed B Sadique
Sampling Fundamentals Statistics II 33 Sampling Fundamentals Statistics II 34

 Important formulae for computing the standard (iii) Standard error of the difference between proportions of two
errors concerning various measures based on samples: 𝜎 = 𝑝. 𝑞. −
samples are as under:
 where p = best estimate of proportion in the population and
(a) In case of sampling of attributes: is worked out as under:
(i) Standard error of number of successes = 𝑛 . 𝑝 . 𝑞
where n = number of events in each sample, Note: Instead of the above formula, we use the following
formula:
p = probability of success in each event,
q = probability of failure in each event.
when samples are drawn from two heterogeneous populations
where we cannot have the best estimate of proportion in the
Shahed B Sadique

Shahed B Sadique

(ii) Standard error of proportion of successes, universe on the basis of given sample data. Such a situation often
arises in study of association of attributes.
Sampling Fundamentals Statistics II 35 Sampling Fundamentals Statistics II 36

9
 2/4/2025

(b) In case of sampling of variables (large samples): (iii) Standard error of standard deviation when population
 (i) Standard error of mean when population standard standard deviation is known:
deviation is known: 𝜎 ̅ =
Note: This formula is used even when n is 30 or less.
(iv) Standard error of standard deviation when population
standard deviation is unknown:
(ii) Standard error of mean when population standard deviation
is unknown: 𝜎 ̅ =

(v) Standard error of the coefficient of simple correlation:

Where 𝑠 = ∑(𝑥 − 𝑥̅ )
Shahed B Sadique

Shahed B Sadique
Sampling Fundamentals Statistics II 37 Sampling Fundamentals Statistics II 38

(vi) Standard error of difference between means of two samples:  (c) In case of sampling of variables (small samples):
 (a) When two samples are drawn from the same population:  (i) Standard error of mean when p is unknown:

 (b) When two samples are drawn from different populations:

 Where
Shahed B Sadique

Shahed B Sadique

Sampling Fundamentals Statistics II 39 Sampling Fundamentals Statistics II 40

10
 2/4/2025

 (ii) Standard error of difference between two sample means

when σp is unknown

Statistical Inference
Sample Size
Determination
Shahed B Sadique

Sampling Fundamentals Statistics II 41

Size of the sample should be determined by a researcher keeping (iii) Nature of study: If items are to be intensively and
in view the following points: continuously studied, the sample should be small. For a general
(i) Nature of universe: Universe may be either homogenous or survey the size of the sample should be large, but a small sample
heterogenous in nature. If the items of the universe are is considered appropriate in technical surveys.
homogenous, a small sample can serve the purpose. But if the (iv) Type of sampling: Sampling technique plays an important
items are heterogenous, a large sample would be required. part in determining the size of the sample. A small random
Technically, this can be termed as the dispersion factor. sample is apt to be much superior to a larger but badly selected
(ii) Number of classes proposed: If many class-groups (groups Sample.
and sub-groups) are to be formed, a large sample would be (v) Standard of accuracy and acceptable confidence level: If the
required because a small sample might not be able to give standard of accuracy or the level of precision is to be kept high,
a reasonable number of items in each class-group. we shall require relatively larger sample. For doubling the
Shahed B Sadique

Shahed B Sadique

accuracy for a fixed significance level, the sample size has to be

increased fourfold.
Sampling Fundamentals Statistics II 43 Sampling Fundamentals Statistics II 44

11
 2/4/2025

(vi) Availability of finance: In practice, size of the sample depends There are two alternative approaches for determining the size of
upon the amount of money available for the study purposes. This the sample. The first approach is “to specify the precision of
factor should be kept in view while determining the size of estimation desired and then to determine the sample size
sample for large samples result in increasing the cost of sampling necessary to insure it” and the second approach “uses Bayesian
estimates. statistics to weigh the cost of additional information against
(vii) Other considerations: Nature of units, size of the population, the expected value of the additional information.”
size of questionnaire, availability of trained investigators, the The first approach is capable of giving a mathematical solution,
conditions under which the sample is being conducted, the time and as such is a frequently used technique of determining ‘n’.
available for completion of the study are a few other The limitation of this technique is that it does not analyse the
considerations to which a researcher must pay attention while cost of gathering information vis-a-vis the expected value of
selecting the size of the sample. information.
Shahed B Sadique

Shahed B Sadique
The second approach is theoretically optimal, but it is seldom
used because of the difficulty involved in measuring the value of
information. Hence, we shall mainly concentrate here on the first
Sampling Fundamentals Statistics II 45 approach.
Sampling Fundamentals Statistics II 46

Sample size when estimating mean Sample size when estimating Population Proportion
i) In case of infinite Population :
𝑧 ×𝜎 i) In case of infinite Population : 𝑛 =
𝑛=
𝑒

ii) In case of finite Population :

× × ×
𝑁×𝑧 ×𝜎 ii) In case of finite Population : 𝑛 =
𝑛= × ×
(𝑁 − 1)𝑒 +𝑧 × 𝜎
Shahed B Sadique

Shahed B Sadique

Sampling Fundamentals Statistics II 47 Sampling Fundamentals Statistics II 48

12
 2/4/2025

Example:
Determine the size of the sample for estimating the true weight Solution: In the given problem we have the following:
of the cereal containers for the universe with N = 5000 on the N = 5000;
basis of the following information: p = 2 ounces (since the variance of weight = 4 ounces);
(1) the variance of weight = 4 ounces on the basis of past e = 0.8 ounces (since the estimate should be within 0.8 ounces of
records. the true average weight);
(2) estimate should be within 0.8 ounces of the true average z = 2.57 (as per the table of area under normal curve for the
weight with 99% probability. given confidence level of 99%).
Will there be a change in the size of the sample if we assume Hence, the confidence interval for µ is given by
infinite population in the given case? If so, explain by how much?
𝜎 𝑁−𝑛
𝑥 ±𝑧×
𝑛 𝑁−1
Shahed B Sadique

Shahed B Sadique
49 50

and accordingly the sample size can be worked out as under: But if we take population to be infinite, the sample size will be
worked out as under:
𝑧 ×𝜎
𝑛=
𝑒
(2.57) × 2
= = 41.28 ≅ 41
(0.8)
Thus, in the given case the sample size remains the same even if
we assume infinite population.

Hence, the sample size (or n) = 41 for the given precision and
Shahed B Sadique

Shahed B Sadique

confidence level in the above question with finite population.

51 52

13