Module 7 Research Design

Time Frame:
OVERVIEW
According to Cooper and Schindler (2001), a researcher adopts a research design to
guide the measurement, collection and analysis of data. When a researcher uses
appropriate research design, his/her study will have accurate results and valid
conclusions. In this module, you will study research designs commonly used in
language research.
Learning Outcome/s
At the end of this module, you are expected to:
1. identify the features of a research article that make it qualitative, descriptive or
experimental;
2. determine an appropriate research design to a given research problem; and
3. identify the method/s of data collection to a given research topic.

INITIAL ACTIVITY
To test your background knowledge and expectations about research design please
fill out the matrix below.
Research Design What I know? What I want to know?
Qualitative

Descriptive

Experimental

1|of 10
CONTENT FOCUS
Determining a research design requires a careful plan on the most
appropriate approach to carry out the objectives or purpose of the research. Will the
researcher utilize the synthetic or analytic approach or the heuristic or deductive
approach? Or, to what extent can the design be a combination of the different kinds
of research approaches? Do the data gathered require quantitative or qualitative
analysis? As such, the principal distinctions of the intrinsic nature of the object of
study, the role of the researcher in relation to the subject of investigation, whether
the validity and reliability of the findings require empirical study must be considered.
As Shulman (1981) explained:
While qualitative analysis will allow us to study individual performance closely, it may
not represent the behavior of other learners and is therefore of questionable value
for generalization to language acquisition by others. … when our interest is in the
normative acquisition behavior of population, quantification represents a reality of the
group.

A. QUALITATIVE RESEARCH

1. Scope of Qualitative Research

Qualitative research is heuristic and not deductive, since decision about

research questions or data are made before the study begins based on observations
and not focus on a hypothesis based on existing theory like ethnographic
methodologies. Since research has been viewed from a synthetic or holistic
perspective the research limits the focus of the coverage as the investigation
progresses.
Qualitative methods will primarily concerned with describing the observable
behaviors and activities and how they occur in certain natural context. Data gathered
are presented from the perspectives of the respondents, so the biases of the
researcher do not influence nor distort the data collection, interpretation, and
analysis. Researchers avoid describing selected aspects of behavior, for qualitative
methods avoid established hypothesis or a priori knowledge, thereby any variable
become the focus of the research.
Qualitative research may be utilized in language research when the
researcher is concerned with discovering and describing language acquisition in the
natural situation where it cannot be predicted specifically how an activity will affect
certain linguistic behavior. The ultimate goal, therefore, is to discover patterns of
language behavior not yet explained such that such behavior is understood from the
perspective of the language learner.
Researchers engaged in qualitative research are participant-observers
themselves. They participate and record their observations and experiences in
diaries while participating in the very act they are describing. In some cases, the

2|of 10
study involves non-participant observation. The researcher record his observations
of the linguistic events, but without the use of a questionnaire or other research tools
(Long 1983). It aims to ‘reconstruct’ accurately what the respondents experience to
understand the language issues from the perspective of the subjects themselves.
Brian Mittman characterize qualitative research by its emphasis on
describing, understanding and explaining complex phenomena – on studying, for
example, the relationships, patterns and configurations among factors; or the
complex in which activities occur. The focus is on understanding the full multi-
dimensional, dynamic picture of the subject of study.
Its approaches contrast with quantitative methods that aim to divide
phenomena into manageable, clearly defined pieces, or variables. Quantification is
good for separating phenomena into distinct and workable elements of a well-defined
conceptual framework. But when we focus research on what we already know how
to quantify, (e.g., what can be reliably quantified), we may miss factors that are key
to a real understanding the phenomena being studied. The downside of
quantification is that it does not always support (as well as qualitative methods)
understanding of complex, dynamic and multi-dimensional wholes
Qualitative methods are useful, not only in providing rich descriptions of
complex phenomena, but in constructing or developing theories or conceptual
frameworks, and in generating hypothesis to explain those phenomena.
A wide range of tested qualitative research methods are available to address
these challenges. The selection of method, or combination of method, will be tailored
to the questions being studied and the setting for research.

Main Types of Qualitative Research

Case study Attempts to shed light on a phenomena by studying
indepth a single case example of the phenomena. The
case can be an individual person, an event, a group, or an
institution.
Grounded theory Theory is developed inductively from a corpus of data
acquired by a participant-observer.

Phenomenology Describes the structures of experience as they present

themselves to consciousness, without recourse to theory,
deduction, or assumptions from other disciplines

Ethnography Focuses on the sociology of meaning through close field

observation of socio cultural phenomena. Typically, the
ethnographer focuses on the community.

3|of 10
 Historical Systematic collection and objective evaluation of data
related to past occurrences in order to test hypotheses
concerning causes, effects, or trends of these events that
may help to explain present events and anticipate future
events. (Gray, 1996)

Using Qualitative Research for Hypothesis Testing

Typical Methods of Qualitative Research
Naturalistic inquiry, or ethnography, has its roots in anthropology and sociology
and involves long-term exposure to a setting or a group of people. Extensive use of
unstructured observations and conversations documented by detailed field notes
form the basis for this type of research, often considered the purest form of
qualitative research. Naturalistic inquiry is used when situations are unique and
complex, when the level of uncertainty about the questions to ask is high and when
there is little or no theory to direct the investigator.
A subset of this type of inquiry involves participant observation in which the
investigator becomes a part of the setting or the process being studied. (Sofaer)
reports that she was able to learn more from attending a few group meetings in a
particular setting than she could have by using more structured qualitative methods
such as interviews or surveys.
Case studies are the preferred strategy when ‘how’ or ‘why’ questions are being
posed, when the investigator has little control over events, and when the focus is on
a contemporary phenomenon within some real-life context. The case study is
especially appropriate when the boundaries between phenomenon and context are
not clearly evident. The case study copes with the technically distinctive situation in
which there will be many more variables of interest than data points, and as one
result relies on multiple sources of evidence, with data needing to converge in a
triangulating fashion.
The case study approach can involve a single event or multiple cases and can be
short or long term. However, rather than requiring total immersion in the setting or
culture, sampling of sites, experiences and/informants is typical. The methods used
in case study research is similar to those of naturalistic inquiry. However the data
collection is often more structured, using key informant interviews, structured
observations of events and interactions and the collection and content analysis of
relevant documents (e.g., to help establish the facts, the assumptions, values and
priorities, or to illuminate differences in perceptions). Case studies often also include
quantitative data for background or to help generate questions to ask informants
(e.g., data on demographics, heath status, utilization, finances, etc.). 10
Structured Observations of meetings. This involves attending meetings of the
group that you wish to research on. This can also be extended to observation of
individuals in their daily work routine or on special tasks. The purpose of observing is

4|of 10
to learn what is going on at the meeting ant witness the group dynamic in process.
This can be a rich information source as it can give researchers insight into the
group.
Content analysis of documents. This is a non-intrusive form of research. This
involves reviewing documents, memos or other pieces of written information for
content and themes. By examining written word, the researcher is studying one type
of communication that occurs in the selected sample.
Focus groups usually explore specific issues. The focus group brings together
individuals chosen to meet a specific profile. They may be homogenous along some
dimensions and heterogeneous along others and a structured, yet informal, setting is
used to explore a limited number of questions. Focus groups, unlike individual
interviews, provide the added dimension of the interactions among members. Focus
groups are often combined with more quantitative approaches such as surveys that
can be administered at different points in the group discussion and even used as
grist for additional discussion.
Cognitive interviews are typically used in survey development. One-on-one
interviews are conducted (with people meeting the criteria for completing) a
particular survey) as the individuals complete the instrument being tested. This
method helps investigators understand how people perceive and interpret language
and their own experiences as they refine the survey instruments.
Mail and telephone surveys are a method of collecting information by sending
surveys via email or post mail. Participants return completed forms to the researcher
or an outside vendor. Surveys may ask respondents to rate items on a scale (e.g.,
Likert scale of 1-5). Some surveys also allow respondents to write their feelings or
attitudes about a particular event or to elaborate in more about a particular even or to
elaborate in more detail on an item, or to express suggestions, etc.
Main Types of Qualitative Data Collection and Analysis
“Those who are not familiar with qualitative methodology may be surprised by the
sheer volume od data and the detailed level of analysis that results even when
research is confined to a small number of subjects” (Myers, 2002).
These are three main methods of data collection:
Interactive interviewing People asked to verbally describe their
experiences of phenomenon
Written descriptions by participants People asked to write descriptions of their
experiences of phenomenon
Observation Descriptive observations of verbal and non-
verbal behavior

 Analysis begins0 when the data are first collected and are used to guide
decisions related to further data collection.

5|of 10
  “In communicating—or generating—the data, the researcher must make the
process of the study accessible and write descriptively to tacit knowledge may
best be communicated through the used of rich, thick descriptions” (Myers,
2002).
Although the approach is holistic, qualitative research observes certain
procedures. Seliger and Sholamy (1980) outline the following:
2. Process of Conducting Qualitative Research
a. Define the phenomenon or problem situation of language to be described.
At the beginning of the study, the focus of the research situation may not be definite,
however, as the investigation progresses, narrowing the scope of the observations
as regards what specific areas of the language activity to be studied is essential.
b. Use qualitative methods to gather data.
Data collection may range from observations, tapes, questionnaires, interviews, case
histories, and field notes. The variety of sources of data provides insights not
available in an experimental method. Moreover, it facilitates “validation and
triangulation” of data.
c. Look for patterns in the data.
Data collected may not have been gathered based on specific research questions or
hypotheses, however, they must undergo careful examination to identify the
recurring patterns arising from them. Based on these recurring patterns, the
researcher establishes hypotheses generates principles, or develop paradigm to
explain the findings.
d. Validate initial conclusions by returning to the data or collecting more data.
Once patterns have been established, the researcher must verify the findings
through triangulation, i.e. confirming the findings by re-inspection or demonstrating
the same findings through different sources. Replication of the study will show
whether there are variables in the initial investigation that might have caused the
language learning behavior different from what it would be if not studied. Because
subjectivity might have influenced the interpretation of the data, these subjective
factors may be controlled by triangulation.
e. Return to step 1 and repeat the cycle to redefine the area of focus based on the
first cycle.
Although the data have been analyzed in the initial stage, it is still important ‘to
redefine’ and to gradually narrow down the focus of the study. This can be done by
re-examining the data or collecting additional data utilizing the same methods but
‘more focused methods.

6|of 10
B. DESCRIPTIVE RESEARCH
1. Scope of Descriptive Research
Descriptive research, on the other hand, can be heuristic or deductive. The
investigation employs existing data or non-experimental research with a
preconceived hypothesis. The researchers begin with a specific focus or general and
specific questions regarding the problem-situation to be studied. Therefore, the
research deals with specific aspects of the available data in the context of language
learning. Kamil et al (1985) claimed that descriptive research differs from qualitative
research on the basis on how data analysis is carried out. Data in the descriptive is
analyzed quantitatively, while quantitative analysis or only done in qualitative
research to determine study. Unlike the qualitative research, a descriptive study may
provide an in-depth analysis of some areas of linguistic ability of the language
learner.
To specify or describe natural phenomena, descriptive methods involve collection
techniques without manipulation of variables. It deals with first hand data or already
existing knowledge taken from previous studies or student records. It seeks ‘what is’
and emphasizes on the discovery of ideas and insights that may affect the
development of generalizations, principles, or theories (Best and Khan 1988).
In summary, descriptive methods when used in language research provide
information and insights related to language development which may serve as bases
for drawing essential universal conclusions about language acquisition and learning.
2. Uses of Descriptive Research
How can the descriptive studies be used in case and group studies, particularly in
second language acquisition? Should a researcher be interested in in describing in
detail the language performance in particular grammatical forms of an individual, a
case study approach is most appropriate. On the other hand, the researcher may
use the group studies in descriptive approach if he is interested in probing deeply the
different factors that motivate a group of learners of second language and how these
factors correlate with their language performance.
3. Tools in descriptive Research
To collect data in descriptive research, Seliger and Sholamy suggest the following
tools:
a. Surveys and questionnaires are useful tools to gather data from large group of
respondents. The questions may vary depending on the relevance of the data
to the study goals. It is also important that the variables of the study should
be evident in the content of the questionnaire.
b. Self-reports and questionnaires are utilized to describe the ‘language status’
of the learner during the period of language learning situations. Since the
study is based on preconceived hypothesis, only identified data may be
selected.

7|of 10
 c. Language tests may take in the forms of writing assignments or
communicative classroom activities are carefully planned to elicit specific data
such as the learner’s proficiency to use wh-questions or any identified
language forms. Standardized test may also be used
d. Observations are used normally to gather data already identified before the
investigation begins. However, the collection od data should be well-planned,
including the design of the observation instrument, to ensure the reliability of
the observations.
4. Observation in Conducting Descriptive Research
The steps in conducting a descriptive study may be summarized as follows: (1)
identify the specific problem which you want to describe; (2) the study; for case
studies one or two will do; (3) decide on the method/s of data gathering based on the
types of data to be elicited; (4) gather data, utilizing the field-tested instrument
chosen; lastly, (5) organize and analyze the data from which conclusions may be
drawn.
C. EXPERIMENTAL RESEARCH
1. Scope of Experimental Research
Like other methods, experimental research begins with a hypothesis-driven
problem. If descriptive research can cause either synthetic or analytic approach,
experimental research must always be analytic in nature. As claimed by (Gay 1976),
the experimental method is the only research design that can truly test hypothesis
concerning cause-effect relationship since investigation is rigorous, intensive and
exacting. The experimental method also involves a more thorough analysis of
variable factors. This type of research also involves manipulation of and control of
variables, specially the independent variables, to measure both internal and external
validity.
Control refers to the efforts of the researcher to get rid of the influence of any
variable other than the independent variable which might influence the performance
of the dependent variable. If the research is experimenting on the effect of the use of
MI-based instruction on the language achievement of the learners, the one group is
given lesson/strategies in the experiment, this should be controlled. Control is
important since the purpose of the research is to determine which treatment is most
effective in the language learning performance of the learners.
2. Major Components of Experiment Research
The Population. An experiment research involves two groups: an
experimental group and a control group. The experimental group receives treatment
under investigation – as in receiving multiple intelligences-based instructional
approach- and the control group receives another treatment or the usual approach
used by the language teacher.

8|of 10
 The researcher, however, should decide whether the experiment will use pre-
existing group or to form a particular group for the experiment, for this aspect
distinguishes the distinction between experiment and quasi-experiment.
The Treatment. This refers to the independent variable in the research, such
as the exposure of the experimental group to the MI-based instructional approach to
determine its effect on the language achievement of the students. The effect of the
manipulation of the independent variable on the dependent variable (language
achievement) is observed or measured.
Measurement or observation may be done by means of a language test or
communicative tasks.
Campbell and Stanley (1963) establish the following components of experimental
designs where:
X – represents an experimental treatment, e.g. exposing a group of students to
instructional materials designed applying metacognitive strategies,
O – symbolizing the observation or the effects of the treatment being measured in
relation to the language achievement of the students
R – refers to the random grouping of the respondents
3. Categories of Experimental Research
Seliger and Shohamy (1989) categorize experimental design into single
group designs where the study involves only one group of respondents; control
group designs involve two groups, where one group which represents the
experimental group is subjected to a treatment, while the other group of the same
population is not; factorial designs involve the investigation of two or more
independent variables and allow the researcher to measure the effect of each
independent variable as well as the interaction of these variable; and quasi-
experimental designs involve only one group in which the investigation is conducted
in conditions which cannot be manipulated.
a. Single Group Designs
 One-Spot Case Study: X= treatment; O= observation.
This research design is considered a pilot study or pre-experimental design, in
which one group receives a treatment (X) and then observed and given a post test
(O). The defect of this design is it is devoid of control over the variables involved.
According to Gay (1976), these designs are problematic since the results of the
study are so questionable, for it does not do much good in controlling threats to test
validity. For this reason, when interpretation of the findings is made, variables that
may have undue influence during the investigation must be given with great caution.
However, in some instances, this design is used since the conditions limit the
researcher to few subjects thereby not allowing randomization of the population. An
example is studying the language acquisition of foreign students, particularly Iranian
whose educational records are not available.

9|of 10
 Example: A second language teacher employs a task-based approach in
enhancing the English proficiency of the learners. After a month of exposing the
subjects to the approach, a post-test is administered to the students. The results
reveal that the learners perform well.

 One-Group Pre-Test-Post-Test Design: O1=; X = treatment; O2

This design is preferred than the single-spot case study design. The group is
exposed to a certain experimental treatment, but before the start of the study, the
group is given a pre-test about the subject matter to be carried out in the
investigation. After the experiment the same group of subjects is given a post-test. It
involves only one group, so it does not need to be compared or matched to another
group. Subjects are observed, or tested twice on the dependent stimuli.
However, one disadvantage of this design is that it cannot be ascertained that
the excess of the results of the post-test over the pre-test is the result of the
treatment (X) Another disadvantage is that there is a possibility that the subjects may
have become familiar with some specific aspects of the treatments (X) and this may
have influenced the result of the post-test (O2) Experts say possible threats are
attrition, history, instrumentation, maturation and statistical regression.
Example: A language instructor would like to determine the grammatical
knowledge and fluency of the students on embedding sentences into word, phrase,
and clause modifiers which becomes the experimental factor. Thus, the teacher
gives the group a pre-test for the target grammatical structure which serves as a
‘base line norm’ of the extent of the respondents’ knowledge on the structure being
measured. After the instruction or the treatment is given, a post-test is administered
to the same group of students.
 Time-Series Designs
Respondents are subjected to a series of observations and measurements
before and after a treatment is administered. This procedure allows the researcher to
disregard any events, such as incidental exposure to a language material outside the
classroom instruction, that occur between the series of measurements over an
extended period of time. The researcher establishes a normal pattern of change
emerging from the series of measurements spaced over several weeks. Hence, any
significant difference or change manifested between the observation before and after
the treatment can be attributed with certainty to results of the treatment.
Example: To investigate the efficacy of the instruction in relation to the
learners’ accuracy and fluency on the correct form and usage of the three degrees of
comparison of adjectives and adverbs, a treatment consisting of lessons on the
subject of study is designed. Before the treatment, the learners are given series of
writing descriptive paragraph exercises in which students’ usage of the language
forms is tallied and measured. Likewise, after the treatment the same group of

10 | o f 1 0
leaners are given writing assignments on descriptive paragraphs and their usage of
adjectives and adverbs are observed and measured. Any noticeable difference
between the observation prior to the treatment and observation after the treatment is
ascribed definitely to result from the treatment.

b. Control Group Design

 The Static-Group Comparison
This is a design in which a group which has experienced treatment is
compared with a group which has not receive treatment, for the purpose of
determining the effect of X (treatment). It involves two groups, one experimental, the
other control, and both groups are given post-test. Although this design allows the
use of language classes that already exist, it also poses problems since threats to
internal validity are maturation, learners’ motivation, or some participants dropout
from the experiment, and even selection interaction of subjects since there is a
possibility that subjects are equal in characteristics.
 The Pretest-Posttest Control Group
This design involves two groups formed by randomization. This procedure
allows a better control of the variables that may influence the internal validity.
Randomization of subjects distribute the effect of extraneous variables equally to
both groups, particularly if the subject population is large. Both groups are pretested
and post-tested, but only one group receives the experimental treatment. The
second group representing the control group prevents the threats to internal validity.
Selection Is taken care by randomization.
c. Factorial Design
Factorial means that two or more independent variables are being
investigated. Factorial designs allow the researcher to measure the main effect of
each independent variable on the dependent variable as well as the interaction
effects of these independent variables (Sevilla, 1999). The use of more than one
variable will make the experiment true and more realistic. Factorial analysis enables
researchers to determine whether underlying pattern of relationships exists such that
data may be 'rearranged' or 'reduced' to a smaller group of components. (Catane,
2000).
Language research oftentimes employs factorial designs with minimal
variations. As cited by Seliger and Shohamy (1989:147), variation is best illustrated
by Flynn's study which attempts to determine the levels of ability of Japanese and
Spanish speakers "to imitate sentences with right or left branching of adverbial
phrase". Flynn's hypothesis was that the 'branching direction' of their first language
would be translated in imitating sentences in English. A pre-test was administered to
the subjects and were classified as beginning, intermediate, and advanced
depending on their level of proficiency. They were also grouped based on their first

11 | o f 1 0
language. Instead of an experimental treatment, the learners were given an
experimental task to imitate a set of sentences to evaluate their proficiency on the
target study. The pre-test establishes the level of the subjects' proficiency before
they were exposed to the experimental task.
d. Quasi-experimental Designs
The design is considered quasi-experimental since there is difficulty in
controlling variables. These designs involve just one group, pre-tested and post-
tested four times. The multi-testing prevents the incidence of maturation, testing, and
regression as threats to internal validity.
Quasi-experimental research is best appropriate for pilot studies and
classroom based research because it is designed under conditions familiar to the
existing situations on real language classroom setting, thus it is met with less
resistance by language teachers

ANALYSIS
Read the research article posted in your subject’s FB page and answer the following
questions:
1. What is the research design of the study?
___________________________________________________________________
2. Explain the features of the study that justify the chosen research design.
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

ABSTRACTION
To sum up what you have learned in this module, draw a generalization on how a
researcher chooses a research design for his/her study?
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

12 | o f 1 0
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

APPLICATION
In module 5 you started your research plan, you were asked to select a research
topic, write specific questions and choose a research design (descriptive or
experimental). To enrich your knowledge, you will justify your chosen research
design, identify the respondents and methods of collecting data.
Specific topic: (Copy your chosen topic in module 5 here.)

Specific questions: (Also copy your specific research questions here)

Research methodology: (also copy your chosen research design)

Respondents:
Based on your chosen design, who will serve as your respondents? How will you seek
informed consent?

Methods of Collecting Data:

What methods of collecting data is appropriate to your chosen research design?

13 | o f 1 0
Now, can you justify your research design? What particular characteristic/s of your
research problem make/s it descriptive, experimental or qualitative?
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________
___________________________________________________________________

ASSESSMENT
Directions: Study the following research problems. Suggest an appropriate research
design and justify the advantages of the suggested research design.
Research Problem What appropriate Why is that the suggested
research design could design advantageous
you suggest? over other designs?
1. The Effect of
Gender Sensitivity
Training on Men’s
Involvement in the
School Activities of
their Children

2. Gender Variations
in the Level of
Motivation of
Second Language
Learners

14 | o f 1 0
REFERENCE/S
Posecion, Ofelia T., Go, Mildred B., Albano, Heide P. (2011) Language Research:
Principles and Applications. LORIMAR Publishing, Inc. Philippines

Prepared by:

Dr. SHIRLEY D. DANGAN

Associate Professor V

15 | o f 1 0