Analyzing Research Articles: A Guide for Readers and Writers 1

Sam Mathews, Ph.D.

Department of Psychology
The University of West Florida
and Visiting Professor
Educational Sciences
Tallinn University

The critical reader of a research report expects the writer to provide logical and coherent
rationales for conducting the study, concrete descriptions of methods, procedures, design, and
analyses, accurate and clear reports of the findings, and plausible interpretations and conclusions
based on the findings. The literature review is generally focused on providing two rationales. The
first is a rationale for the research question or hypothesis that the authors purpose is to generate
evidence to answer an important research question. These questions might reflect an addition of
knowledge to a field by filling a gap in understanding, a resolution of some controversy, or a
practical application of some body of knowledge to remedy some problem. Thus, the first rationale
serves to persuade the reader that the authors selection of a research question is nontrivial and
follows from the existing literature on the topic. The second rationale is intended to justify the
methods used in the study and their appropriateness for generating valid and reliable evidence to
answer the research question posed.
In addition to the two rationales the author presents in the Introduction, the author also
typically presents a Methods and Design section in which the process of conducting the research is
reported. The author then typically presents the Results or Findings section. This is the part of the
study in which the author presents the outcomes of the analyses of the data. While conclusions are
not typically included in this section, the analyses and the results of the analyses should provide a
response to the research question and be consistent with the type of data collected.
The final element of the study appears in the form of the Discussion, Conclusions, or
Interpretation section. The term is frequently dependent on the type of the research and the style of
the publication to which the article is submitted. This section is where the author presents the
implications of the findings. In this case, the author draws on the evidence, findings, and
observations presented in the results section and literature from the field to argue for particular
interpretations or set of conclusions. These should be related to the initial research question.
Additional arguments may be included that concern the relevance of the findings for future
research. In addition, this is the section where the researcher presents potential limitations of the
study.
An important part of a careful and critical reading of a research article is understanding and
evaluating these elements. The more effectively an author builds each of these elements, the
stronger the research paper will be. However, you will find that some authors do not necessarily
present strong, valid arguments for the importance of their research question, their methods, and/or
their conclusions based on findings. So, in order for you to form reasonable and valid arguments in
your own papers, you are well advised to attend to the way other authors craft their arguments. In
your literature review, you are ethically bound to read your sources completely and critically and to
include only those studies that reflect a scientific approach to inquiry. In addition, you are ethically
bound to present divergent perspectives on your topic should you discover such divergence. The
following guidelines reflect one perspective on the standards to which readers might hold authors
of research articles and standards to which authors might hold themselves.

1. Rationale for the research question. (Introduction and Literature Review)

Does the author, in the initial portion of the article, provide an overview of the problem
to be addressed in the study?
This might be a statement of the problem or research question following some introductory
remarks. This helps the reader form a framework about what is to come in the article and
make a decision about whether the article fits the readers purpose. The structure of this
introductory statement of the problem typically provides a sort of outline for the
organization of the introduction. This is characteristic of authors who craft articles that
appeal to readers.

Did the author present evidence of knowledge of the topic of research through references
to and descriptions of relevant and current research?
Knowledgeable authors typically provide evidence in the form of summaries of major
empirical findings and positions on some issue. These summaries flow from one to the next
as the author builds the case for the importance of the research question or problem. The
way the author makes the transition from one topic to the next is also an indication of the
level of knowledge the author possesses about some topic. The summaries are well
organized and the logical pathway from one topic to the next is smooth and obvious when
the author is knowledgeable and the manuscript is prepared and edited with the audience in
mind. While in some cases it is important to present historical perspectives, it is critical that
the author report the most recent findings related to the problem. Incomplete or out-of-date
literature reviews typically receive less than favorable ratings from editors and often lead
the author to use designs that are not the most effective.

Did the author address the major current theoretical perspectives on the problem at
hand?
In addition to relevant and recent empirical studies, if major theoretical issues are being
addressed, authors are well advised to present those perspectives in concise, accurate, and
fair language. Lengthy discourses on a particular theory when a brief summary with
citations for original sources might be sufficient, take away from the impact of the article.

How did the author make the case for the importance of the research question?
Did the author convince you of the importancewhy? Why not?
When you completed reading the introduction, did you feel that the issue(s) addressed
in the study was significant given the topic? You might not find the article focusing
directly on the topic for which you were searching, but given the theme, did the author
make the case for the question or problem being addressed? What was it about the
authors work that convinced you? Authors tend to build the rationale for a particular
issue by answering important questions. A sample of the goals authors have for their
research includes:
filling a gap in existing knowledge.
resolving a conflict between existing theories/interpretations of data.
explaining why conflicting findings have been reported in the literature.
determining the impact of some intervention or treatment.

Does the authors rationale form a coherent argument for the research problem through
the use of sound logic and relevant citations?
Here, the authors arguments for the importance of the question and the authors
conceptualization of the issue are the main points. If the author proposes to address a gap
in the existing knowledge about some issue, has the author presented arguments to support
the existence of a gap in knowledge. If the author is evaluating some intervention or
practical application, has the author provided evidence that the application is based on
empirical evidence and that it addresses some significant problem?

Does the author end the introduction with a concise and focused summary and statement
of the problem or research hypothesis?
Authors who are successful in writing research articles tend to end the introduction and
literature review with a brief summary of the major points and a statement of the specific
question or problem addressed in the study. For quantitative studies, it this is often stated as
a statistical hypothesis. However, whether the study focuses on a qualitative or quantitative
question, the statement of the particular direction of research is critical for the reader. In
most cases this is stated in a sentence or two. As in the initial statement of the problem at
the beginning of the introduction, this provides the reader with an anticipation guide for
reading and evaluating the sections that follow.

Does the Introduction overall make sense logically and empirically?

Upon completing the Introduction, the reader of a well crafted article will have a clear
understanding of the research question or problem to be investigated. The reader will be
convinced that the author has a clear understanding of relevant and current literature and
has presented a clear organization based on that knowledge and logic. Readers, especially
new researchers, sometimes become overly concerned that a particular study familiar to the
reader has not been cited or that a famous person who has conducted research or written in
the area has not been cited. Remember, this is of concern only to the extent that the writer
has, by omitting the particular citation, demonstrated a gap in the logic of the rationale. If
there is a perspective not addressed, that in itself might be a stimulus for the reader to
conduct a study to test that particular perspective.

2. Methods and design of the study used to answer the research question.

This section typically follows the Introduction and Literature Review. The authors task in
this section is to provide a clear and concise description of how the study was conducted.
In this section, the author typically includes several pieces of information. These are most
frequently labeled with subheadings. These sections often include descriptions of the
participants, materials or assessment instruments, any interventions or treatments
administered to the participants, the procedures used to test or evaluate participants
performance in the study, and, frequently, a section describing the strategies used to analyze
the data collected, whether qualitative or quantitative data. The level of description in this
section should be sufficient for a reader to repeat the study with a different sample of
participants. However, lengthy and quite detailed descriptions might best be placed in an
appendix. Another strategy is for the author to provide a footnote noting how a reader
might acquire the more detailed descriptions or more complete materials. In the review
process, however, editors and reviewers may wish to see such documentation.

o ParticipantsThis section will provide the reader with a clear description of the
individuals who participated in the studythose from whom the data were
collected. It is customary in this section to provide data on the characteristics of the
participants that are related to the study. So, for example, this section might include
average age (with standard deviations) for each relevant group of participants (e.g.
males, females; prisoners in minimum security prisons for different types of crimes;
pupils from urban schools or rural schools), frequency data for relevant groups (e.g.
number of males or females; number of prisoners in minimum security prisons for
different types of crimes; number of pupils from rural or urban schools;
proportionate representation of racial or ethnic groups).
The reader is best served when the researcher includes sufficient information to
allow the reader to understand how the participants were selected for the study and
how principles of ethical research1 with human participants were followed (e.g.
participation was voluntary, participants were informed of their rights to withdraw
from the study, anonymity of participants was guaranteed, confidentiality of data
was assured). The reader should find the participants selected for the study and the
strategies for selecting the participants make sense given the rationale in the
Introduction and Literature Review and the research question, problem statement or
hypothesis.
o Materials and assessment instruments used in the studyThis section will provide
the reader with information about any tests, observation checklists, instruments,
surveys, questionnaires, or interview items selected or developed for the study.
Several questions about the materials and instruments should be answered for the
reader. These include:
Are the materials and assessment devices consistent with the research question
or hypothesis?
Do materials and assessment devices match the characteristics of the
participants?
Is there evidence for the validity and reliability of the instruments?
If the author developed the instruments, how did the author determine their
validity and reliability?
How are the instruments or measures used in the study analyzed or scored? The
response to this question should include clear and concise descriptions of
scoring schemes. If the scoring requires multiple raters, the author is advised to
include descriptions for obtaining inter-rater reliability or inter-rater agreement.
For other materials such as videotapes, texts, or any instructional materials designed
for or used in the study, a description that includes how the materials were
developed or selected and their relevance for the research question, problem
statement, or hypothesis are typically provided. The materials themselves may not
necessarily be included, but authors typically provide instructions for the reader to
gain access to the materials.

Procedures for collecting the data and conducting the study--Here the author's
purpose is to provide the reader with a description of the participants' experiences in

the study and the ways the author and other researchers interacted with the
participants. Articles typically include descriptions of the setting. (E.g. in the
participant's home; a classroom in a school; a face-to-face interview in a room
located in a university building) in sufficient detail to allow the reader to judge
whether the setting was consistent with the research question and the characteristics
of the participants. In addition, the, steps in the process of collecting data and
interacting with the participants are also included. Again, descriptions are best
when they are clear, concise, and provide the reader a picture of what occurred in
the research setting (E.g. surveys were given to participants during a group meeting;
participants were asked to read a text and then write what they could remember;
pupils were observed during regular classes).
The author of the study is well advised to provide other information necessary for
the reader to understand how the data were collected and how the participants were
involved in the study Remember, the purpose of this section is twofold. First, the
reader relies upon this section to evaluate the adequacy of the procedures in leading
to an answer to the research question. Second, should the reader wish to repeat the
study, the information within this section will provide adequate information for
doing so.

Analysis of the DataThis section sometimes is placed in the Methods section and
is sometimes included as an introductory paragraph of the Results section. The
decision about where to place this section depends on the flow of the article or the
format favored by the journal selected. One way to make the decision about the
placement of this section is to refer to articles published in the journal to which the
article is to be submitted. Should the work be for a thesis or dissertation, the author
might refer to style guides adopted by the university in question. Regardless, in this
section the author describes the strategies used to analyze the data. Whether the
data are qualitative and in the form of narratives or quantitative data in the form of
measurements or frequencies, the author is well advised to describe in some detail
the strategies for analysis of the data. The authors work is strongest when the
following questions are addressed:
Do the analysis schemes reflect the research question, problem statement or
statistical hypothesis?
Is the analytical plan appropriate for the type of data collected? (e.g. Are the
assumptions required for the analytical schemes met by the data collected?)
When the planned analytical strategies are changed, does the author provide a
rationale for the change and a description of the alternative plan?
Is the sample size reasonable for the analytical strategy?
Do the design of the study and analytical strategy account for rival explanations?

Overall Methods and Design of the Study: Does the design of the study including
selection of the participants, instruments and materials, procedures, and analytical
strategies address the research question, problem statement, or research hypothesis?

Does the design allow the author to rule out alternative explanations when appropriate?
Does the design reflect the nature of the study (description, prediction, explanation,

testing applications)? Does the design of the study follow from the Introduction and the
Literature Review?
3. Results or Findings of the Study
In this section the author reports the results of the analyses of the data, whether quantitative
or qualitative. Ethics and professional responsibility dictate that the researcher make a
complete and accurate report of the findings even when those findings deviate from the
predictions made, perspectives held, and personal and biases held by the researcher or the
sponsoring institution.
Findings are most effectively reported when they are organized around the research
question, problem statement or statistical hypothesis. A clear and unambiguous report of the
findings serves both the author and the reader well. The initial paragraph of the Results
section should contain a restatement of the research question, problem statement, or
statistical hypothesis. This will provide the reader with a guide for reading the Results
section.
Presentation of summary data in the form of tables appropriate for the type of data
collected (e.g. means, standard deviations, frequencies) typically provide an initial
overview for readers of quantitative, and in some cases qualitative articles. Any tables
or figures used to present the data must be discussed in the text. Coupled with reports
of statistical tests, tables allow readers to draw their own conclusions about the
findings. You might refer to particular journal or professional guidelines for format and
presentation of tables in the article.
Typically authors will present the results in the same order in which the key variables
were addressed in the Introduction and in the Methods Sections. Establishing a
consistent order of presentation throughout the article makes the reader's task easier.
When reporting results of statistical tests, it is customary to report the type of test, the
particular groups being compared, the obtained value of the statistic, the degrees of
freedom, effect size, and obtained probability of error.
When statistical tests are conducted and no significance is obtained the author is best
advised to state that clearly and concisely. Typically authors omit the presentation of the
actual test information for non-significant outcomes. Some authors use the Results
section to explain nearly significant differences or non-significant trendsthis
typically weakens the author's article. Such discussions might best be placed in the final
section of the paper (Conclusions or Discussions).
When the research question or problem statements dictate that authors use qualitative
methods, the findings typically follow one of two formats or a combination of the two.
One format is organized around categories developed from the review of the literature
and the other is based on categories that arise from the data (see Levi-Strauss work on
grounded theory).
In the first case, the categories of information that are used to guide the analysis of
narrative or other qualitative data are most convincing when they are derived from
the review of the literature.. The author is ethically bound to include findings that
contradict presence of predicted categories. When presenting the categories, authors
of qualitative studies typically include examples from the data. These are in the form
of direct quotes and include information that defines the context in which the data
were collected and the type of data (e.g. participants own words, authors field

notes). The information must be sufficiently complete to allow the reader to judge
the validity of the categorical membership.
When categories are derived from the data as in grounded theory work, authors
serve the reader well when they present the process for identifying categories and
constructing theories from those categories. In this case, as the author constructs
the Results or Findings section, additional literature review is often presented to
provide support for the categories arising from the data and the authors theory
about the categories. In the case of grounded theory work, authors typically account
for all data. That is, work with the data typically continues until categories account
for the entire body of data.
Providing the reader with sufficient information on the validity and reliability of the
findings of qualitative research is a much different task than with quantitative
research. There are several ways to address this issue. Two are presented here:
Participant checkinghere the researcher returns to participants or a
representative sample of the participants with the emerging categories and
theory to verify that the categories and theory accurately reflect the meaning
intended by the participant. When disagreements occur between the author and
participants perspectives, the author is ethically bound to present both
perspectives and construct a discussion of possible reasons for the differences.
Multiple readers and ratershere the researcher enlists individuals who have
not had contact with the participants to read and categorize the data. Whether
the researcher is using existing theories drawn from research or grounded theory
approaches, once the data have been categorized by the researcher, others are
asked to complete a categorization of the data or a sample of the data. Typically,
inter-rater agreement or inter-rater reliability is presented.

4. Conclusions drawn based on research findings.

What conclusions did the author draw?
Do these conclusions answer the original question posed?
How did the author use the research findings to support the conclusions drawn? That is,
which findings were used to support the conclusion(s) and why do such findings support
this conclusion (rather than some other conclusion).
What other conclusions (if any) might be proposed that would also be supported by these
findings? (Just because an author prefers one set of conclusions does not imply that these
are the only reasonable conclusions that might be drawn.)
How do the author's conclusions compare with findings from other studies?
From the authors perspective, what are the next questions to be answered?
From your perspective, what additional questions come to mind?
These ideas might guide both reading and writing. Given one tenet of the scientific enterprise is
that the work of a scientist, whether in the field of biology, education, chemistry, or psychology is
that the scientific process is available for public scrutiny. The clarity of the written product and the
care with which the arguments are developed support this public scrutiny and ultimately lead to
greater potential impact on the field.
For the purposes of our work, the guide for typical psychological research is found in the

Publication Manual of the American Psychological Association, 5th edition (2001).

Please see the websites below for ethical standards in the treatment of human participants:

American Psychological Association: http://www2.apa.org/ethics/code2002.doc

American Educational Research Association:

http://www.aera.net/about/policy/ethics.htm).