Guideline Booklet

Research Methodology

Instructor: Jozsef Zoltan Malik, PhD.

1
The course is composed of two classes: a weekly organized lecture and a weekly organized
seminar.

THE GOAL OF THE COURSE:

Research methodology is one of the most basic subjects in the entire curriculums of
universities, regardless of the specialization of the students. All professional knowledge is so
"professional or scientific," and its results are so authentic as its methods make it so. The aim
of this course is to summarize the essentials of research methodology that all students with a
degree in social sciences ought to have today. Achieving this goal includes challenges
because students need to be familiar with the philosophical foundations of methodology; they
need to learn good questioning, research design, data collection and processing.

THE EVALUATION OF THE COURSE:

Seminar (practice):
In the middle of the term, the students select a research topic that interests them and that they
are familiar with. After the approval by the instructor, they will then prepare a presentation
on the topic, in which they will introduce the topic to their classmates and all the
methodological aspects that will be explained in more detail in their Literature Review. At
this phase, each student can consult with the instructor if it is necessary. The presentations
take place at the end of the semester. After then, the students compose their Literature
Review and will submit it to the instructor (Literature Review).
Total Points (1-50 points, at least 25 points are needed for the final exam – from 40 to 50
points, you did a good job, so don’t ask me about this):
1. Activity + Presentation: 50%
2. Literature Review: 50%

Lecture:
After getting the points for the seminar work, student can sign up the exam. Student writes a
final, written exam (via Coospace) at the exam period, which contains some questions
covering the whole subject of the course. There will be ten questions. The first five are short,
multiple-choice test, which probe your knowledge in the professional jargons and concepts at
master's level in Research Methodology. The remaining five questions require longer
answers, and a comprehensive view of the course and a bit of creativity. To reply them,
students always need to write up arguments, that is, not just statements of opinion but
developed reasoning that is intended to lend support to answers.

2
 A SAMPLE OF EXAM QUESTIONS:

1. a. Why is this question not a good research question: “Why do some corporations pollute
water if they are not regulated?”
b. Reformulate it to have a good research question.

2. Here we have a hypothesis: “Media attention is necessary for a candidate to succeed in an

election.”
a. What are the dependent and the independent variables in the hypothesis?
b. How do you operationalise the hypothesis?

3. Design a questionnaire with 5 questions. Name which types of questions did you use for
each question.

4. The aim of the research is to explore contraceptive habits in Hungary, especially

regarding hormonal oral pills. Discuss which research method seems to be the most
suitable for use.

5. What statistics would you use to examine whether there is a relationship between
museum visitors and theatre-goers? Discuss your choice. For the statistical analysis,
formulate exactly the null hypothesis and the alternative hypothesis.

Important Note!
As homework preparation for the exam, students make a questionnaire and a semi-
structured interview guide in advance, the topics can be chosen arbitrarily. Either a
questionnaire or an interview guide is asked at the exam. (Samples can be found in my
Lecture Notes and in Seminar Notes, but, of course, you cannot use exactly the same
questionnaire or interview guide, find out a different topic from one of mine, which your
questionnaire or interview guide may be based on. – Please, be creative in designing them at
home.) After creating your own questionnaire and interview guide, you may just use copy-
paste at this question (it is question No. 3 in the sample exam questions) on the exam.

3
The Fundamentals of Research

4
 Deductive vs. Inductive way of Theorising
Theories, as sets of ideas that attempt to explain aspects of the social world, are central to
your research methodology and the nature of the data you collect. Depending on your
epistemological position you may work with theory in different ways.

If, for example, you take a positivist, or possibly a critical realist, approach you may set out
to test a hypothesis that is based on an existing theory. This is called a deductive approach to
using theory. If, however, you are taking an interpretivist approach, you are likely to start
with research questions (which may be informed by existing theory), and you will then gather
data and derive your explanations – tentative theories – from the data itself; this is an
inductive approach. In most research, a theory is used both inductively and deductively. If we
think of research as both informed by theory and as generating theory, the research process
can be described as a cycle as shown in the Figure below:

For more details and examples, see Seminar Notes 01.

5
 Research Questions
The idea of a question will not be new to you; we become familiar with questions and
answers at an early age. When we want to find out about something, get some information or
read a chapter to help us write an essay, we construct questions to help us to find and then
focus on the information we need. Often this is a process that we do without explicitly
thinking about it, but in the case of research questions, we need to think carefully and clearly
about our questions to ensure that we design research that will enable us to answer the
questions. Basically, if there is no question, you cannot find an answer.
Research questions are central to your project and provide the focus of your research. A
research question, therefore, both states the purpose of your research project and guides you
through the process of research. Obviously, then, it is more than a simple statement because
it must include some detail of the complete nature of the research. This means that research
questions are specific to a particular research undertaking.
Much of what we might believe about things is not empirical, but rather normative – it
reflects our judgments about what should be. Normative questions deal with value judgments,
that is, questions of what is good or bad, desirable, undesirable, beautiful, or ugly.
Though scientific research can deal with normative phenomena, it can do so only indirectly
as it seeks to answer empirical questions; this can be done by taking the normative questions
that motivate our interest and reformulating them as empirical questions.
There are two techniques to use:
• To change the frame of reference:
1. The gist is moving from a normative judgment to a question about the normative
judgements some person or persons make. In practice, this means that we make a
restriction on the scope of the original question to a group of people (or a
country); or restrict the matter of question in time (e.g., How the economics of the
UK developed after the BREXIT?).
• To ask empirical questions about the assumptions behind normative judgements.
1. Most normative judgments are based at least in part on beliefs about what is
empirically true. But are these assumptions true?
2. This way of reformulating is more sophisticated than the first one, it requires a
much deeper analysis.
• Examples (for other examples, see Seminar Notes 01):
1. Would it be a good idea to legalize drugs? (Normative)
✓
Do most Germans/Dutch favour the legalization of drugs? (Frame)
✓
Would the legalization of drugs decrease the occurrence of other crimes?
(Ass.)
✓
How much would the legalization of drugs increase the frequency of
addiction? (Ass.)

6
 2. The United States should continue to send troops to the third world to attempt
to restore order. (Normative)
✓
Nations in the European Union favour the U.S. sending of troops in
most cases. (Frame)
✓
The support of peacekeeping activities with U.S. troops generally has not
resulted in long-term prevention of the disorder in the past. (Ass.)
For more details and exercises, see Seminar Notes 02.

Conceptualisation

The development of a conceptual framework is arguably the most important part of any
research project and the most difficult. And it is the element which is the weakest in many
research projects. A conceptual framework involves concepts involved in a study and the
hypothesised relationships between them. The development of a conceptual framework can
be thought of as involving four elements:

For more details and exercises, see Seminar Notes 04.

Hypotheses
Hypotheses are specific types of research questions which are not questions but rather
statements or assertions about relationships between two or more concepts. The question is
implied rather than stated and could be in the form ‘Can I prove (or disprove) that there is a
relationship between these two concepts?’ The purpose of the research then is to test the
asserted relationship with a view to proving or disproving it.

7
More Specifically:
• A theory is a set of empirical generalizations about a topic, but its statements are too
general to test directly and the investigated relationship between its abstract concepts
is complex and not directly observable. → Task: we must break it down into more
specific terms.
• This is done by testing hypotheses (educated guesses based on observation). In the
research process, hypotheses are statements about variables.

• Indicators are phenomena which point to the existence of the concepts. Variables are
components of the indicators which can be measured. So, a variable is an empirical
property that can take on two or more different values.

Hypotheses are most used to look at relationships between indicators:

Example:

This hypothesis asserts that there is a relationship between ethnicity and committing crimes.
We could say that the research question is: Is there a relationship between ethnicity and
committing crimes?
In the relationship, the cause is ethnicity, which is the independent variable, and the effect is
the crime rate, which is the dependent variable.

There are two main types of hypotheses:

8
Statistical Hypotheses
In the quantitative method, we use specific forms of hypothesis:
A common feature of the statistical method is the concept of the null hypothesis, referred to
by the symbol H0. It is based on the idea of setting up two mutually incompatible hypotheses
so that only one can be true. For example: Either more people play tennis than golf or the
number of people who play tennis is less than or equal to the number who play golf – if one
proposition is true then the other is untrue.
The null hypothesis usually proposes that there is no difference between two observed values
or that there is no relationship between variables. There are therefore two possibilities:
H0 – Null hypothesis: there is no significant difference or relationship
H1 – Alternative hypothesis: there is a significant difference or relationship.

Note! Usually, it is the alternative hypothesis, H1, that the researcher is interested in, but
statistical theory explores the implications of the H0 hypothesis.

Example: We are inspecting if there is a relationship between museum visitors and theatre-
goers. The standard null hypothesis is there is no relationship between them. However, we
probably favour the outcome that there occurs a statistically significant relationship – so this
alternative hypothesis is the one that we are really interested in and want to point out in the
research. That’s why we are going to make a survey and then a statistical test.

Operationalisation
Before you can produce research questions or hypotheses that will help you to design your
research and choose your data collection methods you need to unpack your initial questions
to ensure that you know exactly what you are researching and that you will be able to gather
data that will help you answer them. This involves creating some operational definitions.
Alongside this, you are advised to begin to review the literature on your research topic as this
will help you with the process of operationalising your definitions, refining your research
questions and, probably, developing some subsidiary research questions.

The purpose of the operational definition is to give you a clear indicator so that you can tell
when the thing you are researching happens. Most research questions include and are about
concepts, the way we want to find out more about them and the way they relate to each other.
Concepts are ideas that can be defined or understood differently by different individuals or
groups or at different times, so it is essential that you have a clear definition of the concepts
you use, and that the definition is workable for your research – it is operational.
An operational definition:
• must be valid for and specific to the research – it must be able to be used to gather;
• data to help you to address your research question;
• is always context-specific, designed for each research project and may be of no
use in other research projects.

9
Example:

The research whose hypothesis we can see above in the box is looking at the relationship
between two concepts – unemployment and mental health. Before we can develop our
research, questions and design our research, we need to create operational definitions of these
two concepts.
In this research what do we mean by being unemployed? Here are some suggestions:
1. An adult who is not in paid work.
2. An adult who has been in paid work but is not now.
3. An adult aged between 16 and 65 years who has been in paid work
but has not been in paid work for more than one year.
4. An adult who is receiving unemployment welfare benefits – ‘on the dole’.
5. An adult who is not in paid work and is looking for paid work – ‘signing on’.
6. An adult who does not work.

We need to know the focus of the research and the research question before we can choose
the definition that is going to be used. For example, if the research question is: “What are the
impacts of long-term unemployment on people’s mental health?” operational definition No. 3
would be appropriate and could be used to identify a suitable sample of research participants.
Operational definitions are not only used to help researchers refine their research questions.
They are also necessary when we begin to think about how we will gather data to address our
research questions. We need to be able to gather the data in terms of our operational
definitions.
Examples:

10
The definition of the participant in tourism uses a very general description saying that “it is a
person who travels away from home for leisure purposes.” The operational definition
specifies this by giving specific definitions: ‘travel at least 40 km’ and ‘staying (there) at least
one overnight in preceding 3 months.’
In this spirit, we can speak about the operationalisation of hypotheses, either:

In this example, “person’s income” as the independent variable and “vote” as the dependent
variable is operationalised as to where to find the relevant data that we can observe (and how
to measure) them.

For more details, see the Lecture Notes, and Seminar Notes 04 and 05.

11
 Research Designs
For details, see the second part of the Lecture Notes (rm_basic.pdf)

12
The research aims to collect information, identify meaningful patterns, and draw general
conclusions based on the gathered information. We can study them along several dimensions
and classify the aspects by which the research goal can be achieved.
1) Inductive and deductive way of theorising – we have already discussed this dimension
beforehand.
2) Unobtrusive methods, of which two fundamental forms are used in the social sciences:
 Historical Research Design: Social phenomena are considered a result of history.
The study of the historical background and the presentation of the geology of the
cases therefore always appear in research in some form.
 Deductive-analytical generalization: in this case, we do thought experiments, and
create models, by basing on theoretical concepts or relationships, rested on
processing and further thinking of the existing literature. We often use case
studies, comparative studies, and content analysis for this.
3) Comparative Research Design: There are strict limits to real-life experiments in socials;
e.g., no labs for social researchers to thorough observations about a regime transition or a
revolution. What we can really do is to compare facts, events or processes that have been
emerged in society.
 Transferability of some paradigmatic or deviant patterns to other cases,
situations or contexts: empirical-qualitative comparative research is based on
this approach.
 Generalization from a carefully prepared sample to the whole population
rested on statistical methods: empirical-quantitative comparative research is
based on this set of methods.
4) The Correlation Designs require only collecting data on an independent and a dependent
variable and determining whether there is a pattern of relationship.

Quantitative vs. Qualitative Research Design

In the Lecture Notes, the key slide is the one below where you can see the most important
features and a comparison between them:

13
Another important terminology is the time dimension of the research, here is some
professional jargoon that we use:

14
 The “Methodology” of Data
Research is about making arguments and supporting those arguments with evidence. The data
that we collect provides an information base for these arguments. If the evidence is bad, then
the argument is misleading. In order to have confidence in our results it is important to ensure
that our concepts and our data are as good as possible. The aim of this topic is twofold:
1. To present the notion and the idea of validity and reliability by which we can assess
conceptual framework: whether we have chosen the suitable indicators (variables) of
our concept (validity), and how accurately we have measured our indicators
(reliability)?
2. The classification of data (Categorical – Scale; Nominal, Ordinal, Interval, and Ratio
variables).

Sample and Sampling

For more details and examples, see Seminar Notes 06.

It is necessary to sample in most surveys and in some observational research. Mainly because
of costs, it is not usually possible to gather data from all the people, organisations or other
entities which are the focus of the research.
The total category of subjects which is the focus of attention in a particular research project is
known as the population. A sample is selected from the population. The use of the term
population makes obvious sense when dealing with communities of people – for instance
when referring to the population of Hungary or the population of Budapest. But in social
research the term also applies in other instances; for example, the visitors to a resort over the
course of a year constitute the population of resort visitors; and the users of a sports facility
are the population of users.
We discuss some theoretical and practical problems of sampling such as the types of
sampling; designing a questionnaire; which type of questions can be used in questionnaires;
or, and how to code them.

15
 Research Methods
For details, see the Lecture Notes.

16
This is the second part of the course where we learn the most important research methods
applied in Socials. When you make or join a research project, these are the most typical
methods by which you discuss, interpret, or test your findings. The basic requirement is to
recognise which method is needed for the purpose of the different research projects,
understand the logic of the methods, and be aware of their validity and reliability.

The relevant research methods that we have learnt:

1. Secondary Research Method:
 Historical Research that involves the investigations of events that either has
an important impact on subsequent developments, or provide an opportunity
of testing the implications of a general theory. Three main types of Historical
Research: 1. Historical Events Study: We have one case and one period of
time. 2. Historical Process Research: We have one case but many time
periods. 3. Comparative Historical Research: We have two or more social
settings or groups (e.g., countries) and our aim is to infer something from the
comparison.
 Deductive-Analytic Method: a standard secondary research method based on
the processing of the literature and the interpretation of secondary (published)
data.
 Example 1: Adapting an abstract financial/macroeconomic/
econometric (mathematical) model to a concrete stock
market/business/economic policy investigation. Maybe with
computer modelling.
 Example 2: Examining statistical sources and databases. Analyzing
company data and figures in a business context.
2. Case Studies provide descriptive contextualisation, apply existing theory to new
contexts, examine expectations of the rule, and generate new theories.
3. Comparative methods: To make a comparison between two or more groups of
agents with their indicators. There are two different camps of comparisons: empirical-
quantitative and empirical-qualitative methods (see later).
4. Qualitative Methods:

17
5. Content Analysis (There are Exercises in Seminar Notes 07): Content analysis was
developed in the early twentieth century and was first used for the analysis of
newspapers, Later it was applied to propaganda, particularly during World War II. It
has been used by researchers in many fields, including literature, linguistics, history,
communications, and education as well as all of the social sciences. Content analysis
can also be used for processing interview transcripts, or in conjunction with answers
to open-ended questions on surveys, or interpretation of visual information (e.g., adds
or video clips).
6. Quantitative Methods (There are Exercises in Seminar Notes 08 and 09):
Quantitative methods can be divided into two large groups: 1) Descriptive Statistics,
2) Inferential statistics with testing first and second type of hypotheses.
Descriptive statistics is a method to characterize the data in order to make decisions
about the nature, tendencies. In descriptive statistics we look at various measures such
as mean, median, mode, and standard deviations. Inferential statistics are used to
analyze the data. The tests all relate to relationships between variables (First Type of
Hypothesis) and comparisons between variables (Second Type of Hypothesis).

1) First Type of Hypotheses: We assume that there is a relationship between

two variables (independent and dependent variables), and the question is
whether this is a causal (cause-and-effect) relationship or just an association
that can be attributed to chance. We can use two statistical tests for this:
 The Chi-square test: The Chi-squared is a test that can be used to assess
whether the difference between the mean values of two samples is
statistically significant (by significant we mean worthy of consideration
and note). It can be used to compare variables that are nominal or ordinal,
for example, male and female.

 Linear regression that is a statistical model describes the relationship

between interval variables by fitting a line to the observed data.

2) Second Type of Hypotheses: The logic of the second type of hypothesis is that the
dependent variable has been changed and we study the cause of the alteration (before
and after the change), whether the difference was statistically significant or was due to

18
 chance. So, here we have a comparative research study. If our sample is appropriate
(large N-sample, i.e., the size of the sample is large enough or at least meet the
requirement for normality), the statistical test that we can use is t-test (or if we have
more than two variables, the so-called ANOVA). If our sample is small, we use
qualitative methods for the comparison (MSSD, MDSD):

Briefly on the Reliability of Statistical Tests: The Chief Ideas

Much of inferential statistics is based on measurements and manipulations of the means.
When we have some sample data the measures of central tendency, such as mean, median,
and mode, are very simple to calculate and compare. When these measures are compared
across categories we begin to find differences in their means.

Confidence Interval
Confidence Interval measures the degree of uncertainty or certainty in a sampling method.
They can take any number of probability limits, with the most common being a 95% or 99%
confidence level. The confidence interval is a calculation of the range of values for the true
mean. We can know with a certain amount of “confidence”, typically at the 95% confidence
level, that the true mean will within the specified confidence interval.

For example, we may find that the mean of our

sample is 52.65 for some measure. The
calculated confidence interval could be from
51.34 to 53.95 for example. Therefore, given
that the sample mean is 52.65, we can state with
95% confidence that the true mean lies
somewhere between 51.34 and 53.95 for the
population.

19
P-Value
What is a P-value? In statistical analysis the way we measure the significance of any given
test is with the P-value. This value indicates the probability of obtaining the same statistical
test result by chance. Our calculated p-values are compared against some predetermined
significance level. The most common significance levels are the 95% Significance Level,
represented by a p-value of 0.05, and the 99% Significance Level, represented by a p-value of
0.01. A significance level of 95%, or 0.05, indicates that we are accepting the risk of being
wrong 1 out of every 20 times. A significance level of 99%, or 0.01, indicates that we risk
being wrong only 1 out of every 100 times.

The most common significance level used in the Social Sciences is 95%, so we are looking
for p-values < 0.05 in our test results. However, in statistical analysis we are not looking to
prove our test hypothesis with the p-value. We are actually trying to reject the Null
Hypothesis.

When we perform a statistical test, the p-value helps determine the significance of the test
and the validity of the claim being made. The claim that is always “on trial” here is the null
hypothesis. When the p-value is found to be statistically significant, p < 0.05, or that it is
highly statistically significant, p < 0.01, then we can conclude that the differences,
relationships, or associations found in the observed data are very unlikely to occur if the null
hypothesis is actually true. Therefore the researcher can “reject the null hypothesis”. If you
reject the null hypothesis, then the alternative hypothesis must be accepted. And this is often
what we want as researchers.

The only question that the p-value addresses is whether or not the experiment or data provide
enough evidence to reasonably reject null hypothesis. The p-value or calculated probability is
the estimated probability of rejecting the null hypothesis of a study question when that null
hypothesis is actually true. In other words, it measures the probability that you will be wrong
in rejecting the null hypothesis. And all of this is decided based on our predetermined
significance level, in most cases the 95% level or p < 0.05.

Effect Size
Whereas statistical tests of significance tell us the likelihood that experimental results differ
from chance expectations, effect-size measurements tell us the relative magnitude of those
differences found within the data. Effect size is not as sensitive to sample size since it relies
on standard deviation in the calculations.

Suggested Effect Size Magnitude Chart (In Behavioral Sciences the magnitudes are concessive)
Effect Size Calculation Statistical Test Small Medium Large
Effect Effect Effect
Cramer’s Phi Chi Squared 0.1 0.3 0.6

t-Test (Paired 0.2 0.5 0.8

Cohen’s d
t-Test independent)

Eta Squared ANOVA 0.01 0.06 0.14

Omega Squared

20
 Effect Size Calculation Statistical Test Small Medium Large
Effect Effect Effect
r Correlation 0.1 0.3 0.5

Correlation and
0.01 0.09 0.25
r2 t-Test (Independen)
Values from Cohen (1988) Statistical Power Analysis for the Behavioral Sciences

Videos:
This is a folder that consists of six videos.
1) Chi-Square_video: This video presents how to test if there is a relationship
between dependant and independent variables providing that one type of variables is
categorical, i.e., nominal or ordinal. In the video the instructor presents how to carry
out chi-square test step by step in Excel. As Excel doesn’t have built-in statistical tool
for Chi-Square Analysis, the user needs to create the necessary tables (the so-called
cross-tables) for calculation. In the video the instructor uses three tables (Observed,
Expected, and Chi-Square table), in my Excel spreadsheets I just use two, for the
third, it’s enough to use Excel functions:

2) Mixing up association and correlation: In developing the hypothesis, we make a

relationship between dependent and independent variables. Suppose we have two
arrays of data that show a positive or a negative relationship. If we can use an
established theory (the deductive way of theorising) or can create a theory on
experience (the inductive way of theorising) that explains the relationship, meaning
that causation (independent variable as cause and dependant variable as consequence)
can be found between the data, we can call this relationship correlation, otherwise
association. Statistics are for testing if the relationship is statistically significant (the
correlation of data), or if it is just due to randomness (the association of data). This
video presents examples of why it is dangerous, and we may have misleading findings
if association and correlation are mixed up.
3) Regression_video: In this video, we can see statistics that evaluate the relationship
between scale, i.e., measurable variables (interval or ratio). The statistics are derived
from a procedure called regression. Regressions are statistical models that describe the
relationship between variables by fitting a line to the observed data. The video
presents the point of this method and how to make scattergrams and test correlation
and regression in Excel. Contrary to the Chi-Square test, Excel gives a built-in toolkit
for carrying out this statistical analysis.
4) Semi-Structured Interview presents the main points of making an interview in
practice.
5) Simpson’s Paradox presents why it is important in statistical studies to fix the unit
of analysis and don’t mix it up because it might imply contradictions.
6) t-test: In this video you can watch how to run t-tests in Excel.

21