METODOLOGIA EXPERIMENTAL E MNCC – MNP 2023-2024

Niccolò Bonacchi -

ANÁLISE DE DADOS [email protected]

 PROGRAMA
Experimental Methodology
1. Strategy of Experimentation
2. Basic Concepts
3. Guidelines for the design of experiments
4. Some applications of experimental design
5. Major biomedical studies’ types and experimental designs

Data Analysis techniques

1. Descriptive Statistics and Graphics
2. Inferential statistics for quantitative variables
3. Student's t tests for cohort and case control studies
4. Analysis of variance for single factor, factorial designs and longitudinal designs
5. Tests for counts and proportions
6. Tests for orders
7. Correlation and Regression Analysis
CALENDÁRIO
Data Dia Horas Teoria Prática
2024-02-01 Quinta 17h30 -20h30 Apresentação / Conceitos basicos Apresentação / Instalação JASP
2024-02-08 Quinta 17h30 -20h30 Continuação Using JASP / Histograms
2024-02-15 Quinta 17h30 -20h30 Graphics Data Entries and Graphs (Excel based)
2024-02-22 Quinta 17h30 -20h30 Descriptive Statistics, Distributions, Skewness, Kurtosis… Descriptive statistics
2024-02-29 Quinta 17h30 -20h30 Inferential statistics for quantitative variables Single sample t-test
2024-03-07 Quinta 17h30 -20h30 Student's t tests for cohort and case control studies Paired samples t-test
2024-03-14 Quinta 17h30 -20h30 … Independednt samples t-test
2024-03-21 Quinta 17h30 -20h30 Revisões Ficha

2024-04-04 Quinta 17h30 -20h30 Analysis of variance for single factor One way between ANOVA

2024-04-11 Quinta 17h30 -20h30 Factorial designs and longitudinal designs One way repeated ANOVA
2024-04-18 Quinta 17h30 -20h30 Correlations Correlations
2024-05-02 Quinta 17h30 -20h30 Tests for Orders Man-Whitney, Spearman, Kruskal-Wallis...
2024-05-09 Quinta 17h30 -20h30 Linear Regression Analysis Linear/Multiple Regression
2024-05-16 Quinta 17h30 -20h30 Tests for counts and proportions Chi-Square
2024-05-23 Quinta 17h30 -20h30 Revisões Ficha
AVALIAÇÃO
Avaliação Contínua:
 80% Presenças (12/15)
 Ficha intercalar e ficha final (50% + 50%)

Avaliação Exame ou Recurso:

 Exame final compreensivo (2 h)
 Formato escolha múltipla
 Conceitos teóricos e escolha de metodologias (50%)
 Resolução de exemplos práticos (JASP) (50%)
BIBLIOGRAFIA

Montgmorey, D. C. (2009). Design and Analysis of Experiments. 7th. Ed. John wiley & Sons.
London

Navarro, D.J., Foxcroft, D.R., & Faulkenberry, T.J. (2019). Learning Statistics with JASP: A Tutorial
for Psychology Students and Other Beginners. (Version 1/ 2). https://learnstatswithjasp.com/
SUMMARY
- Introdução ao Design Experimental
- Terminologia, estratégias e princípios básicos.
- Populações e amostras
- Tipos de amostragem
- Variáveis
- Fidelidade e validade de medidas
- Tipos de estudos em Psicologia e Neurociências.
- Instalação e familiarização com a interface gráfica do JASP
WHAT IS EXPERIMENTAL DESIGN?
Experimental design is a systematic approach to planning, conducting,
analyzing, and interpreting controlled tests or experiments.
The goal is to determine the effect of one or more variables (factors)
on another variable (response).
It involves carefully considering the selection and assignment of
experimental units, the manipulation of variables, and the control of
other influencing factors.
By using statistical methods, experimental design helps in making valid
and objective conclusions about relationships between variables,
minimizing bias and maximizing reliability and accuracy of the results.
 TERMINOLOGY
Experiment: This is a methodical procedure carried out with the goal of verifying, refuting, or establishing the validity of a
hypothesis. It involves deliberate manipulation of variables (independent variables) to observe the effect on other variables
(dependent variables).

Experimental Unit: The primary object or individual under study in an experiment (N). This could be a person, animal, or any
other object that is being observed or measured. Each experimental unit is exposed to a certain level of the independent
variable(s) and the response (dependent variable) is measured.

Treatment: The specific condition applied to the experimental units. This could be a drug in a medical trial or a teaching
method in an education study. A treatment is a specific level or combination of levels of the independent variable(s) that the
experimental units are exposed to.

Response: The outcome or res. This variable is expected to change as a result of the experimental manipulation. The
response is the dependent variable in the experiment. It is the outcome ult that the researcher is interested inthat the
researcher measures to determine the effect of manipulating the independent variable.

Error: The variability in data points that can’t be attributed to the independent variable(s). Errors can arise from various
sources including measurement inaccuracies, uncontrolled variables, or random fluctuations. Understanding and minimizing
errors is crucial for enhancing the validity and reliability of experimental results.

In an experiment, the researcher manipulates the independent variable (the treatment), applies it to the experimental units,
and then measures the response (dependent variable). The goal is to understand the relationship between the independent
and dependent variables describing and minimizing the error.
 EXAMPLE 1: CLINICAL TRIAL - DRUG
EFFECTIVENESS
Study: "Heterogeneity in Blood Pressure Response to
4 Antihypertensive Drugs: A Randomized Clinical
Trial"1.
Researcher’s Goal: The researchers aimed to
understand the heterogeneity in blood pressure
response to different antihypertensive drugs.
Experimental Unit: The unit of interest in this study
was the patient.
Treatment: The treatments were different
antihypertensive drugs.
Response: The response variable was the blood
pressure of the patients (mmHg).
Error: The error in this study would be the variation in
blood pressure not due to the drugs or other factors.
HOW DO WE GET THIS?
How does one “do” science?
 Some like to start with a problem
 Observe
 Question
 Hypothesize (and operationalize)
 Predict
 Experiment
 Collect data
 Analyze data
 Conclude something
 Communicate it!
 Submit to review and replication by peers
STRATEGY OF EXPERIMENTATION
Planning:
 Define question, treatments, responses, design
Conducting:
 Implement plan, collect data, check quality
Analyzing:
 Summarize data, test hypotheses, estimate effects, draw conclusions

Roughly maps to the Methods and Results sections of a paper

Iterative and adaptive process

Theory vs reality and Murphy’s law

 PRINCIPLES OF EXPERIMENTAL DESIGN
Replication: Reduce error, increase precision, repetition of the basic experiment:
 Applying each treatment to multiple experimental units/groups/.
 Replication allows us to estimate the experimental error and the precision of the treatment effects, as well as to increase the
power of hypothesis tests

Randomization: Avoid bias, validate tests

 Randomization is the random process of assigning treatments to the experimental units. Randomization helps to eliminate bias
and confounding factors, as well as to provide the basis for valid statistical tests. Randomization can be done by using cards,
balls, tables of random numbers, or computer-generated random sequences.

Blocking: Reduce confounding, increase accuracy

 Blocking is the grouping of experimental units into homogeneous blocks, such that the units within each block are similar with
respect to some factors that may affect the response. Blocking helps to reduce the variability within each block and to isolate
the sources of variation that are of interest. Blocking can be done by using physical or geographical factors, such as location,
time, or batch, or by using covariates, such as age, gender, or weight.

 Population? Subjects? How do we select?

SO NOW WHAT?
We have a carefully designed experiment how many and which subjects should I
choose?
Which population?
Which subjects?
SAMPLES AND POPULATIONS
A population is a collection of all possible members of a defined group
 Could be any size

A sample is a set of observations drawn from a subset of the population of interest

 A portion of the population

Sample results are used to estimate the population

So, why would we use samples rather than test everyone?
- Feasibility, cost, time, accessibility
SAMPLING STRATEGIES
Why?
- Ensure that the data collected is representative of the broader population.
- Selecting the right sampling strategy can significantly affect the validity and
applicability of the findings.

Probabilistic sampling strategies vs Non-probabilistic sampling

PROBABILISTIC SAMPLING
Probabilistic sampling methods involve random selection, allowing each member of the
population an equal chance of being included in the sample. This approach facilitates the
generalization of results to the population.
• Simple Random Sampling: Every member of the population has an equal chance of being selected.
Example: Explore the prevalence of migraine headaches across different age groups in a country.
• Systematic Sampling: Every nth member of the population is selected, following a random starting point. This
method ensures that the sample is spread evenly over the population.
Example: Investigate the effects of a new medication on cognitive function in Alzheimer’s patients
• Stratified Random Sampling: The population is divided into strata (subgroups) based on a characteristic, and
random samples are taken from each stratum. This method ensures that specific groups are adequately represented
in the sample.
Example: Examine the impact of early-life stress on brain development, the population of interest is stratified by
socioeconomic status and early-life stress indicators.
• Cluster Sampling: The population is divided into clusters (groups) that are randomly selected. All individuals within
chosen clusters are included in the sample. This method is useful when the population is geographically spread out.
Example: Study parenting styles in rural areas. Divide the area into clusters based on geographical locations
(villages) and randomly select several villages.
NON-PROBABILISTIC SAMPLING
Non-probabilistic sampling methods do not provide every member of the population
with an equal chance of being included.
• Convenience Sampling: The sample consists of members of the population who are easily accessible to the
researcher. While convenient, this method may introduce bias.
Example: All pilot studies in Psychology are about psychology students
• Judgmental or Purposive Sampling: The researcher selects participants based on their judgment about who
would be most useful or representative for the study. This method is often used in qualitative research.
Example: To understand the neural mechanisms of elite athletes’ quick decision-making, researchers might
purposively select professional athletes known for their exceptional performance.
• Snowball Sampling: Existing study subjects recruit future subjects from among their acquaintances. This
method is particularly useful for reaching populations that are difficult to access.
Example: Researching rare neurological disorders might require starting with a few known cases and then
using these individuals to refer others with the same condition, thereby expanding the sample through their
networks.
• Quota Sampling: The researcher ensures that the sample reflects certain characteristics of the population,
based on quotas for subgroups. Unlike stratified sampling, the selection within subgroups is not random.
Example: A study focused on the differences in brain plasticity among musicians and non-musicians might set
quotas for professional musicians, amateur musicians, and non-musicians. Researchers then recruit individuals
to fill these quotas, ensuring a specific composition of the sample to compare across groups.
CHOOSING SAMPLING STRATEGIES
The choice between these sampling methods depends on several factors, including:
•Research Objectives: Whether the study aims for broad generalizability or in-depth
exploration of specific phenomena.
•Population Accessibility: Whether the entire population or specific subgroups are accessible
for research.
•Resource Constraints: The budget, time, and manpower available for the study.
•Accuracy and Precision Required: The level of detail and representativeness needed in the
results.
Probability sampling methods are preferred for quantitative studies aiming for generalizable
results, while non-probability sampling may be more suitable for exploratory or qualitative
research where specific insights or in-depth understanding is the goal.
SAMPLES OF WHAT?
Stuff, anything and everything you can measure!
Statistics == Numbers
Measurements == observations
Observations are expressed in numbers
These numbers vary across different measurements, subjects, contexts, …
We call these Variables!
VARIABLES
Variables
 Observations that can take on a range of values
 An example: Reaction time in the Stroop Task
 The time to say the colors compared to the time to say the word
TYPES OF VARIABLES
Discrete
 Variables that can only take on specific values
 Number of students
 Tricky part … we can assign discrete values to things we’d normally consider words.
 Political party

Continuous
 Can take on a full range of values (usually decimals)
 How tall are you?
MORE CLASSIFICATION OF VARIABLES
Discrete Variables
 Nominal: category or name
 Ordinal: ranking of data

Continuous Variables
 Interval: used with numbers that are equally spaced
 Ratio: like interval, but has a meaningful 0 point (absence of the thing you are measuring)
 Generally described as scale/scalar variables
EXAMPLES OF VARIABLES
Nominal: name of cookies
Ordinal: ranking of favorite cookies
Interval: temperature of cookies
Ratio: How many cookies are left?

The type of number you have / the type of your variable leads to the type of
statistical test you can/should use
VARIABLES
Independent Variables (IVs)
 Variable you manipulate or categorize
 For a true experiment: must be manipulated – meaning you changed it
 Generally dichotomous variables (nominal) like experimental group versus control group
 For quasi experiment: used naturally occurring groups, like gender
 Still dichotomous, but you didn’t assign the group

 Special case: when IVs are categorical, the groups are called levels
 If political party is an IV, levels could be Democrat or Republican
VARIABLES
Dependent Variables (DVs)
 The outcome information, what you measured in the study to find differences/changes based on the IV
 Generally, these are interval/ratio variables (t-tests, ANOVA, regression), but you can use nominal ones too (chi-square)

Confounding Variables
 Variables that systematically vary with the IV so that we cannot logically determine which variable is a
work
 Try to control or randomize them away
 Confounds your other measures!
RELIABILITY AND PRECISION
VALIDITY AND ACCURACY
A reliable or precise measure is consistent
 Measure your height today and then again tomorrow

Standardized tests are supposed to be reliable

A valid or accurate measure is one that measures what it was intended to measure
 A measuring tape should accurately measure height

A good variable is both reliable and valid

 How would we measure this?
TYPES OF VARIABLES AND STUDIES
Variables:
 Discrete/qualitative and Continuous/quantitative (NOIR)
 IV and DV (or confounding)

Studies
 Observational (correlational)
 Prospective, retrospective and sampling
 Cohort or case control studies
 Experimental
 Controls group vs experimental group
 Placebo effects
 Nuisance variables
 Blind designs, (single, double, triple, quadruple)