Research Methodology in
Research Methodology in
R e c u r re n t Preg n a n c y L o s s
a,b,
Ole B. Christiansen, MD, DMSc *
KEYWORDS
Recurrent pregnancy loss Recurrent miscarriage Medical research
Research studies Methodological flaws
KEY POINTS
There is currently substantial disagreement concerning the diagnostic criteria for recurrent
pregnancy loss (RPL), which renders comparisons between research studies in the area
difficult.
There are numerous methodological pitfalls that threaten the validity of research studies in
the field of RPL and it is necessary for scientists and clinicians to be aware of them.
Some of the methodological pitfalls are common for medical research in general, whereas
some are specific for RPL research.
Frequently seen methodological flaws in case-control studies are comparisons of bio-
markers between patients with RPL and controls that differ from patients with regard to
previous ongoing pregnancies, relevant endocrine factors, and viability of the fetal tissue
at the time of sampling.
In cohort studies, incomplete follow-up of patients in many studies has resulted in huge
variations in estimates of the prognosis after RPL.
Only a few small and heterogeneous double-blinded placebo-controlled trials of treat-
ments of RPL have been carried out with very heterogeneous results.
Proposals are given for improvements in the design of research studies in RPL that hope-
fully can improve the quality of studies in the future.
INTRODUCTION
Compared with the situation in other reproductive medicine disorders, such as tubal or
male factor infertility and in other areas of medicine, there is very little consensus
about which investigations are useful for identifying causes or estimating the prog-
nosis and which treatments are effective in recurrent pregnancy loss (RPL). It is gener-
ally agreed that when the tubes are occluded, as diagnosed by laparoscopy or
a
Fertility Clinic 4071, Copenhagen University Hospital Rigshospitalet, Blegdamsvej 9, DK-2100,
Copenhagen, Denmark; b Department of Obstetrics and Gynecology, Aalborg University Hospi-
tal, Reberbansgade, DK-9000, Aalborg, Denmark
* Fertility Clinic 4071, Copenhagen University Hospital Rigshospitalet, Blegdamsvej 9, DK-2100,
Copenhagen, Denmark.
E-mail address: [email protected]
CONTROVERSIES OF DEFINITION
It is disputed how to define RPL. It is important to realize that RPL is defined quite
differently from most other diseases. Most diseases are defined by some unique path-
oanatomical, clinical, or paraclinical findings being permanently present, whereas RPL
is defined by a series of transient events in the past that may have be poorly registered.
The controversies concerning how to define RPL deal with
The number of miscarriages needed for the diagnosis
The role of nonconsecutive miscarriages
The role of preclinical losses
Until 10 years ago, the definition of RPL was undisputedly 3 or more consecutive
miscarriages, because it was commonly agreed that after 3 miscarriages the chance
of live birth the next pregnancy without treatment was substantially decreased.6 How-
ever, during the recent years, some national guidelines have adopted an RPL definition
of only 2 clinically recognized miscarriages1 or 2 not necessarily consecutive miscar-
riages.7 This redefinition is based on finding similar frequencies of selected factors
suggested to cause RPL: uterine abnormalities; antiphosphospholipid antibodies
Research Methodology in Recurrent Pregnancy Loss 21
Different pitfalls characterize the 3 main types of research studies done in RPL:
Case-control studies
Cohort studies
Intervention and treatment studies
For each category of studies, I provide an overview of the pitfalls that threaten the
validity of the studies and the flaws often seen in publications: some of them can be
22 Christiansen
seen in other areas of medical research and they are discussed superficially, whereas
others that are specific for RPL research are discussed in more detail.
CASE-CONTROL STUDIES
Table 1
Important methodological pitfalls typical for research studies in RPL
Abbreviations: CCS, case-control study; RCT, randomized controlled trial; RPL, recurrent pregnancy
loss.
Ascertainment bias
In case-control studies, the frequency of a potential risk factor for the disorder under
study (eg, RPL) is compared between patients and controls. Ascertainment or selec-
tion bias happens when patients with some clinical or paraclinical risk factor are pref-
erentially referred to a specific clinic because of knowledge of the clinic’s expertise
or interest, and a study focusing on this particular risk factor is undertaken in the
clinic. An example of ascertainment bias in RPL has been reported by Out and col-
leagues.14 In this study, the frequency of the APLs anticardiolipin and lupus antico-
agulant was higher in patients with RPL referred to a Dutch center for APL research
than in controls. However, when patients with RPL with a history of thromboembolic
or lupuslike symptoms were excluded, the prevalence of APL in the remaining pa-
tients with RPL did not differ from that of controls. The high prevalence of APL in
the total group of patients resulted from the preferential referral of patients with
RPL with APL-associated symptoms to the clinic due to its special expertise (see
Table 1).
Ascertainment bias can also work in controls. In studies of risk factors for adverse
pregnancy outcome, information is often obtained or blood samples drawn from
women with healthy pregnancies coming for routine pregnancy control. Women giving
24 Christiansen
informed consent to become controls are often better educated than average women,
thereby decreasing the occurrence of lifestyle factors and exposures that may be
harmful in pregnancy. When staff members or healthy blood donors are used as con-
trols, these also undergo positive selection for being healthier than the average
population.
Confounding
In case-control studies, a confounding factor is a clinical or paraclinical factor that is
associated with both the risk factor and the disease/outcome under study. If adequate
measures are not taken, a confounding factor can be mistaken as a causal factor or
diminish the estimate of the impact (OR) of the risk factor under study. In RPL studies,
age is an important and common confounding factor. In studies of the prevalence of
autoantibodies in patients with RPL and controls, age is associated with both the risk
of miscarriage and RPL and the occurrence of autoantibodies. Elimination of the con-
founding effect of age during the inclusion phase of such studies can be undertaken by
age-matching patients and controls and in the analysis phase by reporting autoanti-
body frequencies in different age strata of patients and controls and subsequently
do adjustment by multivariate statistical methods.
Genetic
Nongenetic Biomarkers Biomarkers
Peripheral Blood/
Peripheral Blood Luteal Phase Endometrium Decidual/Trophoblast Tissue Tissue
RPLa Controlsa RPLb Controlsb RPLc Controlsc RPL Controls
In each double column, the suggested optimal controls are found to the right of the RPL column.
Abbreviations: GW, gestational week calculated from the first day of last menstrual period; NP, not pregnant; RPL, recurrent pregnancy loss.
a,b,c
Women in each comparable pair should ideally have similar estrogen and progesterone levels in the nonpregnant state and similar hCG, estrogen, and pro-
gesterone levels in the pregnant state.
25
26 Christiansen
In studies of external exposures or genetic biomarkers, the ideal control group for
patients with RPL is women with proven fertility: typically women with 2 or more un-
complicated births and no miscarriages, as genetic polymorphisms will not change
according to reproductive history or be affected by endocrine factors or inflammation
(see Table 2).
However, it is much more complex to identify the ideal controls for the investigation
of nongenetically determined biomarkers in the blood, the endometrium, or tropho-
blast tissue. Some biomarkers, although exhibiting no impact on pregnancy outcome,
may be different in patients with RPL and women with previous births merely because
of their different reproductive histories.
An illustrative example is anti-HLA antibodies and studies of their role in RPL.
Several years ago many articles reported that these antibodies could be detected
much less frequently in the blood of patients with primary RPL (no previous ongoing
pregnancies) than in control women who had previously given birth. It was postulated
that lack of these “blocking antibodies” was a cause of the RPL and that deliberate
immunization of the patients with paternal lymphocytes with the aim to stimulate anti-
body production would improve the pregnancy prognosis. It is now generally recog-
nized that these antibodies are a common feature of normal ongoing pregnancy
due to passage of fetal cells through the placenta into the maternal circulation during
late pregnancy and they often persist for many years.15 Therefore, it is not surprising
that patients with primary RPL normally lack these antibodies, whereas multipara are
often positive.
It has also been shown that cellular immunity is being permanently changed after a
birth. Clones of cytotoxic lymphocytes with specificity for male-specific minor histocom-
patibility (HY) antigens develop in half of the women pregnant with a male fetus and the
anti-HY cytotoxicity remains unchanged up to 18 years after delivery.16 Long-term
persistence of regulatory T cells17 or/and persistence of fetal cells in the maternal circu-
lation (fetal microchimerism) after a first ongoing pregnancy18,19 may be the reason that
most women remain immunologically tolerant to the fetus in spite of production of anti-
bodies and lymphocytes with reactivity toward fetal antigens. The maternal immune sys-
tem therefore recognizes many paternal/fetal alloantigens during an ongoing pregnancy
and this very often induces permanent changes in immune reactivity to the fetus or
trophoblast that may reside in lymphocytes carrying immunologic memory (memory
T cells) in the peripheral blood, endometrium, or regional uterine lymph nodes. It is
also possible, although much less studied, that transcription of messenger RNA
(mRNA) and expression or production of proteins that are not related to the immune
function (eg, receptors for hormones in the uterus, production of coagulation factors
in the liver) can be permanently altered subsequent to the extensive physiologic changes
taking place during a prior ongoing pregnancy (a pregnancy passing GW 22).
Because of these considerations, in RPL case-control studies controls should be
matched to patients with RPL regarding a history of previous ongoing pregnancies
(see Table 2). The ideal control group for patients with RPL who had never had an
ongoing pregnancy would be women with repeated first trimester terminations due to
social reasons; however, because this group is fortunately small, an alternative suitable
control group would be women with no previous pregnancy. It may be argued that 1%
of nulligravida would later experience RPL, but this small error is insignificant compared
with the error associated with comparing nulliparous patients with RPL with multiparous
controls. Patients with secondary RPL who had previously had a birth should of course
be compared with controls with 1 previous live birth and no miscarriages.
Another factor that can confound case-control studies in RPL research and should
be adjusted for as much as possible is differences in hormones relating to
Research Methodology in Recurrent Pregnancy Loss 27
reproduction and pregnancy. Many factors relating to immune function (eg, cytokines)
and the coagulation system (eg, proteins C and S) are influenced by estrogen and pro-
gesterone levels.20 The level of expression of a series of cytokine mRNA in endometrial
cells increases markedly from the follicular to the late secretory phase, probably influ-
enced by cyclic changes of estrogen and progesterone.21 It has been shown that lym-
phocytes that can induce immunologic tolerance (regulatory T cells) are attracted to
the feto-maternal interface by hCG produced by the trophoblast,22 which may have
profound importance for measurements of immune biomarkers in the uterus during
pregnancy. Because concentrations of hCG, estrogens, progesterone, cortisone,
and pregnancy-associated placental protein A (PAPP-A) change markedly according
to menstrual cycle phase or progressing gestation, the level of biomarkers affected by
hormones is dependent on phase of menstrual cycle or time of gestation. Therefore,
as a main rule, nongenetic biomarkers should be investigated in patients with RPL
and controls matched by menstrual cycle phase in nonpregnant women and length
of gestation during pregnancy (see Table 2).
A further problem arises when biomarkers in the blood or uterus, which are affected
by hormones, are investigated in patients with RPL just before or at the time of miscar-
riage and compared with similar measurements in controls with a healthy ongoing
pregnancy. Most of the pregnancy-related hormones, hCG, estrogen, and PAPP-A,
decrease in threatened miscarriage and finding differences in a specific biomarker be-
tween women with a miscarriage or a healthy pregnancy, respectively, may be
severely confounded by the fact that hormones in the former group are lower than
in the control group.
A last but important factor that may confound case-control studies in RPL research
is differences in the viability and inflammatory status of the uterine content before or at
the time of miscarriage in patients and at the time of induced abortion in controls. At
the time of embryonic death, the trophoblast will undergo necrosis and intrauterine
hemorrhage will often induce inflammation in the decidual tissue, which can be re-
flected in measurements of immunologic biomarkers in the blood or decidual/tropho-
blast tissue. In contrast, the same biomarkers in the blood of controls with an ongoing
normal pregnancy or in the blood or decidual/trophoblast tissue from controls with an
induced abortion on social indication will not be influenced by inflammation. Unfortu-
nately, in many publications, levels of biomarkers associated with immune function or
apoptosis (programed cell death) in the blood or decidual/trophoblast tissue are
compared between women with a missed abortion and women with a normal ongoing
pregnancy or women undergoing induced abortion. In many of these publications,
finding differences in levels of such biomarkers are interpreted as proof that changes
in maternal immune reactions or apoptosis are causing miscarriage or RPL. To avoid
detecting biomarkers in patients with RPL that differ from those in controls merely due
to processes being a result of rather than a cause of miscarriage, measurements in pe-
ripheral blood should be undertaken in very early pregnancy (eg, GW 5) at a time when
the feto-placental unit is very tiny and not expected to affect systemic inflammatory
responses and before the confounding effect of declining hormones in patients take
place (see Table 2).23
Another approach to counteract the methodological problem associated with the
comparison of necrotic and vital tissue is to do karyotyping of embryos from missed
abortions in women with or without RPL. Biomarkers in the blood or decidual/tropho-
blast tissue can then be compared between patients with euploid male embryos (to
avoid erroneous karyotyping of maternal tissue) and patients with embryos with a
chromosome abnormality that definitively will cause early embryonic death. Levels
of biomarkers that are influenced by embryonic and trophoblast necrosis and
28 Christiansen
inflammation may be equally affected in women with euploid and aneuploid miscar-
riages. Therefore, differences in expression of biomarkers between the women/em-
bryos from the 2 groups can probably be attributed to factors that may have
caused euploid miscarriage. Examples of such studies that point to a causal role of
inflammatory cytokines in RPL is the study by Calleja-Agius and colleagues,24 finding
significantly higher plasma tumor necrosis factor-a, interferon-g, interleukin (IL)-6, and
IL-10 levels in women with euploid miscarriages than in healthy pregnancy, whereas
these cytokines were not increased in women with aneuploid miscarriages. Another
study similarly found increased numbers of activated leucocytes in RPL women
with a miscarriage with an euploid compared an aneuploid embryo.25
As illustrated previously, finding a suitable control group for measurement of nonge-
netically determined biomarkers is a difficult task. Adherence to the recommendations
given previously (see Table 2) may diminish the risk of conducting a case-control
study that is methodologically flawed but offers no guarantee. The factors that should
not differ between cases and controls in RPL research are the following:
Number of previous ongoing pregnancies
Levels of estrogen and progesterone in nonpregnant women
Levels of estrogen, progesterone, and hCG in pregnant women
Degree of viability of decidual or trophoblast tissue
I realize that very few case-control studies in RPL meet these criteria, especially the
criteria of similar hormonal levels. More research should be done regarding genetically
determined biomarkers because these are robust to the confounding effects dis-
cussed in this section. It is possible that some nongenetic biomarkers may also be
robust to the effects of the mentioned confounding factors (eg, reproductive hor-
mones). However, only when it has been proven in separate studies that a specific fac-
tor (eg, hormone) is not affecting the biomarker under study, a confounding effect of
the factor can be ignored in subsequent case-control studies.
This illustrates the problem that in many studies (especially case-control studies) deal-
ing with RPL and other disorders, it is often unclear whether the result being reported as
the main finding really was part of the a priori hypothesis from the start or whether it was a
finding discovered during the conduct of the study: a post hoc finding (see Table 1).
As a preventive measure, high-ranking journals request the investigators to give
more details of the study protocols or the protocols should be available on an online
public registry (www.ClinicalTrials.Gov) before the beginning of the study. All journals
should, as a minimum, request the investigators in the “materials and methods” sec-
tion to provide information from the study protocol about the a priori hypothesis, main
outcome measures, and sample size calculations, and to list all risk factors and bio-
markers that were planned to test and actually tested.
Multiple Testing
Most case-control studies in RPL can be categorized as discovery studies in which
there is no prior hypothesis about which associations are probable and therefore a
series of biomarkers are investigated openly or disguised. Testing of multiple bio-
markers is facilitated by the introduction of multiplex testing panels that can test hun-
dreds of biomarkers at a time, thereby reducing the costs immensely. If, for example,
40 different biomarkers are tested in patients and controls and a P value for signifi-
cance of less than .05 was chosen, it is expected that 2 (40 0.05 5 2) biomarkers
will be significantly increased or decreased in patients merely by chance. If no a priori
hypothesis about which biomarker was the primary focus of the study was made (dis-
covery study) the P values for all tested biomarkers should be subject to the Bonferroni
adjustment by multiplying the P value with the number of comparisons made. If 1 or
2 biomarkers are still significantly associated with RPL after this adjustment, the asso-
ciation must be confirmed in at least 1 independent study (replication study) with the a
priori hypothesis that the variables are associated with RPL.
Interpretation
Causality can rarely, if ever, be stated after finding a potential risk factor statistically
significantly associated with a disease. According to Bradford-Hill criteria,26 there
are several demands for stating causality, the most important being the following:
The criterion of temporality: that the risk factor occurs before the occurrence of
the disease.
The criterion of a biologic gradient: there should be a positive correlation be-
tween the severity of a putative risk factor and the severity of the disorder.
In studies of RPL, the first criterion can be documented only in prospective studies
finding that the presence of a potential risk factor increases the risk of a new miscar-
riage compared with its absence. In RPL, the other criterion can be documented if
there is a correlation between the severity of a potential risk factor, for example, con-
centration of APL antibody, and the severity of RPL 5 number of miscarriages in the
patients. Findings in some previously quoted studies may indeed be interpreted as ev-
idence against the investigated variables being causative for RPL because no biologic
gradient was discovered.7,8
COHORT STUDIES
Nonrandom Misclassification
Cohort studies are prone to methodological errors, which are important to recognize
when conducting or assessing them. Nonrandom misclassification is a significant
threat to the validity of cohort studies; however, unlike case-control studies in which
the patients or controls are the main source of the erroneous information leading to
misclassification, in cohort studies it is the researchers who are responsible. In occu-
pational medicine, workers are exposed to substances thought to increase the risk for
some disease. Because of that knowledge, these workers are often monitored more
closely than nonexposed workers and symptoms of disease (outcome) may be
Research Methodology in Recurrent Pregnancy Loss 31
registered more often in the exposed than in the nonexposed cohort. Biased intensity
of monitoring is also a problem in reproductive medicine research. In a cohort study of
women with obesity (the exposure) or no obesity, to investigate whether obesity
increases the risk of miscarriage, many obese women will have polycystic ovary syn-
drome (PCOS) and anovulation and therefore undergo assisted reproductive technol-
ogy (ART) treatment, whereas most nonobese women will be able to conceive without
the need of ART. In ART cycles, a b-hCG test is normally done 14 days after ovulation
or embryo transfer, and all biochemical pregnancies will be detected, whereas among
non-ART patients, they will often remain undetected. Therefore such a cohort study
may show that obese women have an increased risk of pregnancy loss but this may
be a result of nonrandom misclassification of the outcome.
In RPL cohort studies, misclassification is also a significant problem. The prognosis
after a diagnosis of RPL is heavily disputed: various prospective studies have
reported the chance of live birth in the first pregnancy after referral in patients with
3 miscarriages to be between 63% and 87%, with 4 miscarriages between
44% and 73%, and with 5 miscarriages between 25% and 52%, respectively.11 These
very different estimates are frustrating: for a patient with RPL, it is of utmost im-
portance to know whether her chance for live birth is 44% or 73%. This variation
may be caused by random misclassification within individual studies. Data from
2 placebo-controlled trials of intravenous immunoglobulin (IvIg) conducted in Swe-
den28 and Denmark,29 respectively, may clarify the issue. Patients with 3 or more mis-
carriages who met the inclusion criteria for participation in the trials were encouraged
to conceive and call the clinics as soon as they got pregnant. When the Swedish trial
(cohort 1) was concluded after 4 years, 50.6% of the patients were classified as not
having achieved pregnancy, only 3.4% had pre-embryonic losses, and 10.1% had
embryonic losses (Fig. 1). When the Danish trial (cohort 2) was concluded after
6 years, only 14.7% did not report pregnancy, 22.4% had pre-embryonic losses,
and 27.9% had embryonic losses. When live birth rates in cohorts 1 and 2 are calcu-
lated with the number of recognized pregnancies in the denominator, they become
72.7% and 44.8%, respectively (P<.005). The most striking difference between the
2 cohorts is the very much higher frequency of nonconception in cohort 1 than in
cohort 2. This may be because of the simple fact that the Danish patients were told
that they could not be included in the trial if they contacted the clinic more than
60
50
40
Not pregnant
% 30 Pre-embryonic losses
Embryonic losses
20 Live births
10
0
Cohort 1 Cohort 2
N = 89 N = 68
Fig. 1. Classification of pregnancy outcome in 4 categories of patients with RPL eligible for
participation in a Swedish (cohort 1) and a Danish (cohort 2) placebo-controlled trial of IvIg.
In each cohort, patients receiving IvIg and placebo are combined.
32 Christiansen
5 days after the missed menstrual period, whereas in the Swedish trial, patients were
included only when fetal heart action could be demonstrated in GW 6 to 7. In cohort 2,
every conception was probably registered, whereas in cohort 1, many pre-embryonic
losses may have been misclassified as nonconception. Another published cohort of
patients with RPL that estimated the chance of live birth in the first subsequent preg-
nancy, also found that 30% of the patients reported no further pregnancies after
referral to the clinic or were lost to follow-up.30 This is again a much higher frequency
that the 14.7% in the Danish cohort. As in the Swedish cohort, this difference is prob-
ably caused by incomplete follow-up resulting in misclassification of preclinical preg-
nancy losses as nonconception. When results from different cohort studies are
compared, this misclassification becomes nonrandom (see Table 1).
TREATMENT TRIALS
Historical Controls
In some treatment trials, pregnancy outcome in a group of patients treated in the pre-
sent was compared with outcome in nontreated controls from the past, so-called his-
torical controls.
In a small study of PGS in the treatment of RPL in women aged 35 years,31 the live
birth rate after PGS was for each patient compared with the expected rate in women
with similar age and number of miscarriages calculated from a historical cohort.30 The
investigators found that the live birth rate after PGS was higher than the expected rate
using the data from the historical cohort. Comparing outcome in contemporary RPL
patient groups with historical control groups carries a substantial risk of producing un-
reliable results (see Table 1). It is extremely difficult to be sure that a group of patients
with RPL identified in the past is comparable with a present group, as ascertainment of
Research Methodology in Recurrent Pregnancy Loss 33
Several RCTs in the RPL area are unfortunately not blinded. None of the trials testing
the effect of heparin plus low-dose aspirin in patients with RPL with or without APL has
compared this intervention against a placebo for heparin but only against low-dose
aspirin alone or a peroral placebo for aspirin.36–38 This is probably due to the reluc-
tance to use a placebo drug for daily subcutaneous injections during the whole period
of pregnancy. Although the outcome in RPL, a new miscarriage, can be determined by
objective methods that are not impacted by blinding/nonblinding, nonblinding can
negatively or positively influence outcome (see Table 1) in the groups receiving
nontreatment or expected less-efficient treatment compared with the expected
more-efficient treatment (tablets vs injections plus tablets).
Although a double-blinded randomized placebo-controlled trial is the best design to
obtain valid data concerning treatment effect, this method does not guarantee that the
results are trustworthy, as other circumstances can confound or weaken the results.
Confounding in RCTs
Most RCTs in RPL have included a small number of patients and this increases the risk
that a confounding variable, such as number of prior miscarriages and age, by chance
is unevenly distributed between the 2 allocation groups. In a small RCT, it is not suf-
ficient to show that the mean number of previous miscarriages or age is not statisti-
cally significantly different in the 2 allocation groups; these variables can still
significantly confound the results. The best way to counteract uneven distribution of
prognostic variables in RCTs is by doing block-randomization, whereby separate allo-
cation of patients is done according to groups with comparable number of miscar-
riages and age.
Inclusion after GW 6
In numerous RCTs, patients with RPL have been allocated and included in the trial only
when an ultrasonographic examination has demonstrated a viable intrauterine preg-
nancy in GW 6 to 7.28,38,39 This design can unfortunately substantially diminish any ef-
fect of the active intervention (see Table 1). Twenty-two percent of closely monitored
patients with RPL have pre-embryonic losses (see Fig. 1), which comprise almost half
of all their pregnancy losses.29 When treatment in RCTs starts only after the demon-
stration of fetal heart action, almost half of the risk period for miscarriage has thus
passed and the spontaneous prognosis at that time is good.40 Therefore, very large
numbers of participants are needed to show any treatment effect. Furthermore,
when treatment is started only in GW 6 to 7, the pregnancy has been exposed to
the full harmful effect of thrombophilic or immunologic factors for 2 to 3 weeks before
initiation of the potential beneficial therapy. The trophoblast or fetus, although viable at
that time, may have suffered irreversible damage that cannot be counteracted by initi-
ating active therapy.
participants; if the difference had been larger the trial had been allowed to continue.
RCTs stopped after interim analyses will therefore be expected to report intervention
effects that are smaller than the true effects (see Table 1) and, if included in systematic
reviews, their scientific quality should accordingly be down-graded.
Systematic Reviews
A systematic review is a review that uses systematic and explicit methods to critically
appraise a research topic; statistical methods such as meta-analyses may be used to
analyze and summarize the results of the included studies. Both case-control studies
and RCTs can be subject to such reviews and their results are considered the highest
level of evidence. In RPL research, systematic reviews have in particular focused
on intervention/treatment studies. The pooled OR calculated from the combination
of results in the meta-analysis is considered a good measure for the overall effect of
the intervention/treatment under study. However, because most treatments exhibit
different effects in different subsets of patients (eg, men and women or patients
with severe vs less severe disease), an important use of meta-analyses is to identify
subgroups of patients who have the largest benefit of treatment.45 I focus my discus-
sion regarding systematic reviews in RPL on those dealing with immunotherapy with
IvIg, because these reviews have included the largest numbers of studies and patients
and the discussion illustrates the pitfalls and methodological flaws that characterize
systematic reviews in the area of RPL treatment. Four different systematic reviews
have been published concerning IvIg treatment in RPL,4,42,46,47 which have provided
very different results and conclusions. This variability is in my view caused by either
(1) a failure to recognize the need of doing separate analysis in relevant subgroups
of patients or (2) by unfounded exclusions of RCTs in the systematic reviews. A
Cochrane review4 did not find any difference in live birth rate between IvIg and placebo
in all included patients with RPL but made no distinction between patients with pri-
mary and secondary RPL, which would be relevant, as 2 published trials had found
the IvIg efficient exclusively in the latter group.29,48 In contrast, another systematic re-
view based on almost the same patients recognized the relevance of doing this sub-
group analysis and found the live birth rate significantly higher in IvIg-treated than in
placebo-treated patients with secondary RPL.46
The 2 most recent systematic reviews on IvIg in RPL included only patients with “un-
explained” RPL, which meant that trials that had included patients with RPL positive
for APL according to the assays used in the individual clinics and according to the
local cutoff values were excluded.42,47 The investigators considered that 2 RCTs
included APL-positive patients and all 92 patients in these RCTs were completely
excluded from the systematic reviews comprising 25.3% of all 364 patients with
RPL participating in RCTs of IvIg. Exclusion of this substantial proportion of all ran-
domized patients with RPL appears to be an enormous waste of valid information.
In one of the excluded RCTs, none of the patients in fact had APL and in the other
almost all patients who tested APL-positive had very low titers.49 Only 3% would be
36 Christiansen
considered APL-positive according to the current criteria.50 On the other hand, an RCT
that was not excluded did in fact include APL-positive patients51 and several of the
other RCTs did not report details of APL assays or cutoff levels. Because both
excluded RCTs had found a substantial effect of IvIg in patients with secondary
RPL, doing the meta-analyses without these resulted in the conclusion in both system-
atic reviews that there is no benefit of IvIg either in primary or secondary RPL.42,47 As
stated by my group and others, we find the methodology of these systematic reviews
questionable and without the exclusions it can be proved that IvIg is indeed efficient in
secondary RPL.49,52 As previously mentioned, it is legal and desirable to do meta-
analyses in subgroups of patients to identify subgroups that benefit best from treat-
ment, and such a subgroup could be patients with RPL without APL. However, a
systematic review excluding these patients should of course exclude only those
patients who are positive for APL and not complete RCTs with only a tiny minority
of participants being APL-positive. Furthermore, in most published relevant RCTs,
insufficient information is given about the APL assays and cutoff values, which may
vary substantially between the trials. To carry out a meta-analysis of therapies in
APL-negative patients with RPL based on good research methodology, it is therefore
necessary to collect data on APL levels for each included patient. In most cases, these
data on individual patients can be obtained only by contacting the investigators of the
RCTs. If such a collection of raw data is not done, the resulting systematic review and
meta-analysis will produce biased results.
This discussion illustrates that even conclusions from systematic reviews should be
evaluated critically and a systematic review is not per se the highest level of evidence.
If they are not conducted according to rigorous and systematic rules, they will end up
being no more evidence-based than the narrative reviews of the old days (see
Table 1).
SUMMARY
Most causes of RPL are poorly elucidated and may have a more multifactorial etiology
than infertility and in many instances the cause-effect relationship is unclear. Further-
more, the RPL population is much smaller that the infertile population, thereby making
it difficult to conduct studies with adequate statistical power.
These problems render research in RPL inherently difficult and only methodologi-
cally high-quality studies are expected to produce useful results. Unfortunately,
such high-quality research studies are rare in the area of RPL.
All types of studies in RPL research are characterized by methodological pitfalls,
which are often not recognized by the researchers or readers. Failure to recognize
these can completely invalidate studies in the area. It is hoped that this review, by
setting focus on the problem, can help improve studies in the area of RPL in the future.
REFERENCES