Statistical issues in subgroup analyses
Download as PPTX, PDF0 likes755 views
The document discusses the significance of baseline covariates in clinical trials and the importance of subgroup analyses in estimating treatment effects. It covers risks associated with subgroup analyses, including increased chances of type I errors due to multiple comparisons and emphasizes the need for proper design and adjustment to maintain statistical validity. Two types of subgroup analyses are outlined: pre-planned and post-hoc, with a strong recommendation for pre-planned analyses to avoid misleading results.
Statistical issues in subgroup analyses
- 1. © NICE 2018. All rights reserved. Subject to notice of rights. Subgroup analyses F. Maignen
- 2. Why subgroups? The importance of (some) covariates… Baseline covariate: qualitative factor or quantitative variable measured or observed before a subject starts taking the study medication (usually before randomisation) and expected to influence the primary variable to be analysed. Examples: Previous therapies Age Severity of disease Given phenotype of a gene (BRCA2 mutation) Ethnicity, etc. 2
- 3. You can take into consideration the influence of a covariate in your analysis either … Stratification: performing a statistical procedure separately in groups (strata) with implied pooling across groups to reduce the effect of group factor. OR Subgrouping: estimate of the treatment effect in a subset of a clinical trial population. Patients excluded from a particular subgroup are described as the complement subgroup. 3
- 4. Baseline covariate … analysis … 4 Subgrouping Unstratified (black line) Stratified (green line)
- 5. Baseline covariate … what could go wrong? Assuming that a particular covariate plays an important role in the efficacy of a given product … what could go wrong during the conduct of your trial? 5
- 6. What can you do to avoid this situation and achieve … 6 Stratified randomisation … include your covariate in the randomisation factors.
- 7. Baseline covariates … You should: - Adjust your statistical analysis for your baseline covariate OR you can: - Perform a subgroup analysis AND/OR if this covariate is likely to greatly influence the efficacy of your new technology. - Perform a stratified randomisation (i.e. add your baseline covariate in your stratification factors) to avoid any imbalance (wrt this covariate) between the arms of your study. 7
- 8. The risk of subgroup analyses … FALSE CLAIMS!!!!!!!!!!!!!!!!!!!! 8
- 9. The risk of subgroup analyses … For a hypothesis test, I have (usually) a 5% risk of rejecting a null hypothesis which is actually true (risk alpha) … What would happen if I was performing simultaneous tests (say 6) in the same trial? 9
- 10. Multiplicity 1 – (1 – 0.05)6 = 26.5% The more you perform statistical tests, the higher the risk alpha is inflated i.e. the higher the likelihood of wrongly rejecting a null hypothesis … 10
- 11. Circumstances in which you will increase your alpha error rate … include: • Multiple endpoints (primary, then secondary) or comparisons (e.g. doses) • Interim analyses • Subgroup analyses … It is important to control the error rate i.e. to maintain the OVERALL error rate at 5% … 11
- 12. Bonferroni correction Use alpha/n (n = number of hypotheses tested) i.e. use 0.05/6 (0.008) instead of 0.05 … BUT WHAT COULD BE THE CONSEQUENCE OF THIS CORRECTION ON YOUR HYPOTHESIS TESTING??? 12
- 13. What we’ve seen so far … • Some baseline covariates can influence the efficacy of a product. • These covariates may or may not have been identified before the conduct of a trial. • You can stratify or adjust your clinical trial analysis according to this (these) covariate(s) = 1 estimate for the treatment effect. • You can perform a stratified randomisation according to a limited number of covariates (to avoid imbalances in your trial). • You probably want to make sure that the efficacy is homogeneous across different subgroups of patients in which the efficacy is NOT expected to differ (or may differ). • You may want to estimate the efficacy of your new product in a particular subgroup of patients. 13
- 14. What we’ve seen so far … • Stratification can have different meanings (stratified analysis OR stratified randomisation). • Subgrouping and stratification mean different things. • Subgroup analyses are attached to additional (statistical and financial) costs (i.e. adjustment for multiple comparisons). • Post-hoc subgroup analyses are BAAAAAAAAAAAD. 14
- 15. Does this medicine … 15
- 16. work (better or differently) … 1/ in men than in women? 2/ in adults and elderly (say people over 60)? 3/ in persons who own a female hamster (like I just found in a subgroup analysis)? 4/ only in patients carrying a specific mutation (e.g. patients carrying the BRAC2 mutation) compared to the general population of patients? WHAT ARE YOU LOOKING FOR IN EACH OF THESE SUBGROUPS?!?!?!? 16
- 17. The different types of subgroup analyses - Exploratory analyses: differential responses from early trials or from clinical trials that failed to establish treatment efficacy in its intended population. Hypothesis generating at best. - Supportive analyses: consistency of treatment effect across subgroups that has established treatment efficacy in its intended overall population. - Inferential analyses: aimed at establishing treatment efficacy in a pre-defined targeted subgroup and/or overall population. 17
- 18. Connect the dots … Covariate Men vs women Adults vs elderly Owns a pet hamster BRCA 2 mutation 18 Interpretation Homogeneity Heterogeneity Inferential Spurious / exploratory
- 19. Connect the dots … Covariate Men vs women Adults vs elderly Hamster BRCA 2 mutation 19 Interpretation Homogeneity Heterog. (adjust.) or safety Inferential Spurious / exploratory
- 20. Supportive subgroup analyses The statistical analysis will include an interaction term (interaction treatment*subgroup), the efficacy is assumed to be homogeneous across strata (subgroups) if the interaction term is not significant. Supportive subgroup analyses must be interpreted with caution: - Lack of statistical power to show differences - Differences can show up by complete chance - Multiple comparisons … - Credibility, replication and confirmation of the result +++ 20
- 21. Inferential subgroup analyses Completely different objective: the result of the subgroup analysis will result in a specific efficacy claim in the population pre-defined by the subgroup. The trial and the analysis must be designed a priori to establish (greater) efficacy in this subgroup (compared to the rest of the population – the tested product may OR may not be efficacious in the complement subgroup). 21
- 23. Inferential subgroup analyses The issues which will come up in SA might relate to: - Power of the test - Relative treatment effect and relative size of the subgroup - Adjustments for multiple comparisons (and allocation of alpha) - Method chosen for the analysis. 23
- 24. Biomarker stratified design 24 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 BM- BM+ BM- BM+ control Product Prognosis biomarker control BM- control BM+ Product BM- Product BM+
- 25. Biomarker stratified design 25 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 control Product control Product BM- BM+ Predictive biomarker BM- control BM- Product BM+ control BM+ Product
- 26. Biomarker stratified design 26 0 0.1 0.2 0.3 0.4 0.5 0.6 control Product control Product BM- BM+ Prognosis and predictive biomarker BM- control BM- Product BM+ control BM+ Product
- 27. Example of a biomarker H0: Null hypothesis for the overall population H+: Null hypothesis for the subgroup 3 possibilities: - Reject H0 and fail to reject H+: broad indication in the entire population of the trial - Reject H+ and fail to reject H0: restricted indication in the subgroup of patients who express the biomarker - But … what happens if H0 and H+ are both rejected? 27
- 28. Example of a biomarker If H0 and H+ are both rejected … Make sure that the effect in the overall population is not driven by an effect in the subgroup (i.e. that the product is effective in the BM- subgroup of patients) and that the effect is different in the BM+/BM- populations (otherwise he BM is pointless) … If not: “Enhanced” indication both in the overall population and pre-specified subgroup of patients (who express the biomarker). 28
- 29. Conclusions Two types of subgroup analyses: - Pre-planned (adequate when properly designed) or post-hoc (are baaaad unless purely exploratory) What is the role of the baseline covariate? Is the trial designed to address the key question of the analysis (design or the trial and conduct of the analysis): - Baseline covariate adjustment - Stratified randomisation - Subgroup analysis Objective of the subgroup analysis (exploratory, supportive or inferential) +++ MAKE IT PRE- PLANNED. 29
- 30. Conclusions Statistical issues might relate to: - Power of the test, - Relevance of the subgroup / biomarker, - Relative effect and size of the subgroup, - Adjustments for multiple comparisons. 30