Design and statistical analysis of method

transfer studies for biotechnology products

Meiyu Shen, Lixin Xu

Center for Drug Evaluation and Research,
U.S. Food and Drug Administration

This presentation reflects the views of the author and should not be
construed to represent FDA’s views or policies
 Outline
• Method development and its life cycle management

• Purpose of analytical method transfer studies

• What parameters compared in analytical method transfer studies

• Testing materials

• Analysis

• Conclusion
2
New analytical method development
• Parameters evaluated
• Specificity
• Linearity
• Accuracy
• Precision
• Limits of detection (LOD)
• Limits of Quantitation (LOQ)
• Range
3
Life cycle management of analytical procedures
• Including, but not limited to
– Trend analysis on method performance at regular intervals
• to optimize the analytical procedure
• to revalidate all or a part of the analytical procedure

– Development and validation of a new or alternative

analytical method
• A new impurity

– Method transferred to a new testing site

4
 Analytical method transfer studies
• The purpose of method transfer studies (internal)
– To determine if the two laboratories provide comparable
results across the range of interest.
• If so, then to transfer a fully validated analytical method from
the originating lab to a new lab (receiving lab)

– Once transferred, the method is suitable for its intended

use and can be used to ensure process consistency
and meet product specifications

5
 Analytical method transfer studies
• How to achieve the goal?
– Obtaining the comparative data from method transfer
studies

– Checking the receiving lab’s bias (difference between the

true value and the mean of the receiving lab)

– Determining success of implementation of the fully

validated analytical method in the receiving lab

6
 Important Factors in Method transfer studies

• Suppose that the same type of instrument from the

same manufacturer, same reagents, same
experimental conditions, and same testing
procedure, we investigate the following factors:
– operators
– days
– runs
– Replicates
– lots

7
 Key parameters in method transfer studies
• Mean shift (often incorrectly cited as accuracy)
– Comparing means of two labs

• Precision
– Comparing the standard deviations of two labs

• Bias (accuracy)

8
 Testing materials
• Is the reference standard appropriate material from
which comparative data is obtained for method
transfer studies?
– No
• Since the method is used to ensure process consistency and meet
product specifications

9
 Testing materials
• Multiple lots of a drug product if the assay is used for
drug releasing tests
• Multiple lots of a drug substance if assay is used for
measuring the content in drug substance
• Forced degradation samples or samples of a drug
substance or a drug product containing pertinent
product-related impurities if the transferred assay is
stability indicating
10
 Literature review: statistical analysis
• Many proposals, just name a few here
– Significance testing approach
• Comparing the means of two labs by the p-value of rejecting H0:
μR=μS
• Comment: Discouraging the sponsors to use a large sample size
and to obtain more precise measurement

– Quality control method

• Checking individual values against the control limit
• Comment: Not quantitative criteria for decision

11
Literature review: statistical analysis
– β-expected tolerance approach
• Calculating the tolerance interval in which a proportion (β) of the receiving
laboratory population is expected to fall within,
• Compares the above tolerance interval to acceptance limits around the mean
estimate of the sending laboratory
• Comments: challenge to define the acceptance limits.

– β-content tolerance interval approach to assure more than 100P% of the

individual difference (or percent difference, d) between individual results
obtained in the sending laboratory and the receiving laboratory are
within the predefined boundary (L, U) with 100(1 - α)% confidence level.
• Two-sided tolerance interval
• Two one-sided tolerance interval
• Comments: challenge to define L, U, and P
12
Our proposal: Equivalence test for comparing means of two
laboratories
• Denote the means of the response variable of
interest by μR and μS , respectively, for the
receiving laboratory and the sending laboratory.
H 0 :  R   S   or  R   S  
H a :    R   S  
. (Equation 1)
• Here δ is a pre-specified constant, also called an
equivalence margin.

13
 Challenge of setting equivalence margin for equivalence
approach
• Fixed margin
– Based on the experts’ knowledge
– Different margin for a different assay
• 1% , a reasonable margin for HPLC
– Too stringent for bioassay
• 2.5%, a reasonable margin for a specific bioassay
– Too liberal for HPLC
» Wider than specification 2% for drug substance assay

– Challenge: It is hard to have a number

14
Challenge of setting equivalence margin for equivalence
approach

• Non-fixed margin: a function of assay variability

– Unified rule for many assays
– Based on statistical power for rejecting the null hypothesis in the
equivalence hypothesis test with a limit number of observations (not
exceeding hundreds)

• All margins sits well within the assay specification.

15
 How to obtain the assay variability
• Long term quality control data
– Not appropriate, e.g.,
• If there is a stability trend over the time
• If there is a drift from assay instruments over the time

– Only good if there is no other confounding factor except

operators, days, and runs
• Hard to meet this criteria

• Comparative method transfer studies

– We may estimate the assay variability from studies

16
 Statistical analysis for the mean difference of two
labs
• Hypothesis testing (1):
– H0: μS – μR ≤ - cσS or μS – μR ≥ cσS
– Ha: - c σS <μS – μR <c σS
• where μS and μR are the mean responses of the sending lab and receiving
lab, respectively, and c > 0 is the constant.
• Equivalence margin
– c σS
• Value of c
– Determined from power function of rejecting the above
hypothesis if σS is known
17
 Power function for the two one-sided tests
procedure
• Let P  ,  be the probability of rejecting H0 under Ha in
*
1

Hypothesis testing (1) when σS =σR= σ.

•P  The
1 ,   power function is:
*

    
( 2 1 ) 2   1 1
 


1
    
4 t2   2  n1 n2  

2  * x   1   * x  1
     t ( )      t ( )    / 2 x / 21 e  x / 2 dx
0   1 1     1 1     2 ( / 2)
               
  n1 n2   n1 n2 
    

– -θ1=θ2=cσS
– n1: # of obs. in receiving lab
–n2: # of obs. in sending lab

– : normal cumulative function 18
 Determination of δ=CσS
  C
c0
x  C x   1
Power function:  *

P   0,  , n,   C ,     
 2 1/ n
 t ( )

  
   t ( )

  /2
  2  ( / 2)
x / 21e  x / 2 dx
0    2 1/ n 

n C
C=0.85 is reasonably chosen
Power=0.80 Power=0.85
such that we can achieve about
20 0.94 0.99 85% power with a sample size

22 0.90 0.95 in the range 20 to 30 per

24 0.86 0.90 laboratory.

26 0.82 0.87 Margin: δ=0.85 σS

28 0.79 0.83

30 0.76 0.80
19
 An example:
internal transfer to a new site
• All equipment moved to the new site
• Personnel transferred to the new site
• At least 2 lots
• >2 analysts and >2 days
• Reasonable sample size per lab: ~20-30
– Margin: e.g., 0.85σS
– Power to pass equivalence test is about 85% under no true
mean difference

20
Statistical analysis for the mean difference of two labs

• Option 1:Treating 0.85 ˆ S as a constant

– Estimating ̂ S from the sending lab

X R  X S  0.85ˆ S X R  X S  0.85ˆ S
– Define T1 
ˆ 2

R ˆ 2
S and T2 
ˆ R2 ˆ S2

nR n S nR nS

T1  t   T2  t  
– Concluding
t  
equivalence criteria is met if and ,
where is the 1-α quantile of t-distribution with degrees
of freedom ν, α is the nominal significance level (e.g., 0.05).
– Inflate both type 1 and 2 error rates

21
 Statistical analysis (continued)
• Option 2:
– Considering 0.85 ˆ S as a random variable

X R  X S  Cˆ S X R  X S  Cˆ S
T1  T2 
– Define ˆ R2 n R  ˆ S2 nS  C 2ˆ S2 C 0  1 and ˆ R2 n R  ˆ S2 n S  C 2ˆ S2 C 0  1

nS  1  nS  1  n 
C0     S 
2  2   2
– Where
T1  Z T2   Z Z

– Concluding equivalence criteria is met if and , where

is the 1-α quantile of standard normal distribution, α is the nominal
significance level (e.g., 0.05).

22
Head-to-head approach for comparing precisions obtained
in two labs
• Hypothesis H0: σR≤σS
– Hypothesis testing: small powers to reject H0 for
small samples.
• Check the point estimate
– ˆ  ˆ
R S

23
 Receiving lab’s bias verification
• Important to check bias since
– the equivalence margin can be large enough such that 90%
confidence interval in mean differences falls within the
equivalence margin
– but the receiving lab’s mean fails the bias criteria.

24
 References
1. U.S. Food and Drug Administration. Draft Guidance on Analytical Procedures and Methods Validation for Drugs and Biologics (2014).
2. International Society for Pharmaceutical Engineering (ISPE). The Good Practice Guide: Technology Transfer. ISBN-13: 978-1931879132 (2003).
3. United States Pharmacopeia. Transfer of Analytical Procedure. 37-National Formulary 32.
https://hmc.usp.org/sites/default/files/documents/HMC/GCs-Pdfs/c1224.pdf.
4. Ermer J, Limberger M, Lis K, Wätzig H. The transfer of analytical procedures. Journal of Pharmaceutical and Biomedical Analysis. 85: 262–276(2013).
5. Briggs .J, Nicholson R, Vazvaei F, et al. Method Transfer, Partial Validation, and Cross Validation: Recommendations for Best Practices and Harmonization from the
Global Bioanalysis Consortium Harmonization Team. The AAPS Journal 2014. 1143-1148 (2014).
6. Wieling J. Robust, fit-for-purpose method transfer: why we should apply equivalence testing. Bioanalysis. 7(7). 807–814 (2015).
7. Lin Z, Li W, Weng N. Capsule review on bioanalytical method transfer: opportunities and challenges for chromatographic methods. Bioanalysis. 3(1). 57–66
(2011).
8. Chambers D, Kelly G, Limentani G, Lister A, Lung KR, Warner E. Analytical Method Equivalency--An Acceptable Analytical Practice. Pharmaceutical Technology. 64-
80 (2005).
9. Dewé W, Govaerts B, Boulanger B, Rozet E, Chiap P, Hubert P. Using Total Error as Decision Criterion in Analytical Method Transfer. Chemometrics and Intelligent
Laboratory Systems. 85.262–268 (2007).
10. Kaminski L, Schepers U, Wätzig H. Analytical method transfer using equivalence tests with reasonable acceptance criteria and appropriate effort: Extension of the
ISPE concept. Journal of Pharmaceutical and Biomedical Analysis. 53. 1124–1129 (2010).
11. Frömke C, Hothorn LA, Sczesny F, Onken J, Schneider M. Analytical method transfer: Improving interpretability with ratio-based statistical approaches. Journal of
Pharmaceutical and Biomedical Analysis. 74. 186– 193 (2013).
12. Zhong JL, Lee K, Tsong Y. Statistical Assessment of Analytical Method Transfer. Journal of Biopharmaceutical Statistics. 18(5). 1005-1012 (2008).
13. Schwenke JR, O'Connor DK. Design and Analysis of Analytical Method Transfer Studies. Journal of Biopharmaceutical Statistics. 18 (5). 1013-1033 (2008).
14. Altan S, Shoung JM. Block designs in method transfer experiments. Journal of Biopharmaceutical Statistics.18(5). 996-1004 (2008).
15. Agut C, Caron A, Giordano C, Hoffman D, Ségalini A. Transfer of analytical procedures: A panel of strategies selected for risk management, with emphasis on an
integrated equivalence-based comparative testing approach. Journal of Pharmaceutical and Biomedical Analysis. 56. 293– 303 (2011).
16. Krause S. Validation of Analytical Methods for Biopharmaceuticals: A Guide to Risk-Based Validation and Implementation Strategies. PDA/DHI, River Grove, IL,
ISBN-10: 1933722061 (2007).
17. Satterthwaite FE. An Approximate Distribution of Estimates of Variance Components. Biometrics Bulletin. 2. 110–114 (1946).
18. Chen Y, Weng Y, Dong X, Tsong Y. Wald Tests for Variance-Adjusted Equivalence Assessment with Normal Endpoints. Journal of Biopharmaceutical Statistics (2016).
http://www.tandfonline.com/doi/full/10.1080/10543406.2016.1265542.
19. Okamoto M. Assay validation and technology transfer: Problems and solutions. Journal of Pharmaceutical and Biomedical Analysis. 87. 308– 312 (2014).

25
 Acknowledgement
• Dr. Yi Tsong, CDER/OB
• Dr. Juhong Liu, CDER/OBP
• Dr. Chikako Torigoe, CDER/OBP

26