Overview of MHRA guidance on Out of

Specification investigation
For Trainee QPs

15 May 2023

1
Introduction

Khash Latifi
I have an Honours Degree in Biotechnology in Medicine. I started my Career with Bio
Products laboratory in Elstree, where I worked as a production technician before
moving to quality. With BPL I gained experience in niche field of plasma fractionation
and sterile plasma derived products.

I moved to Moorfields Pharmaceutical in 2014, initially as a team to oversee the

remediation of the facility. However, with decision of closing the facility I gained
valuable and unique opportunity of working on decommissioning a GMP facility.

I joined GSK in 2015 working within quality, overseeing QC testing of Biopharm

products. I became a QP in 2019 and have been a certifying QP since for DPI. I
routinely support trainee QPs with Mock Viva assessments and deliver training as
part of the internal QP training program.
Agenda

• Relevance to QP role
• History
• Definition
• Scope
• MHRA OOS/Atypical guide
• Phase Ia
• Phase Ib
• Hypothesis testing
• Phase II
• Stability
• Model answer

May 18, 2023 3

Relevance of OOS/Atypical to a QP

• Annex 16 - Certification by a QP and Batch Release – ‘routine

duties’

➢ 1.7.6 The source and specification of starting materials and packaging materials used
in the batch are compliant with the MA.

➢ 1.7.13 Finished product QC test data complies with the Finished Product Specification
described in the MA, or where authorised, the Real Time Release testing programme.

➢ 1.7.16 All investigations pertaining to the batch being certified (including OOS and
OOT investigations) have been completed to a sufficient level to support certification.
 Relevance of OOS/Atypical to a QP

Part I: Basic Requirements for Medicinal Products

Chapter 6 Quality Control

Documentation
➢ Laboratory documentation should follow the principles given in Chapter 4. An
important part of this documentation deals with Quality Control and the
following details should be readily available to the Quality Control Department:
iv - A procedure for the investigation of Out of Specification and Out of Trend
results
Certain types of data (e.g. test results, yields, env controls) should be recorded in
a manner permitting trend evaluation. Any OOT / OOS data should be
addressed and subject to investigation.

On-going stability programme

OOS or significant atypical trends should be investigated. Any confirmed OOS or
significant negative trend should be reported to the relevant competent
authorities. The possible impact on batches on the market should be
considered…….
May 18, 2023 5
 Relevance of OOS/Atypical to a QP

Part II: Basic Requirements for Active Substances Used as Starting

Materials

Chapter 6 Documentation and Records

Laboratory Controls
Any OOS result obtained should be investigated and documented according to
procedure. This procedure should require analysis of the data, assessment of
whether a significant problem exists, allocation to the tasks for corrective actions and
conclusions. Any resampling and / or retesting after OOS should be performed
according to a documented procedure.

Chapter 8 Production and In process Controls

IPC sampling and Controls
OOS are not normally required for IPC tests that are performed for the purpose of
monitoring and or adjusting the process.

May 18, 2023 6

 History
The issue of OOS results first came to prominence with the Barr case nearly 30 years ago.

•In the early 90’S, The FDA took Barr Laboratories, a US generics company to court over
alleged malpractice in the laboratory (e.g. retesting into compliance), QA and validation
practices………THE FDA WON!
•The Barr Decision made the OOS problem into a major update for the QC laboratory by
creating a regulatory requirement where, following an OOS result, an investigation must be
initiated before any retesting can be done.
•Since the Barr case the US FDA has led the way in defining standards for the investigation of
OOS results, culminating in the publication of the final Guidance for Industry on this subject in
October 2006 (New revision published May 22)
•In August 2013 MHRA published its own guidance on handling OOS/T investigations. This was
later updated in 2017 with some minor changes.

May 18, 2023 7

OOS/T investigation guidance

May 18, 2023 8

 Definitions

OOS result
Test result that does not comply with the pre-determined acceptance criteria (i.e. for
example, filed applications, drug master files, approved marketing submissions, or official
compendia or internal acceptance criteria).
Atypical result
Results that are still within specification but are unexpected, questionable, irregular,
deviant or abnormal. Example would be chromatograms that show unexpected peaks
Out of Trend (OOT) Result
Is generally a stability result that does not follow the expected trend, either in comparison
with other stability batches or with respect to previous results collected during a stability
study. However the trends of starting materials and in-process samples may also yield out
of trend data.
The result is not necessarily OOS but does not look like a typical data point.

Hypothesis/Investigative Testing
Is testing performed to help confirm or discount a possible root cause i.e what might
have happened that can be tested.

May 18, 2023 9

 Definitions

Re-test
Performing the test over again using material from the original sample composite, if it has not
been compromised and/or is still available. If not, a new sample will be used.

Re-Sample
A new sample from the original container where possible, required in the event of insufficient
material remaining from original sample composite or proven issue with original sample
integrity.

Reportable result
Is the final analytical result. This result is appropriately defined in the written approved test
method and derived from one full execution of that method, starting from the original sample.

May 18, 2023 10

 Scope

In scope Out of scope

❖Raw materials ❖Customer complaints
❖APIs ❖Cleaning verification
❖Intermediates ❖Environmental monitoring
❖Drug product ❖Incoming packaging
❖Packaging on finished products ❖Equipment calibration and
❖Release testing maintenance
❖Manufacturing process changes ❖Method validation and transfer
❖Stability testing ❖Experimental work
❖In-process testing while trying to
achieve a manufacturing process end-
point e.g. pH, viscosity

May 18, 2023 11

Investigation Overview

Laboratory Analysis

OOS (OOT) Result

Satisfactory

Phase la Investigation

Phase Ib Investigation

Manufacturing Phase II Investigation

Investigation

Batch Disposition

Product Impact
Assessment Phase III Investigation

May 18, 2023 12

 Phase Ia investigation

•The main objective of Phase la

investigation is to determine whether there
has been a clear obvious errors due to
external circumstances such as calculation
error, power failure or those that the
analyst has detected prior to generating
data such as spilling sample that will negate
the requirement of a Phase Ib investigation.

•For microbiological analysis this may be

after the analysis has been completed and
reviewed during reading of the samples.

•It is expected that these issues are trended

even if a laboratory investigation lb or ll was
not raised.

May 18, 2023 13

 Phase Ib investigation

•Phase Ib Investigation – Initial Investigation

conducted by the analyst and supervisor using
the Laboratory Investigation Checklist

•For microbiological analysis where possible

once a suspect result has been identified
ensure all items related to the test failure are
retained such as other environmental plates,
dilutions, ampoules/vials of product,
temperature data, auto-pipettes, reagents –
growth media. No implicated test
environmental plates should be destroyed until
the investigation has been completed.

•On completion of the Analyst and Supervisor

investigation re-measurement can start once
the hypothesis plan is documented and is only
to support the investigation testing.

•This initial hypothesis testing can include

the original working stock solutions but should
not include another preparation from the
original sample (see: re-testing)

May 18, 2023 14

 Phase Ib – Lab investigation checklist

Example of areas to be covered as part of the Additional consideration for microbiological

Phase Ib investigation: test:
•Correct test method • Are the isolates located as expected –
•Equipment calibration on glove dab marks, SAS ‘dimples’, filter
•Appropriate standards used membrane etc.
•Pipette calibration/verification within date • Was the sample media integral – i.e. no
•Equipment logbook cracks in plates.
•System suitability • Was there contamination present in
other tests (or related tests) performed
•Correct storage of glassware
at the same time, including
•Assessment of possible sample environmental controls.
contamination
• Were negative and positive controls
•Analyst training/invalid rate satisfactory.
•Analyst knowledge of the procedure • Were the correct media/reagents used.
•Review of data from other batches • Were the samples integral (not leaking)
performed on the same run
• Were the samples stored correctly
•Correct specification applied (refrigerated)
•Previous issues with the assay • Was the media/reagent stored correctly
before use
• Were the incubation conditions
satisfactory.
May 18, 2023 15
 Hypothesis testing

Hypothesis Testing (Applicable to Phase Ib and Phase II):

Should be started as part of Phase Ib and continue into Phase II if no assignable cause found.

Description of the testing should be written, and then approved by QA/Contract Giver/QA equivalent prior
to initiating investigational testing. The requirements of investigational testing listed below:

The description must fully document

The hypothesis to the test the root cause being investigated.
What samples will be tested.
The exact execution of the testing.
How the data will be evaluated

This Hypothesis testing may continue from the re-measurement of the original preparations.

Examples of hypothesis testing:

•Sample not sonicated correctly
•Potential equipment failure
•Sample carry over
•Dilution error

Investigational testing may not be used to replace an original suspect analytical results. It
may only be used to confirm or discount a probable cause.

May 18, 2023 16

 Phase II investigation

Conducted when the phase I investigations did not reveal an assignable laboratory error.

Prior to further testing a manufacturing investigation should be started to determine whether

there was a possible manufacturing root cause.
Manufacturing investigation can include:
•Batch records
•Batch related deviation
•Change controls
•Logbooks
•Process parameters
•GEMBA

May 18, 2023 17

 Phase II – no assignable cause

If no assignable cause that could explain the results can be identified during the manufacturing investigation
or the assay failure investigation retesting may be considered.

Points to consider for retesting:

•The decision to retest should be based on sound scientific judgement. The test plan must be approved by
quality (QP) before re testing occurs.
•The minimum number of retests should be documented within the procedure and be based upon
scientifically sound principles. The number of retests should be statistically valid; papers have suggested 5, 7,
or 9.
•The retests should be performed by a different analyst where possible.
•Whenever possible it should be performed on the original sample not a different sample.
•If insufficient quantity of the original sample remains to perform all further testing then the procedure for
obtaining a resample must be discussed and agreed by QA/Contract Giver/QA equivalent.

May 18, 2023 18

 Phase II – no assignable cause

Microbiological investigations:

•Difficult to perform due to nature of the test (1-2 weeks after analysis).
•Important to understand the boundary of samples/products/manufacturing.
•Are the organisms of an expected type, determine likely source – would it be likely to be found where it was?
•Review the media – prepared in house or bought in pre-prepared, supplier history, sterilisation history
•Evaluate area/environmental trends for test area and support areas.
•Cleaning and maintenance of the test environment
•GEMBA the area where the contamination occurred
•All potential sources of contamination need to be considered – process flow the issue from sample storage to
the test environment.
•Cleaning and maintenance of the test environment
•Look for other documentation such as deviations and engineering notifications around the area/time of
concern (this is applicable to the laboratory as well as manufacturing).

May 18, 2023 19

 Phase II – Averaging

There are both appropriate and inappropriate uses of averaging test data during original
testing and during an OOS investigation.

Appropriate use:

The validity of averaging depends upon the sample and its purpose. For example, in an optical rotation test,
several discrete measurements are averaged to determine the optical rotation for a sample, and this average
is reported as the test result.
Another example is using multiple replicates in a HPLC run to arrive at a single reportable result.

Inappropriate use:

Reliance on averaging has the disadvantage of hiding variability among individual test results. Averaging can
also conceal variations in different portions of a batch, or within a sample.
For example, the use of averages is inappropriate when performing powder blend/mixture uniformity or
dosage form content uniformity determinations.

In the context of additional testing performed during an OOS investigation, it is critical that the
laboratory provide all individual results for evaluation and consideration by Quality Assurance
(Contract Giver/QP).
May 18, 2023 20
 Phase III

If the batch is rejected there still needs to be an investigation to :

•Determine if other batches or products are affected.
•Identify and implement corrective and preventative action.

•The Phase III investigation should review the completed manufacturing investigation and combined
laboratory investigation for possible causes for the results obtained.

•The batch quality must be determined and disposition decision taken.

•Once a batch has been rejected there is no limit to further testing to determine the cause of failure, so that
corrective action can be taken.
•The decision to reject cannot be reversed as a result of further testing.

•The impact of OOS result on other batches, on going stability studies, validated processes and testing
procedures should be determined by Quality Control and Quality Assurance and documented, along with
appropriate corrective and preventive actions.

May 18, 2023 21

 Conclusion

Conclusion:
•If no laboratory or calculation errors are identified in the Phase I and Phase II there is no
scientific basis for invalidating initial OOS results in favour of passing retest results.

•All test results, both passing and suspect, should be reported and all data has to be
considered in batch release decisions.

•If the investigation determines that the initial sampling method was inadequate, a new
accurate sampling method must be developed and approved by the Quality Assurance
responsible for release.
A consideration should be given to other lots sampled by the same method.

•An initial OOS result does not necessarily mean the batch fails and must be rejected.
The OOS result should be investigated,
The findings, including retest results, should be interpreted to evaluate the batch
The decision regarding release or rejection which should be fully documented.

May 18, 2023 22

 Conclusion

Conclusion continued:

•Where the OOS result is confirmed, the result should be used in evaluating the quality of the
batch

• A confirmed OOS result indicates that the batch does not meet established specifications and
should result in the batch's rejection and proper disposition.
Other lots should be reviewed to assess impact.

•For inconclusive investigations the OOS result should be given full consideration (most
probable cause determined) in the batch or lot disposition decision by the certifying QP

•Any decision to release a batch, in spite of an initial OOS result that has not been invalidated,
should come only after a full investigation has shown that the OOS result does not reflect the
quality of the batch.

In making such a decision, Quality Assurance/QP should always err on the side of caution.

May 18, 2023 23

 Stability

Stability OOS/T must be escalated as soon as the suspect result is found. This is to allow for prompt
assessment of the batches on the market.
. For OOS Situations Regulatory agencies will require notification within a short time point of
discovery due to recall potential.
The investigation will be the same as discussed above, phase I and Phase II.
•Remember to check stability chambers for any excursions during storage.
•Result of other tests performed at this timepoint and previous timepoint.
•Impurity profile.

If abnormal results are found at any stability interval which predicts that the test results may be OOS before
the next testing interval, schedule additional testing before the next scheduled testing interval. This will help
better determine appropriate actions to be taken.

The stability OOS should link to the Product Recall procedures.

May 18, 2023 24

 Stability

Out of Trend

To facilitate the prompt identification of potential issues, and to ensure data quality, it is advantageous to use
objective (often statistical) methods that detect potential out-of-trend (OOT) stability data quickly.
OOT alerts can be classified into three categories to help identify the appropriate depth for an investigation.
OOT stability alerts can be referred to as:
•Analytical,
•process control, and
•compliance alerts,

As the alert level increases from analytical to process control to compliance alert, the depth of investigation
should increase.

A compliance alert defines a case in which an OOT result suggests the potential or likelihood for OOS results
to occur before the expiration date within the same stability study (or for other studies) on the same product.

May 18, 2023 25

 Model answer for OOS scenario
question

•Raise a lab investigation record under PQS

•Place batch on hold
•Using MHRA OOS/T guidance, initiate phase Ia investigation.
•Objective of phase Ia is to determine clear and obvious errors.
•If nothing identified as part of phase Ia, the investigation moves to phase Ib.
•Phase Ib is conducted with analyst and supervisor.
•Phase Ib is more detailed review of all the analytical steps and what could have gone wrong.
•Hypothesis testing is initiated here and moves to phase II.
•As part of phase II, manufacturing investigation is initiated.
•If this is also inconclusive and there is scientific justification, retesting may be considered.
•Number of retest must be scientifically and statistically valid. (5,7 or 9)
•Batch disposition decision (phase III)

May 18, 2023 26