Working document QAS/16.667/Rev.

2
August 2018
Draft document for comments

2 GUIDELINES ON VALIDATION – APPENDIX 5

3 VALIDATION OF COMPUTERIZED SYSTEMS
4 (August 2018)
5 DRAFT FOR COMMENTS
6
7 Please send any comments you may have on the attached text to Dr S. Kopp, Group Lead,
8
9 Medicines Quality Assurance, Technologies Standards and Norms ([email protected]), with a
10 copy to Ms Xenia Finnerty ([email protected]) by 30 September 2018.
11
12 Medicines Quality Assurance working documents will only be sent out electronically
13
14 and will also be placed on the Medicines website for comment under “Current
15 projects”. If you have not already receive our draft working documents, please send
16
17 your email address to [email protected] and we will add your name to our electronic
18 mailing list.
19
20
21

22 .
© World Health Organization 2018
23
24 All rights reserved.
25
26 This draft is intended for a restricted audience only, i.e. the individuals and organizations having received this draft. The draft
27 may not be reviewed, abstracted, quoted, reproduced, transmitted, distributed, translated or adapted, in part or in whole, in any
28 form or by any means outside these individuals and organizations (including the organizations' concerned staff and member
29 organizations) without the permission of the World Health Organization. The draft should not be displayed on any website.
30
31 Please send any request for permission to:
32
33 Dr Sabine Kopp, Group Lead, Medicines Quality Assurance, Technologies Standards and Norms, Regulation of Medicines and
34 other Health Technologies, Department of Essential Medicines and Health Products, World Health Organization, CH-1211
35 Geneva 27, Switzerland, fax: (41 22) 791 4856, email: [email protected].
36
37 The designations employed and the presentation of the material in this draft do not imply the expression of any opinion
38 whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or of its
39 authorities, or concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent approximate border lines
40 for which there may not yet be full agreement.
41
42 The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended
43 by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors and omissions
44 excepted, the names of proprietary products are distinguished by initial capital letters.
45
46 All reasonable precautions have been taken by the World Health Organization to verify the information contained in this draft.
47 However, the printed material is being distributed without warranty of any kind, either expressed or implied. The responsibility
48 for the interpretation and use of the material lies with the reader. In no event shall the World Health Organization be liable for
49 damages arising from its use.
50
51 This draft does not necessarily represent the decisions or the stated policy of the World Health Organization.
52
 Working document QAS/16.667/Rev.2
page 2

53 SCHEDULE FOR THE PROPOSED ADOPTION PROCESS OF DOCUMENT QAS/16.667:

54
55 GUIDELINES ON VALIDATION – APPENDIX 5
56 VALIDATION OF COMPUTERIZED SYSTEMS

Discussion of proposed need for revision in view of the current

trends in validation during the informal consultation on data
29 June–1 July 2015
management, bioequivalence, good manufacturing practices
(GMP) and medicines inspection.
Preparation of draft proposal for revision of the main text and
several appendices by specialists in collaboration with the
Medicines Quality Assurance Group and Prequalification Team
(PQT-Inspections), based on the feedback received during the July 2015-April 2016
meeting and from PQT-Inspections, draft proposals developed on
the various topics by specialists, as identified in the individual
working documents.
Presentation of the progress made to the Fiftieth Meeting of the
WHO Expert Committee on Specifications for Pharmaceutical 12–16 October 2015
Preparations (ECSPP).
Discussion at the informal Consultation on Good Practices for
4–6 April 2016
Health Products, Manufacture and Inspection, Geneva.
Preparation of revised text by Mrs M. Cahilly and
Dr A.J. van Zyl, participants at the above-mentioned consultation,
based on Mrs Cahilly’s initial proposal and the feedback received May 2016
during and after the informal consultation by the meeting
participants and members of PQT-Inspections.

Circulation of revised working document for public consultation. May 2016

Consolidation of comments received and review of feedback. August–September 2016

Presentation to the Fifty-first ECSPP. 17–21 October 2016

 Working document QAS/16.667/Rev.2
page 3

More than 400 comments were received during the public

consultation and were evaluated and prioritized by the German October 2016–April
Expert Group on Computerized System with the assistance of Mr 2017
Menges.
The comments and feedback were discussed and further reviewed
during the Consultation on Good Practices for Health Products, 25–28 April 2017
Manufacturer and Inspection.
The large number of feedback and comments received required
May 2017–December
major restructuring and reworking, therefore assistance was
2017
sought from experts and PQT-Inspections.
Preparation of the revised text by Dr D. Catsoulacos from PQT-
Inspection and Dr V. Gigante from the Medicine Quality
February–April 2018
Assurance Group, based on the comments and all the various
input received.
Circulation of the revised working document for public
June 2018
consultation.
Consolidation of comments received during the public
July 2018
consultation.
Presentation of the revised working document at the WHO
10–12 July 2018
Consultation on Good Practices for Health Products, Manufacture
and Inspection.
Revision of the draft text on the basis of feedback received during
and after the informal consultation by the meeting participants and July 2018
members of PQT-Inspections.
Circulation of the revised working document for public
July–September 2018
consultation.

Compilation of comments received during the public consultation. October 2018

Presentation of updated working document at the Fifty-third

22–26 October 2018
ECSPP.

Any other follow-up action as required,

 Working document QAS/16.667/Rev.2
page 4

57 GUIDELINES ON VALIDATION – APPENDIX 5

58 VALIDATION OF COMPUTERIZED SYSTEMS
59
60 1. BACKGROUND INFORMATION
61
62 The need for revision of the published World Health Organization (WHO) Supplementary
63 Guidelines on Good Manufacturing Practices: Validation (1) was identified by the
64 Prequalification of Medicines programme and a first draft document was circulated for comment
65 in early 2013. The focus, at that time, was the revision of the Appendix on Non-Sterile Process
66 Validation (Appendix 7) which had been revised and was adopted by the ECSPP at its Forty-
67 ninth meeting in October 2014 (2).
68
69 The overarching text, entitled Guidelines on Validation (working document QAS/16.666),
70 constitutes the general principles of the new guidance on validation. This working document,
71 Validation of Computerized Systems, is Appendix 5 of the overarching guidances on validation.
72
73 The following is an overview of the appendices that are intended to complement the general text
74 on validation:
75
76 Appendix 1
77 Validation of heating, ventilation and air-conditioning systems
78  will be replaced by cross-reference to WHO good manufacturing practices (GMP) for
79 heating, ventilation and air-conditioning systems for non-sterile pharmaceutical products.
80
81 Appendix 2
82 Validation of water systems for pharmaceutical use
83  will be replaced by cross-reference to WHO (GMP): water for pharmaceutical use (3).
84
85 Appendix 3
86 Cleaning validation – consensus to retain.
87
 Working document QAS/16.667/Rev.2
page 5

88 Appendix 4
89 Analytical method validation – update in process (working document QAS/16.671).
90
91 Appendix 5
92 Validation of computerized systems – updated text proposed in this working document.
93
94 Appendix 6
95 Qualification of systems and equipment – update in process (working document
96 QAS/16.673/Rev.1).
97
98 Appendix 7
99 Non-sterile process validation – update already published as Annex 3, WHO Technical Report
100 Series, No. 992, 2015.
101
102 2. INTRODUCTION AND SCOPE
103
104 2.1 Computerized systems should be validated in accordance with quality risk management
105 principles and the level of validation should be commensurate to identified risks, complexity and
106 intended use. This guide applies to systems used in GMP (4) but may be extended to
107 systems used in all good practice (GxP) activities, as appropriate.
108
109 2.2 The purpose of validation is to confirm that the computerized system specifications
110 conform to the user’s needs and intended use by examination and provision of documented and
111 objective evidence that the particular requirements can be consistently fulfilled. Validation
112 should establish confidence in the accuracy, reliability and consistency in the performance of the
113 system, and it should also ensure that all necessary technical and procedural controls are
114 implemented confirming compliance with good documentation practices for electronic data
115 generated by the system (5).
116
117 2.3 System elements that need to be considered in computerized system validation include
118 computer hardware and software, related equipment and IT infrastructure and operating system
 Working document QAS/16.667/Rev.2
page 6

119 environment, procedures and systems documentation, as appropriate, including user manuals.
120 Persons should be appropriately trained and qualified, including but not limited to, developers,
121 end-users, system application administrators, network engineers, database administrators and
122 electronic archivers. Computerized system validation activities should address both system
123 functionality and configuration as well as any custom-developed elements.
124
125 2.4 Computerized systems should be maintained throughout the system life cycle in a
126 validated state with risk-based controls for the operational phase which may include, but is not
127 limited to, system planning, preparation and verification of standard operating procedures
128 (SOPs) and training programs, system operation and maintenance, including handling of
129 software and hardware updates, monitoring and review, change management and incident
130 reporting followed by system retirement.
131
132 2.5 Depending on the types of systems or typical applications, such as process control
133 systems (distributed control system (DCS), programmable logic controller (PLC), supervisory
134 control and data acquisition (SCADA)), laboratory information management systems (LIMS),
135 laboratory instrument control systems and business systems (enterprise resource planning
136 (ERP), manufacturing resource planning (MRP II)) used by the manufacturer, documentation
137 covering, but not limited to, the following information and supporting process should be
138 accessible on-site for review:
139
140  purpose and scope;
141  roles and responsibilities;
142  validation approach;
143  risk management approach;
144  approved system requirement/specifications;
145  system acceptance criteria;
146  vendor selection and assessment;
147  configuration management and change control procedures;
148  backup and recovery (application and data);
149  error handling and corrective action;
 Working document QAS/16.667/Rev.2
page 7

150  contingency planning and disaster recovery;

151  maintenance and support;
152  data security; and
153  validation deliverables and documentation.
154
155 3. GLOSSARY
156
157 The definitions given below apply to the terms used in these guidelines. They may have
158 different meanings in other contexts.
159
160 archival. Archiving is the process of protecting records from the possibility of being
161 further altered or deleted, and storing these records under the control of independent data
162 management personnel throughout the required retention period. Archived records should
163 include, for example, associated metadata and electronic signatures.
164
165 audit trail. The audit trail is a form of metadata that contains information associated with
166 actions that relate to the creation, modification or deletion of GxP records. An audit trail
167 provides for secure recording of life-cycle details such as creation, additions, deletions or
168 alterations of information in a record, either paper or electronic, without obscuring or
169 overwriting the original record. An audit trail facilitates the reconstruction of the history of such
170 events relating to the record regardless of its medium, including the “who, what, when and why”
171 of the action. For example, in a paper record, an audit trail of a change would be documented via
172 a single-line cross-out that allows the original entry to remain legible and documents the initials
173 of the person making the change, the date of the change and the reason for the change, as
174 required to substantiate and justify the change. In electronic records, secure, computer-
175 generated, time-stamped audit trails should allow for reconstruction of the course of events
176 relating to the creation, modification and deletion of electronic data. Computer-generated audit
177 trails should retain the original entry and document the user identification, the time/date stamp of
178 the action, as well as the reason for the change, as required to substantiate and justify the action.
179 Computer-generated audit trails may include discrete event logs, history files, database queries or
 Working document QAS/16.667/Rev.2
page 8

180 reports or other mechanisms that display events related to the computerized system, specific
181 electronic records or specific data contained within the record.
182
183 automatic or live update. A process used to bring up-to-date software and system
184 functionalities in a silent or announced way. More specifically, the update takes place
185 automatically with or without the user's knowledge.
186
187 backup. A backup means a copy of one or more electronic files created as an alternative
188 in case the original data or system are lost or become unusable (for example, in the event of a
189 system crash or corruption of a disk). It is important to note that backup differs from archival in
190 that backup copies of electronic records are typically only temporarily stored for the purposes of
191 disaster recovery and may be periodically overwritten. Such temporary backup copies should
192 not be relied upon as an archival mechanism.
193
194 business continuity plan. A documented plan that defines the ongoing process supported
195 by management and funded to ensure that the necessary steps are taken to identify the impact of
196 potential losses, maintain viable recovery strategies and recovery plans and assure continuity of
197 services through personnel training, plan testing and maintenance.
198
199 cloud based. A model for enabling on-demand network access to a shared pool of
200 configurable computing resources that can be rapidly provisioned and released with minimal
201 management effort or service provider interaction. These computing resources should be
202 appropriately qualified.
203
204 computerized system. A computerized system collectively controls the performance and
205 execution of one or more automated processes and/or functions. It includes computer hardware,
206 software, peripheral devices, networks and documentation, for example, manuals and SOPs, as
207 well as personnel interacting with hardware and software.
208
 Working document QAS/16.667/Rev.2
page 9

209 computerized systems validation. Confirmation by examination and provision of

210 objective and documented evidence that computerized system’s predetermined specifications
211 conform to user needs and intended use and that all requirements can be consistently fulfilled.
212
213 configuration management. A discipline applying technical and administrative direction
214 and surveillance to identify and document the functional and physical characteristics of a
215 configuration item, control changes to those characteristics, record and report change processing
216 and implementation status and verifying compliance with specified requirements.
217
218 COTS. Commercial off-the-shelf software; a vendor-supplied software component of a
219 computerized system for which the user cannot claim complete software life-cycle control.
220
221 data. All original records and true copies of original records, including source data and
222 metadata and all subsequent transformations and reports of these data, which are generated or
223 recorded at the time of the GxP activity and allow full and complete reconstruction and
224 evaluation of the GxP activity. Data should be accurately recorded by permanent means at the
225 time of the activity. Data may be contained in paper records (such as worksheets and logbooks),
226 electronic records and audit trails, photographs, microfilm or microfiche, audio- or video-files or
227 any other media whereby information related to GxP activities is recorded.
228
229 data integrity. Data integrity is the degree to which data are complete, consistent,
230 accurate, trustworthy and reliable and that these characteristics of the data are maintained
231 throughout the data life cycle. The data should be collected and maintained in a secure manner,
232 such that they are attributable, legible, contemporaneously recorded, original or a true copy and
233 accurate. Assuring data integrity requires appropriate quality and risk management systems,
234 including adherence to sound scientific principles and good documentation practices (5).
235
236 data life cycle. All phases of the process by which data are created, recorded, processed,
237 reviewed, analyzed and reported, transferred, stored and retrieved and monitored until retirement
238 and disposal. There should be a planned approach to assessing, monitoring and managing the
239 data and the risks to those data in a manner commensurate with potential impact on patient
 Working document QAS/16.667/Rev.2
page 10

240 safety, product quality and/or the reliability of the decisions made throughout all phases of the
241 data life cycle.
242
243 disaster recovery. A documented process or set of procedures to recover and protect a
244 business information technology infrastructure in any event causing the system to be unavailable.
245 It appropriately defines resources and actions to be taken before, during and after a disaster to
246 return the system to operational use.
247
248 functional specifications. The functional specifications define functions and
249 technological solutions that are specified for the computerized system based upon technical
250 requirements needed to satisfy user requirements (for example, specified bandwidth required to
251 meet the user requirement for anticipated system usage).
252
253 legacy system. It refers to an outdated computer system, programming language,
254 application software, or processes that are used, instead of available upgraded versions, that are
255 deemed not to fully satisfy current GMP requirements.
256
257 master data. A single source of business data used across multiple systems, applications
258 and processes and subject to change control to ensure accuracy through the data life cycle.
259
260 metadata. Metadata is data about data that provides the contextual information required
261 to understand those data. These include structural and descriptive metadata. Such data describe
262 the structure, data elements, interrelationships and other characteristics of data. They also permit
263 data to be attributable to an individual. Metadata necessary to evaluate the meaning of data
264 should be securely linked to the data and subject to adequate review. For example, in weighing,
265 the number 8 is meaningless without metadata, such as, the unit, milligram, etc. Other examples
266 of metadata include the time/date stamp of an activity, the operator identification (ID) of the
267 person who performed an activity, the instrument ID used, processing parameters, sequence files,
268 audit trails and other data required to understand data and reconstruct activities.
269
270
 Working document QAS/16.667/Rev.2
page 11

271 production environment. For regulated computerized systems, the production

272 environment is the business and computing operating environment in which the computerized
273 system is being used for GMP regulated purposes.
274
275 regression analysis and testing. A documented software verification and validation task
276 to determine the extent of verification and validation analysis and testing that must be repeated
277 when changes are made to any previously examined software component or system.
278
279 system life cycle. The period of time that starts when a computerized system is conceived
280 and ends when the system is retired, taking into consideration regulatory requirements. The
281 system life cycle typically includes a requirements and planning phase; a development phase that
282 includes: a design phase and a programming and testing phase; a qualification and release phase
283 that includes: a system integration and testing phase; a validation phase; a release phase; an
284 operation and maintenance phase; and, finally, a system retirement phase.
285
286 user acceptance testing. Verification of the fully-configured computerized system
287 installed in the production environment (or in a test environment equivalent to the production
288 environment) to perform, as intended, in the business process when operated by end-users
289 trained in end-user SOPs that define system use and control. User acceptance testing (UAT) may
290 be a component of the performance qualification (PQ) or a validation step separate from the PQ.
291
292 user requirements specification. The user requirements specification (URS), if prepared
293 as a separate document, is a formal document that defines the requirements for use of the
294 computerized system in its intended production environment.
295
296 4. COMPUTERIZED SYSTEM VALIDATION PROTOCOLS AND REPORTS
297
298 4.1 A computerized system needs to be validated according to an approved protocol and a
299 final report including results and conclusions prior to routine use.
300
301
 Working document QAS/16.667/Rev.2
page 12

302 Validation protocol

303
304 4.2 Validation should be executed in accordance with the validation protocol and applicable
305 written procedures.
306
307 4.3 A validation protocol should define the objectives and the validation strategy, including
308 roles and responsibilities and documentation and activities to be performed. The protocol should
309 at least cover the scope, risk management approach, the specification, acceptance criteria,
310 testing, review and release of the computerized system for GMP use.
311
312 4.4 The validation protocol should be tailored to the system type, impact, risks and
313 requirements applicable to the system for which it governs validation activities.
314
315 Validation report
316
317 4.5 A validation report should be prepared summarizing system validation activities.
318
319 4.6 The report should make reference to the protocol, outline the validation process, and
320 include an evaluation and conclusion on results. Deviations from the validation protocol and
321 applicable written procedures should be described, investigated, assessed and justification for
322 their acceptance or rejection should be documented. A validation report should also include a
323 summary of procedures and training.
324
325 4.7 Test results should be recorded, reviewed, analyzed and compared against the
326 predetermined acceptance criteria. All critical and major test discrepancies that occurred during
327 the verification/validation testing should be investigated and, if accepted, they should be
328 appropriately justified.
329
330 4.8 The conclusion of the report should state whether or not the outcome of the validation
331 was considered successful and should make recommendations for future monitoring where
332 applicable. The report should be approved after appropriately addressing any issue identified
 Working document QAS/16.667/Rev.2
page 13

333 during validation and the system should then be released for GMP use.
334
335 5. VENDOR MANAGEMENT
336
337 5.1 When third parties (for example, vendors, service providers) are used, such as, to
338 provide, install, configure, validate, maintain, modify or retain a computerized system or related
339 service, or for data processing or system components, including cloud-based systems. An
340 evaluation of the vendor-supplied system or service and the vendor’s quality systems should be
341 conducted and recorded. The scope and depth of this evaluation should be based upon risk
342 management principles.
343
344 5.2 The competence and reliability of a vendor are key factors when selecting a product
345 and/or service provider. Vendor management is an ongoing process that requires periodic
346 assessment and review. Vendor evaluation activities may include, but are not limited to:
347 completion of a quality-related questionnaire by the vendor; gathering of vendor documentation
348 related to system development, testing and maintenance including vendor procedures,
349 specifications, system architecture diagrams, test evidence, release notes and other relevant
350 vendor documentation; an on-site audit of the vendor’s facilities should be conducted to evaluate
351 the vendor’s system life-cycle control procedures, practices and documentation.
352
353 5.3 A contract should be in place between the manufacturer and the vendor, and/or the
354 service provider defining the roles and responsibilities and quality procedures for both parties,
355 throughout the system life cycle. The contract acceptor should not pass to a third party any of
356 the work entrusted to her/him under the contract without the manufacturer’s prior evaluation and
357 approval of the arrangements.
358
359 6. REQUIREMENTS SPECIFICATIONS
360
361 6.1 Requirements specifications should be written to document user requirements and
362 functional or operational requirements and performance requirements. Requirements may be
363 documented in separate URS and functional requirements specifications (FRS) documents or in
 Working document QAS/16.667/Rev.2
page 14

364 a combined document.

365
366 User requirements specifications
367
368 6.2 The authorized URS document, or equivalent, should describe the intended uses of the
369 proposed computerized system and should define critical data and data life cycle controls that will
370 assure consistent and reliable data throughout the processes by which data is created, processed,
371 transmitted, reviewed, reported, retained and retrieved and eventually disposed. The URS should be
372 written in a way to ensure that the data will meet regulatory requirements such as the WHO Guidance
373 on Good Data and Record Management Practices (5).
374
375 6.3 Other aspects that should be specified include, but are not limited to, those related to:
376
377  the transaction or data to be entered, processed, reported, stored and retrieved by the
378 system, including any master data and other data considered to be the most critical to system
379 control and data output;
380  the flow of data including that of the business process(es) in which the system will be
381 used as well as the physical transfer of the data from the system to other systems or
382 network components. Documentation of data flows and data process maps are
383 recommended to facilitate the assessment and mitigation and control of data integrity
384 risks across the actual, intended data process(es);
385  networks and operating system environments that support the data flows;
386  how the system interfaces with other systems;
387  the operating program;
388  synchronization and security controls of time/date stamps;
389  controls of both the application software as well as operating systems to assure
390 system access only to authorized persons;
391  controls to ensure that data will be attributable to unique individuals (for example, to
392 prohibit use of shared or generic login credentials);
393  controls to ensure that data is legibly and contemporaneously recorded to durable
394 (“permanent”) media at the time of each step and event and controls that enforce the
 Working document QAS/16.667/Rev.2
page 15

395 sequencing of each step and event (for example, controls that prevent alteration of
396 data in temporary memory in a manner that would not be documented);
397  controls that assure that all steps that create, modify or delete electronic data will be
398 recorded in independent, computer-generated audit trails or other metadata or
399 alternate documents that record the “what” (for example, original entry), “who” (for
400 example, user identification), “when” (for example, time/date stamp) and “why” (for
401 example, reason) of the action;
402  backups and the ability to restore the system and data from backups;
403  the ability to archive and retrieve the electronic data in a manner that assures that the
404 archive copy preserves the full content of the original electronic data set, including
405 all metadata needed to fully reconstruct the GMP activity. The archive copy should
406 also preserve the meaning of the original electronic data set;
407  input/output checks, including implementation of procedures for the review of
408 original electronic data and metadata, such as audit trails;
409  controls for electronic signatures;
410  alarms and flags that indicate alarm conditions and invalid and altered data in order
411 to facilitate detection and a review of these events;
412  system documentation, including system specifications documents, user manuals and
413 procedures for system use, data review and system administration;
414  system capacity and volume requirements based upon the predicted system usage and
415 performance requirements;
416  performance monitoring of the system;
417  controls for orderly system shutdown and recovery; and
418  business continuity.
419
420 6.4 The extent and detail of the requirements should be commensurate with the operational
421 risk and the complexity of the computerized system. User requirements should be specific and
422 be phrased in a way to support their testing or verification within the computerized system’s
423 context.
424
425
 Working document QAS/16.667/Rev.2
page 16

426 Functional specifications

427
428 6.5 Functional specifications should describe in detail the functions, performances and
429 interfaces of the computerized system based upon technical requirements needed to satisfy user
430 requirements.
431
432 6.6 The functional specifications provide a basis for the system design and configuration
433 specifications. Functional specifications should consider requirements for operation of the
434 computerized system in the intended computing environment, such as functions provided by
435 vendor-supplied software, as well as functions required for user business processes that are not
436 met by COTS functionality and default configurations that will require custom code
437 development. Network infrastructure requirements should also be taken into account. Each
438 described function should be verifiable.
439
440 6.7 Personnel access roles that provide the ability and/or authorization to write, alter or
441 access programs should be defined and qualified. There should be appropriate segregation of
442 roles between personnel responsible for the business process and personnel for system
443 administration and maintenance.
444
445 7. SYSTEM DESIGN AND CONFIGURATION SPECIFICATIONS
446
447 7.1 System design and configuration specifications should be developed based on user and
448 functional requirements. Specification of design parameters and configuration settings (separate
449 or combined) should ensure data integrity and compliance with the WHO Guidance on Good
450 Data and Record Management Practices (5).
451
452 7.2 System design and configuration specifications should provide a high-level system
453 description, as well as an overview of the system physical and logical architecture, and should
454 map out the system business process and relevant work flows and data flows if these have not
455 already been documented in other requirements specifications documents.
456
 Working document QAS/16.667/Rev.2
page 17

457 7.3 The system design and configuration specifications may include, as applicable, a
458 software design specification in case of code development and configuration specifications of
459 the software application parameters, such as security profiles, audit trail configuration, data
460 libraries and other configurable elements.
461
462 7.4 In addition, the system design and configuration specifications may also include, based
463 upon risk, the hardware design and its configuration specifications as well as that of any
464 supporting network infrastructure.
465
466 7.5 System design and configuration specifications should include secure, protected,
467 independent computer-generated audit trails to track configuration changes to critical settings in
468 the system.
469
470 8. DESIGN QUALIFICATION
471
472 8.1 Following design qualification (DQ), a review should be conducted to verify that the
473 proposed design and configuration of the system is suitable for its intended purpose and will
474 meet all applicable user and FRS.
475
476 8.2 It may include a review of vendor documentation, if applicable, and verification that
477 requirements specifications are traceable to proposed design and configuration specifications.
478
479 9. SYSTEM DEVELOPMENT AND PROJECT IMPLEMENTATION
480
481 9.1 Once the system requirements and the system design and configuration are specified and
482 verified, where applicable, system development activities may begin. The development
483 activities may occur as a dedicated phase following completion of specification of system
484 requirements, design and configuration. Alternatively, development activities may occur
485 iteratively as requirements are specified and verified (such as when prototyping or rapid-
486 development methodologies are employed).
487
 Working document QAS/16.667/Rev.2
page 18

488 Vendor-supplied systems

489
490 9.2 For vendor-supplied systems, the development controls for the vendor-supplied portion
491 of the computerized system should be assessed during the vendor evaluation or supplier
492 qualification. For vendor-supplied systems that include custom components (such as custom-
493 coded interfaces or custom report tools) and/or require configuration (such as configuration of
494 security profiles in the software or configuration of the hardware within the network
495 infrastructure), the system should be developed under an appropriate documented quality
496 management system.
497
498 Custom-developed systems
499
500 9.3 For custom-developed systems and configurable systems, the system should be
501 developed under an appropriate documented quality system. For these systems or modules, the
502 quality management system controls should include development of code in accordance with
503 documented programing standards, review of code for adherence to programing standards, and
504 design specifications and development testing that may include unit testing and
505 module/integration testing.
506
507 9.4 System prototyping and rapid, agile development methodologies may be employed
508 during the system build and development testing phase. There should be an adequate level of
509 documentation of these activities.
510
511 Preparation for the system qualification phases
512
513 9.5 The system development and build phase should be followed by the system qualification
514 phase. This typically consists of installation, operational and performance testing. Actual
515 qualification required may vary depending on the scope of the validation project as defined in
516 the validation plan and based upon a documented and justified risk assessment.
517
518
 Working document QAS/16.667/Rev.2
page 19

519 9.6 Prior to the initiation of the system qualification phase, the software program and
520 requirements and specifications documents should be finalized and subsequently managed under
521 formal change control.
522
523 9.7 Persons who will be conducting the system qualification should be trained to adhere to
524 the following requirements for system qualification:
525
526  test documentation should be generated to provide evidence of testing;
527  test documentation should comply with good documentation practices; and
528  any discrepancies between actual test results and expected results should be
529 documented and adequately resolved based upon risk prior to proceeding to subsequent
530 test phases.
531
532 10. INSTALLATION QUALIFICATION
533
534 10.1 Installation qualification (IQ) - also referred to as installation verification testing -
535 should provide documented evidence that the computerized system, including software and
536 associated hardware, is installed and configured in the intended system test and production
537 environments according to written specifications.
538
539 10.2 The IQ will verify, for example, that the computer hardware on which the software
540 application is installed has the proper firmware and operating system, that all components are
541 present and in the proper condition, and that each component is installed per the manufacturer or
542 developer instructions.
543
544 10.3 IQ should include verification that configurable elements of the system are appropriately
545 set as specified. Where appropriate, this could also be done during operational qualification
546 (OQ).
547
548
549
 Working document QAS/16.667/Rev.2
page 20

550 11. OPERATIONAL QUALIFICATION

551
552 11.1 The OQ - or operational/functional verification testing - should provide documented
553 evidence that software and hardware function is intended over anticipated operating ranges.
554
555 11.2 Functional testing should include, based upon risk:
556
557  challenges on the system's ability to do what it should do, including verification
558 that significant alerts and error messages are raised based upon alarm conditions and
559 according to specifications; and
560  an appropriate degree of testing (such as boundary, range, limit, and nonsense
561 entry testing) to verify the system appropriately handles erroneous entries or erroneous
562 use.
563
564 12. STANDARD OPERATING PROCEDURES AND TRAINING
565
566 12.1 Prior to the conduct of the PQ and UAT, and prior to the release of the computerized
567 system, there should be adequate written procedures and documents and training programmes
568 created defining system use and control. These may include vendor-supplied user manuals as
569 well as SOPs and training programs developed in-house.
570
571 12.2 Procedures and training programs that should be developed include, but are not
572 necessarily limited to:
573
574  System use procedures that address:
575 ‒ routine operation and use of the system in the intended business
576 process(es);
577 ‒ review of the electronic data and associated metadata (such as audit trails)
578 and how the source electronic records will be reconciled with printouts, if any;
579 ‒ mechanisms for signing electronic data; and
580 ‒ system training requirements prior to being granted system access.
 Working document QAS/16.667/Rev.2
page 21

581  System administration procedures that address:

582 ‒ granting and disabling user access and maintaining security controls;
583 ‒ backup/restore;
584 ‒ archival/retrieval;
585 ‒ disaster recovery and business continuity;
586 ‒ change management;
587 ‒ incident and problem management; and
588 ‒ system maintenance.
589
590 13. PERFORMANCE QUALIFICATION AND USER ACCEPTANCE TESTING
591
592 13.1 PQ, that includes UAT, should be conducted to verify the intended system use and
593 administration defined in the URS and DQ, or equivalent document.
594
595 13.2 The PQ should be conducted in the live environment or in a test environment that is
596 equivalent to the live environment in terms of overall software and hardware configuration.
597
598 13.3 PQ testing should also include, as applicable, an appropriate degree of
599 stress/load/volume testing based upon the anticipated system use and performance requirements
600 in the production environment.
601
602 13.4 In addition, an appropriate degree of end-to-end or regression testing of the system
603 should be conducted to verify the system performs reliably when system components are
604 integrated in the fully-configured system deployed in the production environment.
605
606 13.5 UAT should be conducted by system users to verify the adequacy of system, use of
607 SOPs and training programs. The UAT should include verification of the ability to generate and
608 process only valid data within the computerized system, including the ability to efficiently
609 review electronic data and metadata, such as audit trails.
610
611
 Working document QAS/16.667/Rev.2
page 22

612 Legacy systems

613
614 13.6 The continued use of a legacy system should be justified by demonstrating the system
615 continues to be relevant to the GMP process being supported and by ensuring adequate
616 validation of the system has been performed.
617
618 13.7 The validation approach to be taken should aim at providing data and information to
619 support the retrospective documentation of the system. It should demonstrate the system remains
620 in a state of control and is fit for its intended use and, where necessary, it should include an
621 approach for retrospective qualification of the system with relevant evidence.
622
623 13.8 A risk assessment should be undertaken to determine the criticality of the system to the
624 process or operation being supported and a gap analysis should identify the level of completeness
625 of existing qualification documentation (for example, URS, IQ/OQ/PQ, SOPs) and state of
626 system control, operation and maintenance.
627
628 13.9 For legacy systems, because of their age and unique characteristics, the system
629 development documentation and records appropriate for validation may not be available.
630 Nevertheless, the strategy should be consistent with validation principles where assurance is
631 established, based on compilation and formal review of the history of use, maintenance, error
632 report and change control system records. These activities should be based on documented URS.
633 If historical data do not encompass the current range of operating parameters, or if there have
634 been significant changes between past and current practices, then retrospective data would not of
635 itself support validation of the current system.
636
637 13.10 The validation exercise should demonstrate that user requirements and system
638 description have been appropriately established, as well as provide evidence that the system has
639 been qualified and accepted and that GxP requirements are met.
640
641
 Working document QAS/16.667/Rev.2
page 23

642 14. SYSTEM OPERATION AND MAINTENANCE

643
644 Security and access control
645
646 14.1 Manufacturers should have systems and procedures in place to ensure security of data
647 integrity and access control to computerized systems.
648
649 14.2 Suitable security measures should be in place to prevent unauthorized entry or
650 manipulation or deletion of data through both the application software, as well as in operating
651 system environments in which data may be stored or transmitted. Data should be entered or
652 amended only by persons authorized to do so.
653
654 14.3 The activity of entering data, changing or amending incorrect entries and creating
655 backups should be done in accordance with SOPs.
656
657 14.4 Security should extend to devices used to store programs. Access to these devices
658 should be controlled.
659
660 14.5 Procedures for review of audit trails and when necessary metadata, should define the
661 frequency, roles and responsibilities and nature of these reviews.
662
663 14.6 Actions, performance of the system and acquisition of data should be traceable and
664 should identify the persons who made entries and or changes, approved decisions or performed
665 other critical steps in system use or control.
666
667 14.7 Details on user profiles, access rights to systems, networks, servers, computerized
668 systems and software should be documented and an up-to-date list on the individual user rights
669 for the software, individual computer systems and networks should be maintained and subjected
670 to change control. The level of detail should be sufficient to enable computer system validation
671 personnel, information technology (IT) personnel/any external auditor/inspector to ascertain that
672 security features of the system and of software used to obtain and process critical data cannot be
 Working document QAS/16.667/Rev.2
page 24

673 circumvented.
674
675 14.8 All GMP computerized systems, either stand-alone or in a network, should have a
676 system commensurate to identified risks for monitoring through an audit trail events that are
677 relevant. These events should include all elements that need to be monitored to ensure that the
678 integrity (5) of the data could not have been compromised, such as but not limited to, changes in
679 data, deletion of data, dates, times, backups, archives, changes in user access rights,
680 addition/deletion of users and logins, in accordance with WHO Guidance on Good Data and
681 Record Management Practices (5). The configuration and archival of these audit trails should
682 be documented and also be subjected to change control. These audit trails should be accurate,
683 consistent, secure and available throughout the retention period and their generation
684 appropriately qualified.
685
686 Operation and maintenance
687
688 14.9 Entry of data into a computerized system should be verified by an independent
689 authorized person and locked before release for routine use.
690
691 14.10 Validated computerized systems should be maintained in a validated state once released
692 to the GxP production environment.
693
694 14.11 There should be written procedures governing system operation and maintenance,
695 including, for example:
696
697  performance monitoring;
698  change management and configuration management;
699  problem/incident management;
700  program and data security;
701  program and data backup/restore and archival/retrieval;
702  system administration and maintenance;
703  data flow and data life cycle;
 Working document QAS/16.667/Rev.2
page 25

704  system use and review of electronic data and metadata (such as audit trails);
705  personnel training;
706  disaster recovery and business continuity;
707  availability of replacement parts and technical support; and
708  periodic re-evaluation.
709
710 Data Migration
711
712 14.12 Where electronic data are transferred from one system to another, it should be
713 demonstrated that data are not altered during the migration process. Conversion of data to a
714 different format should be considered as data migration. Where data are transferred to another
715 medium, data must be verified as an exact copy prior to any destruction of the original data.
716
717 14.13 Data migration procedures may vary greatly in complexity and measures to ensure
718 appropriate transfer of data should be commensurate to identified risks. Migrated data should
719 remain usable and should retain its content and meaning. The value and/or meaning of and links
720 between a system audit trail and electronic signatures should be ensured in a migration process.
721
722 Periodic review
723
724 14.14 Computerized systems should be periodically reviewed to determine whether the system
725 remains in a validated state or whether there is a need for revalidation. The scope and extent of
726 the revalidation should be determined using a risk-based approach. The review should at least
727 cover:
728
729  maintenance and calibration;
730  review of changes;
731  review of deviations;
732  review of incidents/events (including review of audit trail);
733  systems documentation;
734  procedures;
 Working document QAS/16.667/Rev.2
page 26

735  training; and

736  effectiveness of corrective and preventive action (CAPA);
737
738 14.15 CAPA should be taken where indicated as a result of the periodic review.
739
740 14.16 Automatic or live updates should be subject to review prior to becoming effective.
741
742 15. SYSTEM RETIREMENT
743
744 15.1 System retirement should be considered as a system life cycle phase. It should be
745 planned, risk-based and documented. If migration or archiving of GMP-relevant data (4) is
746 necessary, the process must be documented.
747
748 15.2 Once the computerized system or components are no longer needed, the system or
749 components should be retired and decommissioned in accordance with established authorized
750 procedures, including a change control procedure and a formal plan for retirement.
751
752 15.3 Records should be in a readable form and in a manner that preserves the content and
753 meaning of the source electronic records throughout the required records retention period.
754
755 15.4 The outcome of the retirement activities, including traceability of the data and
756 computerized systems, should be documented in a report.
757
758 16. REFERENCES
759
760 1. Supplementary Guidelines on Good Manufacturing Practices: Validation. WHO Technical Report Series,
761 No. 937, 2006, Annex 4.
762
763 2. Supplementary Guidelines on Good Manufacturing Practices: Validation. Qualification of Systems and
764 Equipment. WHO Technical Report Series, No. 937, 2006, Annex 4, Appendix 7 (update in progress -
765 QAS/16.673/Rev.1).
766
 Working document QAS/16.667/Rev.2
page 27

767 3. WHO Good Manufacturing Practices: Water for Pharmaceutical Use. WHO Technical Report Series, No.
768 970, 2012, Annex 2.
769
770 4. WHO Good Manufacturing Practices for Pharmaceutical Products: Main Principles. WHO Technical
771 Report Series, No. 986, 2014, Annex 2.
772
773 5. Guidance on Good Data and Record Management Practices. WHO Technical Report Series, No. 996,
774 2016, Annex 5.
775
776 Further reading
777
778 Organization for Economic Co-Operation and Development (OECD) series on Principles of Good Laboratory
779 Practice and Compliance Monitoring, No. 17. Advisory document of the Working Group on Good Laboratory
780 Practice (GLP) and Application of GLP Principles to Computerised Systems, 2016.
781
782 Computerised systems. In: The Rules Governing Medicinal Products in the European Union. Volume 4: Good
783 Manufacturing Practice (GMP) Guidelines: Annex 11. Brussels: European Commission:
784 (http://ec.europa.eu/enterprise/pharmaceuticals/eudralex/vol-4/pdfs-en/anx11en.pdf).
785
786 Drug Information Association. Computerized Systems Used in Non-Clinical Safety Assessment; Current Concepts
787 in Validation and Compliance. Horsham, PA: Drug Information Association (DIA), 2008.
788
789 GAMP® – A Risk-Based Approach to Compliant GxP Computerized Systems. Tampa, FL: GAMP Forum,
790 International Society for Pharmaceutical Engineering (ISPE); 2008.
791
792 GAMP® Good Practice Guide: A Risk-Based Approach to GxP Compliant Laboratory Computerized Systems, 2nd
793 edition. Tampa (FL): International Society for Pharmaceutical Engineering (ISPE); 2012.
794
795 GAMP® Good Practice Guide: A Risk-Based Approach to GxP Process Control Systems, 2nd edition. Tampa (FL):
796 International Society for Pharmaceutical Engineering (ISPE); 2011.
797
798 GAMP® Good Practice Guide: A Risk-Based Approach to Operation of GxP Computerized Systems – A
799 Companion Volume to GAMP®5. Tampa (FL): International Society for Pharmaceutical Engineering (ISPE);
800 2010.
801
802 GAMP® Good Practice Guide: A Risk-Based Approach to Regulated Mobile Applications. Tampa (FL):
803 International Society for Pharmaceutical Engineering (ISPE); 2014.
 Working document QAS/16.667/Rev.2
page 28

804
805 GAMP® Good Practice Guide: A Risk-Based Approach to Testing of GxP Systems, 2nd edition. Tampa (FL):
806 International Society for Pharmaceutical Engineering (ISPE); 2012.
807
808 GAMP® Good Practice Guide: Global Information Systems Control and Compliance. Tampa (FL): International
809 Society for Pharmaceutical Engineering (ISPE); 2005.
810
811 GAMP® Good Practice Guide: IT Infrastructure Control and Compliance. Tampa (FL): International Society for
812 Pharmaceutical Engineering (ISPE); 2005.
813
814 GAMP® Good Practice Guide: Manufacturing Execution Systems – A Strategic and Program Management
815 Approach. Tampa (FL): International Society for Pharmaceutical Engineering (ISPE); 2010.
816
817 National Institute of Standards and Technology, U.S. Department of Commerce, (NIST) Cloud Computing
818 References: http://www.nist.gov/itl/cloud/index.cfm.
819
820 Official Medicines Control Laboratories Network of the Council of Europe: Quality assurance documents:
821 PA/PH/OMCL (08) 69 3R – Validation of Computerised Systems – core document:
822 (https://www.edqm.eu/sites/default/files/medias/fichiers/Validation_of_
823 Computerised_Systems_Core_Document.pdf) and its annexes:
824
825  PA/PH/OMCL (08) 87 2R – Annex 1: Validation of Computerised Calculation Systems: Example of
826 Validation of In-House Software:
827 (https://www.edqm.eu/sites/default/files/medias/fichiers/NEW_Annex_1_Validation_of_computerise
828 d_calculation.pdf).
829  PA/PH/OMCL (08) 88 R – Annex 2: Validation of Databases (DB), Laboratory Information
830 Management Systems (LIMS) and Electronic Laboratory Notebooks (ELN):
831 (https://www.edqm.eu/sites/default/files/medias/fichiers/NEW_Annex_2_Validation_of_Databases_
832 DB_Laboratory_.pdf).
833
834  PA/PH/OMCL (08) 89 R – Annex 3: Validation of Computers as Part of Test Equipment:
835 (https://www.edqm.eu/sites/default/files/medias/fichiers/NEW_Annex_3_Validation_of_computers_a
836 s_part_of_tes.pdf).
837
838  PA/PH/OMCL (08) 69 R7 - Annex 17: Application of GLP Principles to Computerised Systems:
839 (http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=env/jm/mono(2016)13&do
840 clanguage=en).
 Working document QAS/16.667/Rev.2
page 29

841 Title 21, Code of Federal Regulations (21 CFR Part 11): Electronic records; electronic signatures. US Food and
842 Drug Administration. The current status of 21 CFR Part 11 Guidance is located under Regulations and Guidance
843 at: http://www.fda.gov/cder/gmp/index.htm — see background: http://www.fda.gov/OHRMS/DOCKETS/98fr/03-
844 4312.pdf.
845
846 PIC/S guide to good manufacturing practice for medicinal products annexes: Annex 11 – Computerised systems.
847 Geneva: Pharmaceutical Inspection Co-operation Scheme.
848
849 PIC/S PI 011-3, Good Practices for Computerised Systems in Regulated GxP Environments. Geneva:
850 Pharmaceutical Inspection Co-operation Scheme
851
852 ***