Smartlogic

Menu

תגית: SAT

ולידציה – GAMP – Definition of Terms

Posted on by אביב צברי

Good Automated Manufacturing Practice (GAMP) Definition of Terms

Definition of Terms Used in Testing Environments

This document provides a definition of a set of testing terms used within pharmaceutical and other life sciences (consistent with those used in GAMP 4), Information Technology (IT) industries, and control and automation industries, in order to facilitate understanding testing environments.

It is recommended that a definition of consistent testing terms should be prepared on an organizational or project basis, where members of User and Supplier organizations work together. It is helpful to agree on these testing terms definition prior to contract signing, to ensure that contractual issues are based on common understanding of activities and milestones.

:The definition of terms listed below is based on three sources

GAMP 4 – Definition from the GAMP Guide for the Validation of Automated Systems

IEEE – Definition from IEEE 100, the Authoritative Dictionary of IEEE Standard Terms

BCS – Definition from Working Draft: Glossary of terms used in software testing, version 6.2, produced by the British Computer Society Interest Group in Software Testing – BCS SIGIST

Terms and Definitions

Terminology Definition Source
Acceptance Criteria Criteria that a system or component must satisfy in order to be accepted by the User, customer or other authorized entity  GAMP 4 , IEEE
Acceptance Test

Formal testing conducted to determine whether or not a system satisfies its acceptance criteria, and to enable the customer to determine whether or not to accept the system

See also Factory Acceptance Test (FAT) and
Site Acceptance Test SAT

  ,GAMP 4 IEEE
Black Box Testing See Functional Testing IEEE
Boundary Condition Testing Testing for correct operation when one or more variables are at a limiting value or a value at the edge of the domain of interest IEEE
Calibration Set of operation that establish, under specified conditions, the relationship between values indicated by a measuring instrument or system, or values represented by a material measure or a reference material, and the corresponding values of a quantity realized by a reference standard  GAMP 4,   ISO 10012

 

 

Terminology Definition Source
Challenge Testing Testing to check system behavior under abnormal conditions. Can include stress testing and deliberate challenges, e.g., to the security access system, data formatting rules, possible combinations of operator's actions, etc
Commissioning Process of providing to the appropriate components the information necessary for the designed communication between them IEEE
Emulation A model that accepts the same inputs and produces the same outputs as the given system IEEE
Environmental Testing Testing that evaluates system or component performance up to the specified limits of environmental parameters, e.g., temperature, humidity or pressure
Firmware FW Combination of hardware (HW) device, computer instructions and data that reside as read-only software (SW) on that device IEEE
Factory Acceptance Test FAT

Acceptance Test in the Supplier's Factory, usually involving the customer

See also Factory Acceptance Contrast to Site Acceptance Test – SAT

 GAMP 4 , IEEE
Functional Testing Testing that ignores the internal mechanism of a system or component, and focuses solely on the outputs generated in response to input and execution conditions
Also known as Black Box Testing		 GAMP 4 , IEEE
Hardware- HW

Physical equipment used to process, store, or transmit computer programs or data

Physical equipment used in data processing, as opposed to programs, procedures, rules, and associated documentation

 IEEE
HW Testing Testing carried out to verify the correct operation of system HW independent of any custom application SW IEEE
Installation Qualification – IQ Documented verification that a system is installed according to written and pre-approved specifications  GAMP 4 , IEEE
Integration

Process of combining SW components, HW components, or both, into an overall system

Sometimes described as SW Integration and System Integration, respectively

 IEEE

 

Terminology Definition Source
Integration Testing

(1) Testing in which SW components, HW components, or both, are combined and tested to evaluate the integration between them

(2) Orderly progression of testing of incremental pieces of the SW program, in which SW elements, HW elements, or both, are combined and tested until the entire system has been integrated to show compliance with the program designed, and the system capabilities and requirements

 IEEE
Instance Single installation of a SW application (plus associated databases, tools and utilities). Usually applied to configurable IT systems
Load Testing Stress testing conducted to evaluate a system or component up to the limits of its specified requirements
Loop Testing Testing in which control system inputs and outputs are exercised and their functionality verified
Market Requirements Specification Statement of generic industry requirements used by the Supplier as an input to its product development life cycle IEEE
Middleware Combination of HW, computer instructions and data that provides infrastructure used by other system modules IEEE
Module Testing Testing of individual HW or SW components, or groups of related components IEEE
Negative Testing Testing aimed at showing that the SW does not work BCS
Operational and Support Testing

(1) Testing conducted to evaluate a system or component in an operational environment

(2) All testing required to verify system operation in accordance with design after the major component is energized or operated

 IEEE
Operation Qualification- OQ Documented verification that a system operates according to written and pre-approved specifications throughout all specified operating range GAMP 4 , IEEE
Performance Qualification – PQ Documented verification that a system is capable of performing or controlling the activities of the processes it is required to perform or control, according to written and pre-approved specifications, whilst operating in its specified environment GAMP 4 , IEEE
Positive Testing Testing aimed at showing that the SW does meet the defined requirements BCS

 

 

Terminology Definition Source
Qualification Process to demonstrate the ability to fulfill specified requirements

GAMP 4

  ISO
Simulation A model that behave or operates like a given system when provided a set of given inputs IEEE
Site Acceptance Test – SAT

Acceptance Test at the customer's Site, usually involving the customer

See also Factory Acceptance Contrast to Site Acceptance Test – SAT

 GAMP 4,

IEEE
Software – SW Computer programs, procedures, and associated documentation and data pertaining to the operation of a computer system IEEE
Stress Testing Testing conducted to evaluate a system or component at or beyond the limits of its specified requirements IEEE
Structural Testing Examining the internal structure of the source code. Includes the low-level and high-level code review, path analysis, auditing of programming procedures, and standards actually used, inspection for extraneous "dead code", boundary analysis and other techniques. Requires specific computer science and programming expertise
Also known as White Box Testing		 GAMP 4
Supplier Organization or person that provides a product   GAMP 4,  ISO
System Testing Testing conducted on a complete, integrated system to evaluate its compliance with its specified requirements GAMP 4
Test

 Procedure in which by executes an activity under specified conditions, the results are observed and recorded, and an evaluation is made of some aspect of that system or component

 Determination of one or more characteristics according to a procedure

,GAMP 4

ISO

GAMP 4

 IEEE
Test Case Set of test inputs, execution conditions and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement

GAMP 4 ,

 IEEE
Test Plan Document describing the scope, approach, resources and schedule of intended activities. It identifies test items, features to be tested, testing tasks, personnel en charged of each task, and risks requiring contingency planning GAMP 4

 

Terminology Definition Source
Test Procedure Detailed instructions for the setup, execution and evaluation of results for a given test case GAMP 4,   IEEE
Test Protocol See Test Specification IEEE
Test Script Documentation that specifies a sequence of actions for executing a given test GAMP 4 , IEEE
Test Specification Document describing the scope, management, use of procedures, sequencing, test environment, and prerequisites for a specific phase of testing
Test Strategy See Test Plan
Unit Testing Testing of individual HW or SW units or groups of related units IEEE
Usability Testing Testing the ease with which Users can learn and use a product BCS
User Person or persons who operate or interact directly with the system GAMP 4
Validation Establishing documented evidence which provides a high degree of assurance that a specific process will consistently produce a product meeting its predetermined specifications and quality attributes GAMP 4 , FDA
Verification Confirmation, through the provision of objective evidence, that specified requirements have been fulfilled GAMP 4 ,    ISO
White Box Testing See Structural Testing IEEE

ולידציה – Function Design Specification – FDS

Posted on by אביב צברי

ולידציה – Function Design Specification – FDS

 This article was written by Iian Shaya, validation,automation and control expert

The Function Design Specification (FDS) is part of the validation documentation that details the solution to be provided to meet the user's requirements. It should be approved by the user and should form the basis of the design for both hardware (HW) and software (SW) designs.

The FDS provides the basis of the design of the system and is used to verify and validate the system during the testing, ensuring all the required functions are present and that they operate correctly. It details all the functions, operator interactions control and sequencing associated with the system, thus allowing the user to confirm, before the system is developed that the proposed solution fully meets its requirements.

FDS Contents

The FDS is structured in a relatively standard fashion, with predetermined chapters and sections, where the final contents are tailored according to the type and size of the system under validation. The FDS presented here includes only to the technical contents. It does not include commercial and contractual requirements, which may also be generally included.

The main chapters and sections of an FDS protocol are:

Relationship to Other Documents – lists all documentation used in the production of the FDS. Includes suppliers' documents (such as URS) and drawings. Each document listed should include the document/drawing number and version number. This allows traceability as documents are updated throughout the project life cycle on any impact on the FDS.

System Overview

Process Overview – includes a description of the process being controlled; this may be taken from the URS, enhancing to detail the interaction with the control system.

Control System Overview – includes detailed control system description, with all the components and interaction between the systems; block and network diagrams can be used to show in detail the system architecture

Scope and Limits of Supply

Scope of Supply – includes a list of deliverables, panels, computers, software, etc

Limits of Supply – includes all items outside the scope of the supply required by the project; where interfacing to 3rd party systems, constraints and assumptions should be included

System Functions and Facilities

Operation Modes – includes all modes of operation for the system

Functional Operation – divides each of the sequences functions into logical areas (determined by the process), and provide complete description of each area

Operator Requirements – describes the interface between the operator and the detailed function

Human/Machine Interface- HMI – details all points of operation, local terminals, remote terminal, message displays, push button stations, etc.

Report Outputs – the format of all reports generated by the system should be detailed, and that the format and explanation of the report contents should be included

System Data – all data gathered, generated or calculated by the system should be detailed

System Interfaces – provide complete details of all inputs and outputs from the control system

System Attributes

Availability – defines expected "working" time of the system between failures

Maintainability – details issues related to maintainability of the plant, in particular for systems that require regular maintenance to ensure the reliable operation

Transport and off loading

Power and services required

Connections to existing/3rd party systems

Changes to existing plant or hardware (HW) equipment

Changes to existing software (SW) systems

Training – details the formal and informal training to be supplied under the contract

Design Factors – details special factors relating to the design of the system, standards and methodologies to be followed for both the HW and SW development

Development Factors

Project Control – includes or makes reference to project plans and timescales, along with details of quality requirements, standards, test and integration and configuration management

Resource Requirements – includes the basic project team provided by the supplier, the access required to the customer's premises, and input and timing required by the customer into the project

Test procedures – including details of all test documentation and responsibilities for testing both offline and online

Module and Integration Testing

Factory Acceptance Testing (FAT) – performed at the suppliers premises

Site Acceptance Testing (SAT) – performed on completion of commissioning to demonstrate pre-handover system operation

:Note

As the final contents of the FDS are tailored according to the type and size of the system under validation, and this document is generic, it covers test procedures that may not be necessary in small or simple systems. The following sections cover the FDS issues that require further details

 About the system functions and facilities in our next article FSD System Function & Facilities

 This article was written by Iian Shaya, validation,automation and control expert

אילן שעיה ilan Shaya

ולידציה -Testing Process Automation Systems

Posted on by אביב צברי

ולידציה – Testing Process Automation Systems  

 Testing Responsibilities  Supplier and User

Where a Supplier has been assessed and its quality management system found to be acceptable, the User may benefit from the testing already carried out as part of the product development life cycle in order to minimize additional testing.

Similarly, the User may benefit from the testing carried out by the Supplier as part of the application, for example, to allow a small sample of tests to be repeated at witnessed FAT.

                        Example of Life Cycle for a Custom Application

Supplier Product Life Cycle

For example in a life cycle where a new application is developed for an embedded control system,  the Supplier's product may be SW or tools used to develop the User specific application. The Supplier of the control system has been assessed and its quality management system found to be acceptable. Thus, the User does not need to repeat the testing of product functionality. The testing should instead concentrate on the application of the control system

     End User Application Life Cycle

Assuming that the application is critical to product quality and includes a custom sequence coded as a sequence function chart, a test approach could be agreed :that includes the following elements

Supplier's module testing of the sequence function chart program.

Supplier's integration testing (100% test) and FAT (sampled) to demonstrate correct interaction configuration of the control system and correct operation of the process equipment.

SAT and IQ/OQ/PQ of the control system as part of the process equipment.

                      Example of Life Cycle for a Standard Application

                   Supplier Product Life Cycle

In the case  of a life cycle where a standard application is purchased containing an embedded control system, the control system Supplier has been assessed and its quality management system found to be acceptable. Thus, the User does not need to repeat the testing of product functionality (including mature library functions such as standard control modules) or of the standard application. The testing has to cover only the setup and use of the application.

                    End User Application Life Cycle

Assuming that the application is still critical to product quality, there is now much lower risk associated with the application development, as the configuration is limited to selecting the required functions and entering setup parameters. A test approach could, therefore, be agreed including the following elements:

FAT to demonstrate correct setup of the control system and correct operation of the process equipment.

SAT and IQ/OQ/PQ of the control system as part of the process equipment.

ולידציה -Testing and HW/SW Type and Maturity

Posted on by אביב צברי

ולידציה -Testing and HW/SW Type and Maturity – GAMP

Process automation systems generally comprise several elements. Categorizing these elements according to those described in GAMP 4 can assist in targeting testing and validation effort.

Some functional elements may be fully contained within the system standard packages or firmware (FW), while others may require different degrees of configuration or customization. In general, the greater the degree of customization, the greater the scope of testing required.

For example, if an item of the process equipment is to be controlled, the User may have several options for implementing the process control system functional element.

:Typical Test Phases for Process Automation Systems

Complex control exists as single, mature 'equipment level' library module, where functions can be selected/ deselected and  setup parameters entered

For example, sterilizer module with selectable cycle types

Application Life Cycle Testing: selection of functions, setup parameters

:Typical Test Phases

FAT and /or

SAT

IQ/OQ/PQ of process equipment

Complex control built up from mature, interlinked 'device level' library modules, which must have correct parameters entered

For example, sterilizer control, built up from valve, motor, PID, set-point program modules.

Application Life Cycle Testing: selection and linking of modules functions to give correct functionality, setup parameters

:Typical Test Phases

FAT

SAT

IQ/OQ/PQ of process equipment

Complex control built up from application specific modules, themselves built up from low-level library modules, or from simple coded steps within a predefined structure.

For example, sterilizer control configured in ladder logic

Application Life Cycle Testing: functionality of custom coded modules, selection and linking of modules functions to give correct functionality, setup parameters

:Typical Test Phases

Module test

FAT

SAT

IQ/OQ/PQ of process equipment

Complex control custom added

For example, sterilizer control coded in C++

Application Life Cycle Testing: functionality of custom coded modules, interaction between custom coded modules, selection and linking of modules functions to give correct functionality,setup parameters.

:Typical Test Phases

Module test

Module integration test

FAT

SAT

IQ/OQ/PQ of process equipment

:Notes

The suggested test phases listed above are typical only; they may be increased or decreased to reflect the complexity of the application, and the impact on product quality, patient safety and data integrity

The above assumes mature, 'industry proven' library modules. If these are created specifically for the application, then they need to be treated as custom code

Smartlogic expertise in validation of current and new systems to meet the most rigid requirements of the FDA and CFR protocols.

ולידציה – GAMP – Test Example – part 3

Posted on by אביב צברי

Good Automated Manufacturing Practice (GAMP) – Test Example

Testing Process Automation Systems

This article cover  the third part of  our  Good Automated Manufacturing Practice (GAMP) test example. This part will cover  the typical test phases for Factory Acceptance Test – FAT, Site Acceptance Test – SAT / Installation Qualification – IQ, Operation Qualification – OQ and Performance Qualification – PQ

Typical Test Phases – GAMP – Test Example

 Typical test phases for a complex process automation system. This example assumes that the system is configured by a Supplier and delivered to site after a FAT. Test system can also be configured by the system integrator or by the User. In this case, the same coverage is required, but the test phasing and location may be different.

The User and Supplier should work together to develop an overall approach to testing that reflects the risk assessment output and ensures adequate test coverage of the functionality, whilst avoiding unnecessary repeated tests.

Factory Acceptance Test – FAT

Done at the Supplier's premises after Supplier's integration testing, and before the system is released for delivery to the User's site

The required coverage should reflect the relative risk priority associated with the system element under test. This coverage can increase if problem are found within the initial sample.

In determining the required coverage, the User needs to base decisions on the risk assessment output taking into account both the potential effect on product quality and safety resulting from the process, and the intrinsic risk likelihood associated with the method of implementation.

Before performing risk assessment to decide on the required coverage, the User should review the supplier's internal tests to confirm that they are adequately documented.

Site Acceptance Test – SAT / Installation Qualification – IQ

Done at the User's premises after installation of the system on site

:The required coverage should include

Checking that the full system, including HW, SW backups, and documentation has been delivered to site in a condition suitable to its intended purpose

Checking that the site environment suits the specification of the installed equipment: temperature, humidity, pressure, dust, vibration, interference, etc

Checking that the system has been correctly installed.

Demonstration the system still operates as it was when accepted during the FAT, typically by re-recording system components and versions, and by re-repeating a small sample of FATs on site

Testing any elements which could be adequately tested in the FAT environment, typically interfaces to other equipment, networks, etc

Re-testing, following remedial action on any element subject to conditional release at the end of the FAT

Operation Qualification – OQ and Performance Qualification – PQ

Done at the User's premises after SAT/IQ

If a system has been fully tested in the FAT, after successful completion of the IQ (along with any additional field functional tests and calibrations), the system is treated as an integrated part of the process equipment qualification. This should ensure that the full system, procedures and trained personnel are ready for production.

End of GAMP – Test Example – part 3.

 

ולידציה – GAMP – Test Example – part 1

Posted on by אביב צברי

Good Automated Manufacturing Practice (GAMP) – Test Example

Testing Process Automation Systems

This article cover  the first part of  our  Good Automated Manufacturing Practice (GAMP) test example.

                                     Definitions

This section provides brief descriptions of three different types of process control systems.

                  Configurable Equipment

Configurable Equipment is the collective name given to simple configurable instruments/ devices, such as 3-term controllers, check scales, bar code readers, etc. Their functionality depends on that their configuration setup meets the process requirements. The software (SW) components of these systems are typically defined as GAMP SW Category 2.

                    Embedded Systems

Embedded Systems is the collective name for systems with a greater degree of configuration and programmability. Devices such as Integrated Circuits (ICs) with configuration setups and Programmable Logic Controllers (PLCs), which are supplied as an integral part to an item of process equipment, e.g., PLCs controlling a centrifuge or packaging machine, or IC embedded in High Performance Liquid Chromatography (HPLC) systems. Embedded Systems typically contain SW components belonging to multiple GAMP categories.

                 Stand-Alone Systems

Stand-Alone Systems is the collective name for large programmable control systems having distributed functionality across a network, e.g., Distributed Control Systems (DCSs), and Supervisory Control and Data Acquisition (SCADA). They are engineered as an entity to control a complete plant. Stand-Alone Systems typically contain SW components belonging to multiple GAMP categories.

                      Testing and the GAMP Life Cycle 

     Stand-Alone Systems

A process automation system developed for a new application typically requires some or all of the following test phases:

Suppliers Module Testing

Suppliers Module Integration Testing

Suppliers Integration Testing

Factory Acceptance Test (FAT)

Site Acceptance Test (SAT)

Installation Qualification (IQ)

Operation Qualification (OQ)

Performance Qualification (PQ)

The exact combination of testing required for a particular system should reflect its complexity, the maturity of its underlying SW and hardware (HW) elements, and the risk impact on product quality, patient safety and data integrity. Collectively these will determine the risk priority. The phrase 'low risk' should be understood as 'having a low risk priority, as determined by a formal risk assessment'.

Testing of modifications, patches or upgrades should be related to the risk priority of the change. For example, it may be appropriate for parameter changes to be applied directly to the production environment, assuming that the system have been range checked for such parameter.

End of ולידציה – GAMP – Test Example – part 1