David Stokes

21.1 Introduction

There is no doubt that the use of free/libre open source software (FLOSS) can offer some significant advantages. Within the life sciences industry these include all of the advantages available to other industries such as the ability to use low or no cost software, but the use of open source software can also provide:

As significant changes are taking place within the life sciences industry (such as the move towards greater outsourcing and collaboration, the greater need to innovate and reduce the time to bring products to market), these advantages can be particularly beneficial in areas such as:

21.2 The need to validate open source applications

When such applications have a potential impact on product quality, patient safety or the integrity of regulatory critical data, some of these advantages can be offset by the need to appropriately validate the software.

There is a clear regulatory expectation that such applications are validated – stated in national regulations (e.g. US 21CFR Part 211 [4] and Part 820 [5]), regulatory guidance (e.g. EU Eudralex Volume 4 Annex 11 [6], PIC/S PI-011 [7]) and applicable industry standards (e.g. ISO 13485 [8]). These documents require that such systems are validated, but do not provide detailed requirements on how this is achieved.

Computer system (or software) validation is a process whereby clear and unambiguous user and functional requirements are defined and are formally verified using techniques such as design review, code review and software testing. This should all be governed as part of a controlled process (software development life cycle) defined in a validation plan, supported by appropriate policies and procedures and summarised in a validation report.

Although some regulatory guidance documents (i.e. FDA General Principles of Software Validation [9], and PIC/S PI-011) do provide a high-level overview of the process of computer system validation, industry has generally provided detailed guidance and good practices through bodies such as the International Society for Pharmaceutical Engineering (ISPE) and the GAMP® Community of Practice.

The need to validate such software or applications is not dependent on the nature of the software or how it is sourced – it is purely based on what the software or application does. If an application supports or controls a process or manipulates regulatory significant data that is within the scope of regulations (e.g. clinical trial data, product specifications, manufacturing and quality records, adverse events reporting data) or provides functionality with the potential to impact patient safety or product quality, there is usually a requirement to validate the software.

This also includes an expectation to ensure that the application or software fulfils the users stated requirements, which should, of course, include requirements to ensure compliance with regulations. Validating the software or application therefore provides a reasonably high degree of assurance that:

Not all open source software has regulatory significance, but where this is the case failure to validate such software can have serious consequences, which includes regulatory enforcement action. There have been cases where a serious failure to appropriately validate a computerised system has directly led to or has partly contributed to enforcement actions including failure to issue product licences, forced recall of products, import restrictions being imposed or, in the most serious cases, US FDA Consent Decrees [10], which have required life sciences companies to implement expensive corrective and preventative actions and pay penalties in the range of hundreds of millions of dollars for non-compliance.

More importantly, the failure to appropriately validate a computerised system may place product quality and patient safety at risk and there are several known instances where faulty software has led to indirect or direct harm to patients.

21.3 Who should validate open source software?

Although the regulations do not provide details on how software should be validated, those same regulatory expectations (and numerous enforcement actions by various regulatory agencies) make it clear that regulated companies (i.e. pharmaceutical, medical device, biotechnology, biological and biomedical manufacturers, CROs, etc.) are accountable for the appropriate validation of such software applications.

Industry guidance such as the widely referenced GAMP® Guide [11] is increasingly looking for ways in which regulated companies can exercise such accountabilities while at the same time leveraging the activities and documentation of their software vendors or professional services providers (systems integrators, engineering companies, etc.). The GAMP® Guide provides pragmatic good practice for the risk-based validation of computerised systems and is accompanied by a number of companion Good Practice Guides. Although all of these recommended approaches can be applied to the validation of open source software, there is very little specific guidance on the specific validation of such software.

Following one of the key concepts of the GAMP® Guide – leveraging supplier involvement, activities and documentation – presents a significant challenge for the validation of open source applications where there is often no software vendor with which to establish a contractual relationship and often no professional services provider who will assume responsibility for implementing and supporting open source software.

There are examples of third-party organisations providing implementation and support services for open source infrastructure (e.g. Linux Red Hat) or applications. This may not, however, be the case for many new or innovative applications which are at the leading edge of functionality and which may offer significant advantages to a life sciences organisation.

In most cases it is the regulated company which is not only accountable for the validation of the open source application but must also assume day-to-day responsibility for ensuring that the application is regulatory-compliant, for validating the application and for maintaining the application in a validated state with respect to their own operational use of the software. In the case of the same instance of open source software used by multiple regulated companies (e.g. Software as a Service running in a community cloud), it is theoretically possible for a number of regulated companies to form their own open source community and share responsibility for the validated state of the shared software, but this is a relatively new and untested model.

21.4 Validation planning

As with any software or application, the validation process starts with validation planning. Many regulated companies have resources to address this in-house, whereas small to medium organisations may prefer to take the acceptable step of using qualified third parties such as consultants.

Thorough and thoughtful validation planning is the key to successfully validating open source software. The use of open source software is different from the validation of commercially developed and supported applications, and the validation planning should be undertaken by experienced resources that understand and appreciate the differences and can plan the validation accordingly.

21.5 Risk management and open source software

As with any software application, in order to be cost-effective the validation of open source software should take a risk-based approach. This should follow established industry guidance such as ICH Q9, ISO 14971 or specifically the risk management approach described in appendix M3 of the GAMP® Guide.

An initial risk assessment should be conducted, as outlined in Figure 21.3, to facilitate the validation planning as described above. This should determine the overall risk severity of the open source software (with a focus on risks to patient safety, product quality and data integrity) and should consider the risks that result from both the package and supplier assessment.

As part of the implementation or adoption of open source software, a detailed functional risk assessment should be conducted as described in the GAMP® Guide. This is the same process as for any other software package and should focus on the risk severity of specific software functions. This is in order to focus verification activities on those software functions that pose the highest risk.

However, with open source software, additional package and supplier risks may need to be considered as outlined above. Although the nature of the open source software will not change the risk severity, there may be specific issues that affect risk probability or risk detectability. These include:

It is likely that any increased risk probability and decreased risk detectability will increase the resulting risk priority, requiring additional scope and rigour with respect to verification activities (design reviews, testing, etc.). For each of these risks appropriate controls need to be identified, understanding that it may not be possible to leverage any supplier activities and that the regulated company will be responsible for any risk control activities.

Additional risk controls for open source software may include:

The effectiveness of these risk controls should be monitored and reviewed to ensure that any risks are mitigated.

21.6 Key validation activities

Whatever the nature (categorisation) of the software, it is important that the regulated user defines and most probably documents clear requirements. Where there is little or no ability to change the functionality, it is acceptable for regulated company users to assess the package and confirm that it meets their requirements (Eudralex Volume 4 Annex 11). Even this will require a clear understanding of the business requirements, even if these are not documented in extensive user requirement specifications. It is recommended that the business process and data flows are documented as a minimum, against which the package can be assessed and this can be done at the package assessment stage.

For open source systems that are highly configurable, it is recommended that formal user requirements are documented and key areas are highlighted in Figure 21.4. It will be important to maintain these requirements in an up-to-date state as the use of the software changes over time. These may record user requirements in a variety of applicable formats including business process flows, use-cases, business rules and traditional singly stated user requirements (‘The system shall . . .’). It is usual to conduct a detailed functional risk assessment at this stage, examining each function (business process, use-case, etc.) to determine:

For functions that are determined to be regulatory significant and requiring validation, specifications will also need to be developed in order to document the setup (configuration) of the open source software by the specific regulated company. For COTS software this will include a record of the actual configuration settings (package configuration specification or application setup document). Where custom development is required this should also be documented in appropriate functional and technical design specifications.

It is worth noting here that where specific functions are developed by the open source community to meet the needs of a specific project, and where the community does not provide functional and technical design specifications, it is recommended that the regulated company’s involvement in the development includes the production of these documents. Not only will these provide the basis for better support and fault finding in the future, but they will also be of use to other life sciences companies wishing to use the same functional enhancements.

The nature of the open source movement implies a moral responsibility to share such developments with the wider community and it is common practice for developers in the open source community to ‘post’ copies of their software enhancements online for use by other users. While it is possible for life sciences companies to develop enhancements to open source software for their own sole benefit, this goes against the ethos of the movement and in some cases the open source licence agreement may require that such developments are made more widely available.

Where such functional enhancements require validation, the associated documentation (validation plans, requirements, specifications, test cases, etc.) is almost as useful as the software. Sharing such validation documentation (devoid of any company confidential or commercial details) fits well with the ethos of the open source movement and such documentation may be shared via open source forums or online repositories. Most open source software sites have a location where documentation can be shared and this practice is to be encouraged in line with the GAMP® key concept of leveraging ‘supplier’ documentation.

The open source community includes developers with widely varying experience. At one end there are professional software developers choosing to apply formal methods to the open source platform, and at the other are ‘amateurs’, developing software on a trial and error basis in their spare time, with little or no formal training and with little or no documentation.

Although all parties can make an active and valuable contribution to the development of open source software, there are inherent risks when regulatory significant applications are developed by those not formally trained in good software development practices. These risks may include:

Actual examples of ‘bad practices’ encountered in open source software developed in this way include:

Although it is possible that part-time, untrained developers can develop software that can be validated, this requires an understanding of both good software development practices and an understanding of software validation in the life sciences industry. Experience is that the development and validation of regulatory significant applications is best accomplished by real subject matter experts.

Where business process owners wish to contribute to the development of open source solutions, this is perhaps best achieved by defining clear requirements and executing user acceptance testing. In many cases the setup and use of the software may follow a ‘trial and error’ process – this may often be the case where documentation is poor or out of date. It is acceptable to follow such a prototyping process, but all requirements and specifications should be finalised and approved before moving into formal testing of the software.

Prior to formal testing, the software and associated documentation should be placed under change control, document and configuration management. It is unlikely that open source community controls will be adequate to provide the level of control required in the regulated life sciences industry (see below) and the regulated company will usually need to take responsibility for this.

It is worth noting that there are a small number of open source software tools on the market providing basic change control, configuration management, document management and test management functionality (e.g. Open Technology Real Services [24], and Project Open OSS [25] IT Service Management suites). Although these do not require formal validation, they should be assessed for suitability to ensure that they function as required, are secure and will be supported by the development community for the foreseeable future. Important areas to consider here are described in Figure 21.5.

Depending on the outcome of the risk assessment and the nature of the software, testing may consist of:

The use of open source software does not infer that there is a need to test to a lower standard. If anything, some of the additional risks arising from using open source software means that testing by the regulated company will be more extensive than in cases where testing from a commercial vendor (confirmed by a traditional supplier audit) can be relied on. All testing should follow accepted good practices in the life sciences industry, best covered in the GAMP® ‘Testing of GxP Systems’ Good Practice Guide. In some cases requirements cannot be verified by formal testing and other forms of verification should be leveraged as appropriate (i.e. design review, source code review, visual inspection, etc.). Supporting IT infrastructure (servers, operating systems, patches, database, storage, network components, etc.) should be qualified in accordance with the regulated company’s usual infrastructure qualification processes, and as required by Eudralex Volume 4, Annex 11.

All of the above should be defined in a suitable validation plan and reported in a corresponding validation report. Based on accepted principles, any of the above documentation may be combined into fewer documents for small or simple software applications, or alternative, equivalent documentation developed according to the applicable open source documentation conventions. Where the software is used within the European Union or used in a facility that complies with EU regulations – and soon to be any country that is a member of the Pharmaceutical Inspection Cooperation Scheme (PIC/S) – the documentation should also include a clear System Description (Eudralex Volume 4 Annex 11). The use of the open source software as a validated application also needs to be recorded in the regulated company’s inventory of software applications (Eudralex Volume 4, Annex 11).

In summary, the validation of open source software should follow a scalable, risk-based approach, just as any commercial software package. The only difference is that the inability to rely on a commercial vendor means that the regulated company may need to validate with broader scope and greater rigour than would otherwise be the case. This is likely to increase the cost of validation, which will offset some of the cost savings inherent in the use of open source software. This is why this needs to be assessed after the package and ‘supplier’ assessment – to confirm that the financial advantages for using open source software are still valid, despite the cost of validation.

21.7 Ongoing validation and compliance

Once the software has been initially validated, it is essential that it is maintained in a validated state and subject to periodic review and, in the case of any changes, appropriate re-validation (which may include regression testing of unchanged functions). This follows well-defined principles and processes best defined in the GAMP® Good Practice Guide ‘A Risk-Based Approach to Operation of GxP Computerized Systems’. Once again, the use of open source software does not provide a rationale for not following the processes defined in this guidance document, which are:

It is, of course, acceptable to scale these processes based on the risk severity, size and complexity of the open source software.

Key to maintaining the validated state is the use of an effective change control and configuration management process. This is often seen as contrary to the ethos of some parts of the open source community, where it is possible (and sometimes encouraged) for users to download and install the latest functional enhancements, patches and bug fixes – this speed of enhancement and the ability to leverage the efforts of other developers is undoubtedly one of the attractions of using open source software, especially in innovative applications. However, if this is allowed to happen the validated state of the software is immediately placed at risk and appropriate controls must be established. It is possible to allow this flexibility, but there are additional costs involved.

This requires a controlled set of environments to be established and these are outlined in Figure 21.6.

Although it is possible for very frequent updates to be made to the development environment (including the download of new functions, bug fixes and patches) – sometimes weekly or daily – the release of such software to the test environment should have a clear scope, defined in terms of which updated requirements will be fulfilled. Release from the development environment to the test environment should be subject to defined version management and release management processes and will occur with a much lower frequency (every two weeks, every month or every three months depending on how much the open source software is changing).

The test environment should be formally qualified and subject to change control. This is to ensure that the test environment is functionally equivalent to the production environment and that testing in the test environment is representative of the production environment. Release of tested software should also be subject to defined version, release and change management processes and will again be on a less frequent basis. Although it is possible to ‘fast track’ new functions, the need to regression test existing, unchanged functionality places limits on exactly how fast this can be achieved. The production instance should, of course, also be qualified.

21.8 Conclusions

The validation process and controls described above apply only to validated open source software – those supporting regulatory significant processes or managing regulatory significant data or records. Non-validated open source software does not always need such control, although the prudent mitigation of business risks suggests that appropriate processes and controls should nevertheless be established.

The use of open source software does provide a number of advantages to life sciences companies and these include speed to implement new solutions, the ability to deploy ground-breaking software and the opportunity to participate in and benefit from a broad collaborative community interested in solving similar problems. However, the need to assure regulatory compliance and the requirement to validate and control such software does bring additional costs and add time to the process. Although this should not completely negate the advantages of open source software, the cost- and time-saving benefits may not be as great as in other non-regulated industries. Life sciences companies should therefore ensure that initiatives to leverage open source software are properly defined and managed, and that the relevant IT and QA groups are involved at an early stage. This will allow relevant package and supplier assessments to be conducted and the true costs and return on investment to be estimated.

Although much of the validation process is similar to the process used to validate commercial software, the nature of open source software means that there are some important differences, and the use of experienced subject matter experts to support the validation planning is highly recommended. Effective controls can be established to maintain the validated state and this will again require a common understanding of processes and close cooperation between the process owners (users), system owners (IT) and QA stakeholders.

In conclusion, there is no reason why free/open source software cannot be validated and used in support of regulatory significant business operations – it just may not be as simple, quick and inexpensive as it first appears.

21.9 References