In order to learn from the past in complex, dynamic systems their factual functioning must be transparent. Knowledge deficiencies and systemic deficiencies should be derived from analysing decision-making and underlying assumptions, uncertainty and knowledge. Such deficiencies are to be identified throughout the design process as well as during operational practices, in order to facilitate systems change and enhancement of the safety performance. Safety investigations should facilitate evidence-based learning. Without a fact-finding mission dealing with on-site observations and interpretations, evidence-based learning will be lacking and achieving consensus will be reduced to a mere negotiation result. Providing evidence is the domain of forensic sciences. By providing a timely transparency, safety investigations may take the role of providing functional requirements for the (re-)engineering design of resilience as a system property. This approach is demonstrated by two major case studies that have been conducted in the Netherlands: the El-Al air crash in Amsterdam and the inquiry into the High Speed Line ERTMS signalling system.
According to the methodological principles for engineering design, systems are assessed with regard to their intended functioning throughout the design process (Stoop, 1990). Historically, technological aspects of safety have been identified at the three consecutive phases of conceptualisation, function allocation and materialisation focusing on, respectively:
• Inherent design principles, to be deployed in selecting specific concepts, configurations and control arrangements dealing with fail-safe, crash worthiness and delegated or distributed responsibilities in human centred control.
• Emergent system properties, described by the acceptability of accident scenarios in their operational context in fulfilling their designed functions. Such properties are identified as redundancy, robustness, reliability, reconfiguration, rescue, recovery and resilience.
• Performance indicators, by quantification of the probability of operational failures and their consequences as either expressed in safety standards, accident frequency rates or safety integrity levels.
Determination of systemic and knowledge deficiencies originating from these phases do not, however, restrict themselves to the technical investigations of single events in the operational phase, commissioned to accident investigation committees in case of a major occurrence. As the HSL/ERTMS and Schiphol/B747 case studies indicate, investigations also can be applied to the level of complex operational management in a multi-actor environment and the design and development phase of a major project (Stoop et al., 2007; Stoop, 2009). The initiating organisation in both investigations was the Dutch Parliament, requesting transparency in the event and underlying decision-making processes.
Accident investigation cannot evade the challenge of dealing with the notion of ‘cause’. Accident investigations have to provide evidence, based on scientific methods, principles and knowledge (Stoop, 1997; Sklet, 2002). By their mandate, accident investigations must render advisory opinions to assist the resolution of disputes affecting life or property. They play a role as public safety assessor in a multi-actor environment in open decision-making processes. During their fact-finding phases, such investigations primarily rely on forensic sciences.
In general, forensic sciences comprise the science, methodology, professional practices and principles involved in diagnosing common types of accidents and failures. Determination of causes of technical failure and gaining oversight over social system performance requires:
• familiarity with a broad range of disciplines
• The ability to pursue several lines of investigation simultaneously.
As such, these are essential skills of accident investigators, engineers, researchers and managers who are dealing with safety investigations.
Safety investigations can be seen in analogy to forensic fact-finding missions as providing learning potential for understanding complex and dynamic systems. Investigations provide a transparency of design and operational decisions that cannot be replaced by management oversight, audits or governmental inspections during the operational phase of a system. Investigations may identify the nature of design and operational uncertainties.
Case 1: Before, During and After the Event; the Boeing 747 Case Study
On October 4 1992, a B-747 freighter of El-Al crashed into an apartment block in Amsterdam, killing four people on board and 43 people on the ground. During its outbound flight, two engines separated from the airplane, rendering the aircraft uncontrollable after which it crashed into the Bijlmermeer, a suburb of Amsterdam. During the investigation, the direct cause of the accident was established as a design flaw in the engine pylon, due to which the mounting of the fuse pins failed. However, the consequences of the crash far exceeded the direct causes and the technical lessons learned on the short term.
In the survey three main phases are discriminated.
• Elaborating the design decisions and the role of incremental change in extrapolating the Boeing 707 engine mounting concept into the Boeing 747 design.
• The actual investigation with respect to its complexity and case specific implication on a national policy making level over a period of about seven years, including a parliamentary inquiry.
• The implications of the crash for the international aviation community on the long term.
The benchmark nature of this disaster is in the wide implications it has had on the safety assessment of full freighters, the establishment of a multi-modal independent accident investigation agency in the Netherlands, public risk perception with respect to external safety and the shift towards a focus on dealing with the aftermath of a disaster and the role of public governance in crisis policy decision-making.
The Reason for Building such Aircraft
The Boeing 707 was developed in the early 1950s and fitted with four turbojets. In the 1960s it was the common passenger aircraft. It established Boeing as one of the largest manufacturers of passenger aircraft and led to later series of aircraft with 7x7 designations. The huge growth in air travel made the 707 a victim of its own success. The 707 was too small to handle the increased passenger densities on the routes for which it was designed. The solution was to design an aircraft with 2.5 times the size of the Boeing 707. The production of the 747 was under enormous pressure to succeed. This was because there was an Anti-Jumbo Lobby, which questioned the safety of the big aircraft (Hoogendorp, 2007).
The design of the engine nacelle and pylon incorporated provisions that prevent a wing fuel cell rupture in case of engine separation. This prevention was achieved with structural fuses.
During the design of the strut-to-wing attachments of the 747, Boeing employed a fuse pin concept that was similar to the 707. This was designed to be fail-safe for vertical loads. The concept of ‘safe separation’ was addressed by mounting a fuse pin in each of the attachments, preventing this catastrophic damage.
Several related accidents with the 707 occurred. Before the Bijlmermeer aviation disaster with the 747, there were four related accidents with the 707. The separation of the engines in the incidents was attributed completely to external forces acting on the engine. Consequently, the industry had no experience with unsafe in-flight separation of the engine due to mechanical defects. In April 1988, Boeing received a report of a crack in a new style fuse pin. In total, 14 instances of cracks in new style mid-spar fuse pins and 9 reports of cracks in new style diagonal brace-fuse pins were reported. With the 747 several related accidents also occurred. In total there have been five accidents with separation of an engine assembly on the 747. As a consequence, the design and certification of the Boeing 747 strut was determined to be inadequate to provide the required level of safety. The new design philosophy:
• designs for strength, durability and maintainability
• employs the damage tolerance design philosophy
• addresses the possibility of damage due to heat, corrosion and accidents
• addresses ultimate load conditions that are not all inclusive, such as multiple blade-out conditions, multiple ultimate load conditions and unusual turbulence.
It can be concluded that Boeing had knowledge about the weakness of the construction before the Bijlmermeer disaster. After some accidents, with and without fatalities, Boeing developed a new design for the engine mounting.
After the crash of the 747 in the Bijlmermeer and the other related accidents with the 747, Boeing designed a new system for the engine attachment. This has been applied to all new types. At this moment, the Boeing 747-400 is the only model still in production.
Immediately after the crash, several activities were undertaken by the Dutch government.
The NASB Investigation into the Crash
In its conclusions the Netherlands Aviation Safety Board (NASB) established the Probable Causes referring to the design deficiencies of the B747 (NASB, 1994):
The design and certification of the B 747 pylon was found to be inadequate to provide the required level of safety. Furthermore the system to ensure structural integrity by inspection failed. This ultimately caused – probably initiated by fatigue in the inboard midspar fuse-pin – the no. 3 pylon and engine to separate from the wing in such a way that the no. 4 pylon and engine were torn off, part of the leading edge of the wing was damaged and the use of several systems was lost or limited. This subsequently left the flight crew with very limited control of the airplane. Because of the marginal controllability a safe landing became highly improbable, if not virtually impossible.
In its 14 recommendations, the NASB referred to the necessity to redesign the pylon structure, including full-scale fatigue and fail-safe testing, a need for incorporating a fail-safe analysis in the certification procedures, and a need for improved airworthiness measures and associated inspection systems. The recommendations also addressed a joint emergency handling between flight crew and ATC in identifying the severity of the event. An improvement in recovery potential was recommended taking into account not only the safety of airplane and passengers, but also the risk to third parties especially where residential areas should be considered.
The RAND Report
Within weeks after the crash, the Dutch Ministry of Transport, Public Works and Water Management commissioned a safety evaluation of Schiphol Airport about the risk to third parties on the ground in case of an accident (RAND, 1993) The report was issued shortly afterwards, focusing on the modelling and calculation of third party risk as a part of the national safety debate on low-probability/high-consequence events. The evaluation used an analytical approach to the subject due to the lack of empirical data on third party risk, the vast amount of uncertainties involved in crash rate data, the consequence assumptions and the inability to predict the timeliness and effectiveness of enhancement measures. The report questioned the imposition of single standards for aviation risk exposure which were advocated in analogy with the Dutch regulations on maximal acceptable levels of exposure to toxic substances and hazardous activities.
The evaluation put the development of Schiphol in an international aviation perspective.
• Safety considerations may change as Schiphol evolves into a mainport. The projected growth (2.7 times passengers and 4.5 times freight tonnage over the 1993 situation) would increase third-party risk if a linear extrapolation model was applied. However, mitigating factors such as modern airplanes, technological improvement in ATC and aircraft avionics, additional runway capacity and improved international control over risky airlines may have a positive influence on the actual risk performance levels.
• Safety is an airport-wide problem. Coordination of safety was dealt with informally across various operating organisations within the airport and national government. Since no ‘magic bullets’ exist, an integrated safety management system was required in order to coordinate, monitor and assess safety procedures of the various actors and stakeholders.
Public Concern about Airport Safety
Public dissatisfaction with the findings and governmental responses eventually lead to a Parliamentary Inquiry in February 1999, shortly before parliamentary elections (Parliamentary Inquiry, 1999). This inquiry was unique in two ways. It was the first time such an inquiry had to deal with a wide variety of stakeholders and actors on a national as well as an international level. At the international level, the relation with Israel was put under pressure due to the preferential treatment of El-Al at Schiphol airport. The inquiry also directly involved the public, and the commission had to deal with the emotions of victims, relatives, rescue and emergency workers and residents of the Bijlmermeer. The population of this suburb represented a wide variety of nationalities, religions and races, each with their own culture, social status and backgrounds.
While the inquiry was of little political significance, on another level it put an end to allegations about poor governance on the cargo manifest and all existing conspiracy theories were rejected by a lack of proof. The public anxiety on the crash was released (Houtman, 2008).
Integral Safety, a Substantive Assessment
In the RAND report, the crash and the expansion of Schiphol airport were considered two critical aspects in aviation risk communication and public decision-making on the expansion of Schiphol airport into a mainport in the international aviation industry (RAND, 1993).
On one hand, regional economic development and urban planning deal with concepts such as compact cities and multifunctional use of space. On the other, transportation corridors in Trans European networks, High Speed Lines and Mainport developments and interoperability demands are addressed from a European Commission perspective. In compliance with the policy of Quantitative Risk Analysis that dominated at the time, and in order to cope with the Post-Seveso Directive of the European Commission, the Dutch Ministry of Internal Affairs has taken up its responsibilities by developing the concept of ‘Critical Size Events’, defining the nature, size and duration of major incidents in order to accommodate the required resources for rescue and emergency handling (Stoop, 2003). The absence of a mandatory application of such a resource assessment has, however, led to questions on rescue and emergency performance levels and efficiency every time a major event occurs as the recent crash of Turkish Airlines at Schiphol in February 2009 has demonstrated.
In the RAND report, recommendations were made to enforce high safety performance standards at the airport level to maintain the public confidence while facilitating the growth in air travel. To protect the interest of European citizens living in the vicinity of airports or travelling on board third country aircraft, the need to enforce safety standards became apparent (Roelen, 2008). At the international level EASA took the initiative to inspect third country aircraft on the spot. This inspection however, cannot substitute the responsibilities of a country to perform proper regulatory oversight. The Safety of Foreign Aircraft Programme was initiated to perform ramp inspections. Non-complying aircraft and airlines could be banned from entering the EU airspace by putting them on a Black List. Such blacklisting has occurred frequently and updates of the Black List are published on a regular basis.
At the national level, the RAND report recommended to install an Integral Safety Management System (ISMS) at the level of the Schiphol airport community. After its foundation in 1993 this ISMS operated in full transparency and openness with an independent status. The Minister of Transport, however, decided to abolish the safety advisory commission in 2006, replacing this expert commission by the Alders Roundtable (PRC, 2006; Ministerie van Verkeer en Waterstaat, 2008). In this roundtable, all stakeholders – residents, local and regional governance, ministries and the aviation sector – came together to achieve a common advice on the development of Schiphol and the region until 2020. The roundtable addressed selective growth combined with noise abatement, while safety was not included in its mandate.
Lessons Learned
Some lessons can be learned from this survey with respect to using accident investigation as an input for Resilience Engineering. This survey on NSAB and RAND investigations revealed some strengths:
• The substantive independence of the investigation. While the NSAB already had its independence assured, the Schiphol Group had to finance the RAND investigations, but had no say in the report, which was sent to the Ministry of Transport directly.
• The prominent role for single event investigations in learning processes to provide factual information on the functioning of the aviation system in practice. Retrospective learning loops may provide valuable factual information and transparency at the level of the airport community (Stoop and Dekker, 2007). Independent investigations identify learning at the institutional level in order to sustainably incorporate safety at a national as well as international safety policy decision-making level.
• Safety at Schiphol has been broadened from aircraft safety towards an integral safety notion incorporating aircraft safety, airport safety management, external safety and rescue and emergency services.
• The focus of the NSAB successor, the Dutch Transportation Safety Board, has enlarged its scope from investigating before and during, towards after the event, and multi-actor involvement at all systems levels, including governance and control at the higher systems levels.
• Institutional independence of a public safety assessor at the airport level is indispensable in order to facilitate and maintain a sustainable, proactive and integral assessment of operations. The functioning of such an assessor should be assured by legal and organisational arrangements.
However, the survey also revealed some weaknesses:
• It is difficult to maintain transparency at the operational level of an airport community. Safety is balanced against other dominant aspects, such as noise abatement and limitations to growth, while safety awareness fades away some time after an accident.
• Safety policy making is fragmented and revolves across policy domains in time, from crash investigation (Ministry of Transport), via rescue and emergency (Ministry of Internal Affairs) to external safety (Ministry of Housing, Spatial Planning and Environment).
• In such a process, a substantive safety assessment approach is replaced by a consensus achieving decision-making approach. Throughout such revolving, safety gradually reduces from a strategic decision-making criterion to a stakeholder based operational constraint.
• Despite the analytical robustness of both investigations major uncertainties remain for a proactive approach to safety at the airport community level, due to the very low probability and limited availability of event data of a historical nature. Rescue, recovery and resilience abilities after such an unpreventable event remain indispensable.
Case 2: ERTMS. An Inquiry into the Safety Architecture of High Speed Train Safety
The European Rail Traffic Management System (ERTMS) is a part of the renovation and upgrading of national railway systems, facilitating interoperability on the EU rail network and fully software controlled train surveillance. ERTMS is a trend shift from technical compatibility across nations towards standardisation and harmonisation on the main EU network corridors. The Dutch High Speed Line is part of Paris-Köln-Amsterdam-London corridor.
For the Dutch ERTMS development several political choices have been made.
• Innovation in Public-Private Partnerships in contracting, mixing public and private interests.
• The development and implementation of technology is done concurrently instead of sequentially. A simultaneous technical development of standards and software components assumes a pragmatic off-the-shelf availability of components from various industrial consortia.
• Based on cost-effectiveness considerations, no redundancy was incorporated to prevent failure during the transition towards these new technologies, although they occurred with respect to the signaling level migration, software version upgrading and full scale testing of the system.
With the increase in traffic intensity, the system is loaded to its design limits. The fault tolerance in hierarchical systems decreases quadratically with intensity. About its saturation point, the traffic flow becomes unstable. Due to the dynamic feedback of multiple operators – train drivers as well as traffic controllers – operator induced oscillation becomes possible; their fault handling may cause abrupt and progressive collapse of the overall system. To avoid initiating disturbances, an even stricter task performance of the train driver is required. Increasing the punctuality of the time table under high traffic intensity conditions demands an increasing control effort by the traffic controller and train drivers. This aggravates the tactical and operational cognitive workload of the traffic control centres that are forced to communicate simultaneously with several train drivers. Eventually a gridlock situation occurs where all traffic operations must be terminated by a fail-safe system breakdown followed by a gradual and safety critical reset. The underlying organisational mechanisms which threaten public values such as safety versus private business values, can be identified as coping behaviour in order to deal with conflicting values (Steenhuisen and Van Eeten, 2008). Will it suffice to discipline organisations with advanced contracts and incentives, piling up requirements without clarifying inevitable trade-offs?
Two principal strategies are applied to overcome design limitations.
• A process approach. Recognition of value conflicts and subsequently a structuring of the process of communication, coordination, and cooperation among all stakeholders in their decision-making processes, coping between quantifiable private performance indicators and qualitative public values.
• A technological approach. Elimination of the human involvement in disguised bad performance due to ambiguous and hybrid decision-making values by developing an innovative train control system, based on modern technology and a new generation of signalling systems.
During the High Speed Line investigations, several value judgements became visible dealing with the project organisation and technological scope. They manifested themselves as emergent properties at the end of the design and development phase, requiring additional design interventions and operational remedies (Stoop and Dekker, 2007).
• The institutional environment has complicated the development and implementation of the project. The divisions that were created during the project between design and construct of the hardware components and the contractual arrangements between stakeholders required a complex interface management. This interfacing has not been accomplished.
• The necessity to create oversight emerged only by the end of the project. There was no role for a systems integrator, responsible for the integral coherence of the overall system. The pivotal role of ERTMS became emergent at the end of the project in the full scale testing phase of the integral system.
• The technological development of ERTMS was underestimated. There has been a continuous tension between incremental progress and implementation in an existing railway network on one hand and the ambitions of innovative ERTMS and public-private partnership arrangements on the other hand.
• Consequences of several technological design decisions should have been submitted to a pro-active safety assessment procedure. Several points-of-no-return in the design process were passed without oversight of their consequences.
• A choice for a new signalling system, which was not yet operational at the time, was not compensated by a qualified fall-back option.
• The choice for an innovative ERTMS system in the Netherlands was not in harmony with the more incremental process and evolutionary development of the Belgian signalling system on the same corridor Amsterdam-Antwerp.
• The choice for connecting the Dutch and Belgian system manufactured by two different signalling system consortia at the country border forced the project management to develop a gateway causing high costs and considerable delays in delivering the integrated system for testing and operations.
• A contractually based testing and deployment of ERTMS version 2.2.2 took place while version 2.3.0 would become the new standard, causing complications, costs and delays.
• The development of ERTMS was considered a conventional technical engineering effort, enabled by a decomposition of the system components in autonomous position finding and communication subsystems. Development and manufacturing of these components was subcontracted across competitive consortia. Each consortium was assumed to be able to deliver these components ‘off-the-shelf’ as proven technology.
• No precautionary measures were taken to assure a smooth and efficient frequent upgrade of the signalling software during its operational phase.
Safety can be considered a normal consequence of performance variability which should be controlled rather than constrained (Hollnagel et al., 2008). The ability effectively to adjust a system’s functioning prior to or following changes and disturbances can sustain its functioning after disruption or mishap, while under continuous stresses. Systems should therefore be able to cope with responses to the actual, critical, potential and factual situations. A transparency in various system states should be available supported by the ability to predict, plan and produce.
But what if such transparency is impossible? If we cannot analyse the complex reality and cannot achieve consensus, are we doomed to restrict ourselves to a battlefield of subjective opinions, submitted to political will and governance resolve (Rosenthal, 1999). Or do we restrict ourselves to a lower level of a single agent at the organisational level, accepting sacrificial losses? The potential of offering opportunities in solving complex problems by taking into account the dynamics, multidisciplinarity and complexity of systems enables a transition from a static and event oriented approach into a dynamic, system oriented approach by applying chaos and complexity theory and a re-introduction of the conceptual design phase in system change (Bertuglia and Vaio, 2005). This dynamic system perspective identifies systems behaviour beyond the level of linear behaviour dealing with deterministic chaos, emergent behaviour, self-organisation, self-conformity, resonance, and bifurcations. From a safety perspective, the most interesting parameter is the existence of multiple system states (Hendriksen, 2008). This perspective eliminates the debate on acceptable and quantifiable system safety performance levels, replacing it by the need to design resilience into such systems in order to cope with change.
Towards a New Train Control Concept
The Resilience Engineering potential has been demonstrated in a feasibility study into the deployment of a new railway concept beyond the boundaries of present railway configurations. It aims at doubling the number of trains for half the costs per passenger kilometre, maintaining present safety performance levels. Regarding the train control functionality, a new Free Ride concept was developed in analogy with the Free Flight concept in aviation (Van Binsbergen et al., 2008). Instead of the full automation paradigm, a human-centred design approach is applied.
The Free Ride concept eliminates the conflict of interest between safety and control, by applying a performance based local control strategy instead of a centralised compliance based approach, restricting incident management and handling to the local level of the network (Van Binsbergen et al., 2008). Such a concept allocates the incident handling responsibilities and control options to the local level, which only can be overruled by a centralised control to avoid a network gridlock state. Four innovations of both a technological and organisational nature are required for a further development of the Free Ride concept (Van Dam et al., 2008; Van Eijndhoven, 2008).
In comparing the safety assessment of the ERTMS system development against the Free Ride concept, some conclusions can be drawn.
• Actors located at different systems levels and life cycle phases create value and control conflicts. A multi-agent process approach is emergent, but not sufficient to guarantee a continuous and explicit interest in safety issues. Safety is a system critical aspect that cannot be reduced to a field test item in a final phase of the project.
• Technological innovation creates major uncertainties. Engineering is not a standard technology application which can be bought off-the-shelf; it also contains software design concepts change, system architecture and integration, oversight/consequence analysis and integral system certification and testing.
• Shifting from a technological perspective in systems development towards a social engineering is not sufficient; there is a need to integrate the technical, human and organisational/institutional design across the various system life phases, taking into account the various system states that may exist in practice.
Engineering design aims at expanding the design scope from form to function and from performance to properties, from aircraft design towards systems design, and from component towards context.
Taking this integrated engineering design approach facilitates identification from emergent properties in practice to inherent properties in the design phase. Integrated design is dealing with dynamic instabilities in the overall Programme of Requirements.
As such, resilience is a property of saturated and mature systems, emerging during the operational phase. It is a major challenge to deal with this system property already in the design and development phase.
What Do We Need to Design Resilient Systems?
In order to be able to design resilience into systems, several requirements have to be met.
• Unravel complexity in the system by investigating system levels, life cycle phases and safety critical decisions, components and interdependencies in order to provide transparency in the systems functioning as designed and as operated and identify how hazards may propagate through this system. Triggering events may provide opportunities for change by creating a sense of urgency, but do not necessarily focus on critical deficiencies.
• Identify system and knowledge deficiencies in order to understand and control the propagation of hazards through a system. Such investigations should be unbiased and impartial. Instead, an understanding of goals and motives should facilitate a perspective on improved systems governance and control and should facilitate change strategies.
• Apply an integral systems approach from a multi-actor and multi-aspect perspective in order to achieve consensus across stakeholders in a common understanding and ability to change the overall system’s performance. Such a perspective reinforces the resilience of a system against unanticipated changes in internal and external operating conditions and constraints.
• Acknowledge the specific role of technology as a mature and often saturated system characteristic that is submitted to a need for innovation at a conceptual level in order to make substantial progress in performance. The need for a system oversight requires an explicit role for an integrator or architect for the system. In particular in aviation, technology has been selected as the flywheel for progress (Freer, 1994).
• Such a need for innovation and systems integrator role cannot be restricted to the technological components: organisational and managerial changes and institutional arrangements have to be taken into account as well. Expanding the design solution space refers to the overall systems design level and integration of new functions and actors.
What has Created Opportunities for Resilience in these two Cases?
In the Schiphol case as well as in the ERTMS case resilience has been designed into the system by:
• Expanding the design solution space beyond existing boundaries. The role of innovation in a technological, organisational and institutional sense has been crucial for achieving new integrated solutions. A new technological concept for wide body aircraft, airport infrastructure, train signalling and control had to provide the necessary technical capacity for the intended growth and expansion.
• Recognition of the need for systems oversight and system integration. Optimising components and single functions does not fulfil the need for an optimum overall performance of such complex systems. Public values such as safety may easily be jeopardised and sacrificed against corporate and private values such as economy.
• The familiarity with a broad range of disciplines and the ability to pursue several lines of investigation simultaneously allows investigators and designers to apply multiple perspectives and to ensure democratic participation in achieving consensus on goals, strategies and assessment of the final results.
In retrospect, the investigations into the air crash and the railway signalling system both provided transparency in the systems, revealing deficiencies which were unnoticed until the investigations were conducted. As such, the investigations offered additional options which came available for enhancing the systems safety. Without these facts, there would have been no glory.