Basic Patterns in How Adaptive Systems Fail
This chapter provides one input to resilience management strategies in the form of three basic patterns in how adaptive systems fail. The three basic patterns are (1) decompensation – when the system exhausts its capacity to adapt as disturbances/challenges cascade; (2) working at cross-purposes – when roles exhibit behaviour that is locally adaptive but globally maladaptive; and (3) getting stuck in outdated behaviours – when the system over-relies on past successes. Illustrations are drawn from urban fire-fighting and crisis management. A working organisation needs to be able to see and avoid or recognise and escape when the system is falling into one of the three basic adaptive traps. Understanding how adaptive systems can fail requires the ability to contrast diverse perspectives.
The Optimist-Pessimist Divide on Complex Adaptive Systems
Adaptive System Sciences begin with fundamental tradeoffs – optimality-brittleness, (Csete and Doyle, 2002; Zhou et al., 2005) or efficiency-thoroughness (Hollnagel, 2009). As an entity, group, system or organisation attempts to improve its performance it becomes better adapted to some things, factors, events, disturbances or variations in its environment (its ‘fitness’ improves). However, as a consequence of improving its fitness with respect to some aspects of its environment, that entity also becomes less adapted to other events, disturbances or variations. As a result, when those ‘other’ events or variations occur, the entity in question will be severely tested and may fail (this dynamic is illustrated by the story of the Columbia space shuttle accident, e.g., Woods, 2005.
The driving question becomes whether (and how) an entity can identify and manage its position in the trade-off space? In other words, can an organisation monitor its position and trajectory in a trade-off space and make investments to move its trajectory prior to crisis events? The pessimists on complexity and adaptive systems (e.g., Perrow, 1984) see adaptive systems as trapped in a cycle of expansion, saturation and eventual collapse. The pessimist stance answers the above questions with ‘No.’ Their response means that as a system adapts to meet pressures to be ‘faster, better, cheaper’, it will become more complex and experience the costs associated with increasing complexity with little recourse.
Resilience Engineering, on the other hand, represents the optimist stance and its agenda is to develop ways to control or manage a system’s adaptive capacities based on empirical evidence. Resilience Engineering maintains that a system can manage brittleness trade-offs. To achieve such resilient control and management, a system must have the ability to reflect on how well it is adapted, what it is adapted to and what is changing in its environment. Armed with information about how the system is resilient and brittle and what trends are under way, managers can make decisions about how to invest resources in targeted ways to increase resilience (Woods, 2006a; Hollnagel, 2009).
The optimist stance assumes that an adaptive system has some ability to self-monitor its adaptive capacity (reflective adaptation) and anticipate/learn so that it can modulate its adaptive capacity to handle future situations, events, opportunities and disruptions. In other words, the optimist stance looks at human systems as able to examine, reflect, anticipate, and learn about its own adaptive capacity.
The pessimist stance, on the other hand, sees an adaptive system as an automatic built-in process that has very limited ability for learning and self-management. Systems may vary in how they adapt and how this produces emergent patterns but the ability to control these cycles is very limited. It is ironic that the pessimist stance thinks people can study and learn about human adaptive systems, but that little can be done to change/design adaptive systems because new complexities and unintended consequences will sabotage the best laid plans. Resilience Engineering admits that changing/designing adaptive systems is hard, but sees it as both necessary and possible. Resilience Engineering in practice provides guidance on how to begin doing this.
This chapter provides one input to resilience management strategies in the form of three basic patterns in how adaptive systems fail. The taxonomy continues the line of work begun by Woods and Cook (2006) who described one basic pattern in how adaptive systems behave and how they fail. The chapter also illustrates these patterns in examples drawn from urban fire-fighting and crisis management. To develop resilience management strategies, organisations need to be able to look ahead and either see and avoid or recognise and escape when they are headed for adaptive traps of one kind or another. A taxonomy of different maladaptive patterns is valuable input to develop these strategies.
Assessing Future Resilience from Studying the History of Adaptation (and Maladaptation)
The resilience/brittleness of a system captures how well it can adapt to handle events that challenge the boundary conditions for its operation. Such ‘challenge’ events occur (1) because plans and procedures have fundamental limits, (2) because the environment changes over time and in surprising ways and (3) because the system itself adapts around successes given changing pressures and expectations for performance. In large part, the capacity to respond to challenging events resides in the expertise, strategies, tools, and plans that people in various roles can deploy to prepare for and respond to specific classes of challenge.
Resilience, as a form of adaptive capacity, is a system’s potential for adaptive action in the future when information varies, conditions change, or when new kinds of events occur, any of which challenge the viability of previous adaptations, models, plans, or assumptions. However, the data to measure resilience comes from observing/analysing how the system has adapted to disrupting events and changes in the past (Woods, 2009a: 500). Past incidents provide information about how a system was both brittle, by revealing how it was unable to adapt in a particular evolving situation, and resilient, by revealing aspects of how it routinely adapted to disruptions (Woods and Cook, 2006). Analysis of data about how the system adapted and to what, can provide a characterisation of how well operational systems are prepared in advance to handle different kinds of challenge events and surprises (Hollnagel et al., 2006).
Patterns of failure arise due to basic regularities about adaptation in complex systems. The patterns are generalisations derived from analysing cases where systems were unable to prepare for and handle new challenges. The patterns all involve dynamic interactions between the system in question and the events that occur in its environment. The patterns also involve interactions among people in different roles each trying to prepare for and handle the events that occur within the scope of their roles. The patterns apply to systems across different scales – individuals, groups, organisations.
There are three basic patterns by which adaptive systems break down, and within each, there is a variety of sub-patterns. The three basic patterns are:
• decompensation
• working at cross-purposes
• getting stuck in outdated behaviours.
Decompensation: Exhausting Capacity to Adapt as Disturbances/Challenges Cascade
In this pattern, breakdown occurs when challenges grow and cascade faster than responses can be decided upon and effectively deployed. A variety of cases from supervisory control of dynamic processes provide the archetype for the basic pattern. Decompensation occurs in human cardiovascular physiology, for example, the Starling curve in cardiology. When physicians manage sick hearts they can miss signals that the cardiovascular system is running out of control capability and fail to intervene early enough to avoid a physiological crisis (Feltovich et al., 1989; Cook et al., 1991; Woods and Cook, 2006). Decompensation also occurs in human supervisory control of automated systems, for instance in aviation. In cases of asymmetric lift due to icing or slowly building engine trouble, automation can silently compensate but only up to a point. Flight crews may recognise and intervene only when the automation is nearly out of capacity to respond and when the disturbances have grown much more severe. At this late stage there is also a risk of a bumpy transfer of control that exacerbates the control problem. Noticing early that the automation has to work harder and harder to maintain control is essential (Norman, 1990; Woods and Sarter, 2000 provide examples from cockpit automation). Figure 10.1 illustrates the generic signature for decompensation breakdowns.
The basic decompensation pattern evolves across two phases (Figure 10.1). In the first phase, a part of the system adapts to compensate for a growing disturbance. Partially successful initially, this compensatory control masks the presence and development of the underlying disturbance. The second phase of a decompensation event occurs because the automated response cannot compensate for the disturbance completely or indefinitely. After the response mechanism’s capacity is exhausted, the controlled parameter suddenly collapses (the decompensation event that leads to the name).
Figure 10.1 The basic decompensation signature
The question is whether a part of the system – a supervisory controller – can detect the developing problem during the first phase of the event pattern or whether it misses the signs that the lower order or base controllers (automated loops in the typical system analysis) are working harder and harder to compensate but getting nearer to its capacity limits as the external challenge persists or grows? This requires discriminating between adaptive behaviour that is part of successful control and adaptive behaviour that is a sign of incipient failure to come.
In these situations, the critical information is not the abnormal process symptoms per se but the increasing force with which they must be resisted relative to the capabilities of the base control systems. For example, when a human acts as the base control system, they would as an effective team member communicate to others the fact that they need to exert unusual control effort (Norman, 1990). Such information provides a diagnostic cue for the team and is a signal that additional resources need to be injected to keep the process under control. If there is no information about how hard the base control system is working to maintain control in the face of disturbances, it is quite difficult to recognise the gravity of the situation during the phase 1 portion, and therefore to respond early enough to avoid the decompensation collapse that marks phase 2 of the event pattern. The key information is how hard control systems are working to maintain control and the trend: are control systems running out of control capability as disturbances are growing or cascading?
There are a number of variations on the decompensation pattern, notably:
• Falling behind the tempo of operations (e.g., the aviation expression ‘falling behind the power curve;’ surges in demands in emergency rooms – Wears and Woods, 2007; bed crunches in intensive care units – Cook, 2006).
• Inability of an organisation to transition to new modes of functioning when anomalies challenge normal mechanisms or contingencies (e.g., a hospital’s ability to manage mass casualty events – see Committee on the Future of Emergency Care in the US, 2006; Woods and Wreathall, 2008 provide a general description of this risk).
Working at Cross-purposes: Behaviour that is locally Adaptive, but Globally Maladaptive
This refers to the inability to coordinate different groups at different echelons as goals conflict. As a result of miscoordination the groups work at cross-purposes. Each group works hard to achieve the local goals defined for their scope of responsibility, but these activities make it more difficult for other groups to meet the responsibilities of their roles or undermine the global or long-term goals that all groups recognise to some degree.
The archetype is the tragedy of the commons (Ostrom, 1990, 1999) which concerns shared physical resources (among the most studied examples of common pools are fisheries management and water resources for irrigation). The tragedy of the commons is a name for a baseline adaptive dynamic whereby the actors, by acting rationally in the short term to generate a return in a competitive environment, deplete or destroy the common resource on which they depend in the long run. In the usual description of the dynamic, participants are trapped in an adaptive cycle that inexorably overuses the common resource (a ‘pessimist’ stance on adaptive systems); thus, from a larger systems view the local actions of groups are counter-productive and lead them to destroy their livelihood or way of life in the long run.
Organisational analyses of accidents like the Columbia space shuttle accident see production/safety trade-offs as similar to the tragedies of the commons. Despite the organisations’ attempts to design operations for high safety and the large costs of failures in money and lives, line managers under production pressures make decisions that gradually erode safety margins and thereby undermine the larger common goal of safety. In other words, safety can be thought of as an abstract common pool resource analogous to a fishery. Thus, dilemmas that arise in managing physical common pool resources are a specific example of a general type of goal conflict where different groups are differentially responsible for, and affected by, different sub-goals, even though there is one or only a couple of commonly held over-arching goals (Woods et al., 1994; Woods et al., 2010: Chapter 4). When the activities of different groups seem to advance local goals but undermine over-arching or long-term goals of the larger system that the groups belong to, the system-level pattern is maladaptive as the groups work at cross-purposes. Specific stories that capture this pattern of adaptive breakdown can be found in Brown (2005), who collected cases of safety dilemmas and sacrifice judgments in health-care situations.
There is a variety of sub-patterns to working at cross purposes. Some of these concerns vertical interactions, that is, across echelons or levels of control, such as the tragedy of the commons. Others concern horizontal interactions when many different groups need to coordinate their activities in time and space such as in disaster response and military operations. This pattern can also occur over time. A sub-pattern that includes a temporal component and is particularly important in highly coupled systems is missing the side effects of change (Woods and Hollnagel, 2006). This can occur when there is a change that disrupts plans in progress or when a new event presents new demands to be handled, among other events. Other characteristic sub-patterns are:
• Fragmentation over roles (stuck in silos; e.g., precursors to Columbia space shuttle accident, Woods, 2005).
• Failure to resynchronise following disruptions (Branlat et al., 2009).
• Double binds (Woods et al., 2010).
Getting Stuck in Outdated Behaviours: Over-relying on Past Successes
This pattern relates to breakdowns in how systems learn. What was previously adaptive can become rigid at the level of individuals, groups or organisations. These behaviours can persist even as information builds that the world is changing and that the usual behaviours and processes are not working to produce desired effects or achieve goals. One example is the description of the cycle of error as organisations become trapped in narrow interpretations of what led to an accident (Cook et al., 1998).
This pattern is also at play at more limited operational time scopes. Domains such as military operations offer a rich environment for studying the pattern. When conditions of operation change over time, tactics or strategies need to be updated in order to match new challenges or opportunities. While such decisions are made difficult by the uncertain nature of the operations’ environment and of the outcome of actions, missed opportunities to re-plan constitute sources of failure (Woods and Shattuck, 2000). Mishaps in the nuclear industry have also exemplified the pattern by showing the dangers of ‘rote rule following’ (Woods and Shattuck, 2000). In all of these cases there was a failure to re-plan when the conditions experienced fell outside of the boundaries the system and plans were designed for. Some characteristic sub-patterns are:
• oversimplifications (Feltovich et al., 1997);
• failing to revise current assessment as new evidence comes in (Woods and Hollnagel, 2006; Rudolph, 2009);
• failing to revise plan in progress when disruptions/opportunities arise (Woods and Hollnagel, 2006);
• discount discrepant evidence (e.g., precursors to Columbia, Woods, 2005a);
• literal mindedness, particularly in automation failures (Woods and Hollnagel, 2006);
• distancing through differencing (Cook and Woods, 2006);
• Cook’s Cycle of Error (Cook et al., 1998).
The three basic patterns define kinds of adaptive traps. A reflective adaptive system should be able to monitor its activities and functions relative to its changing environment and determine whether it is likely to fall into one or another of these adaptive traps. The three basic patterns can be used to understand better how various systems are vulnerable to failures, such as systems that carry out crisis management, systems that respond to anomalies in space flights and systems that provide critical care to patients in medicine. In the next section, we test the explanatory value of these three basic patterns by re-visiting a recent analysis of critical incidents (Branlat et al., 2009) that provided markers of both resilience and brittleness (Woods and Cook, 2006). Urban fire-fighting provides a rich setting to examine aspects of resilience and brittleness related to adaptation and coordination processes. Incident command especially instantiates patterns generic to adaptive systems and observed in other domains or at other scales (Bengtsson et al., 2003; Woods and Wreathall, 2008).
Illustration of the Basic Patterns
High uncertainty and potential for disruptions, new events and surprises all pose challenges for fire-fighting operations. The fire-fighting organisation needs to be able to adapt to new information (whether a challenge or opportunity) about the situation at hand and to ever-changing conditions. For example, consider the following case from the corpus (Branlat et al., 2009).
Companies arrive on the fire scene and implement standard operating procedures for an active fire on the first floor of the building. The first ladder company initiates entry to the apartment on fire, while the second ladder gets to the second floor in order to search for potentially trapped victims (the ‘floor above the fire’ is an acknowledged hazardous position). In the meantime, engine companies stretch hose-lines but experience various difficulties delaying their actions, especially because they cannot achieve optimal positioning of their apparatus on a heavily trafficked street. While all units are operating, conditions are deteriorating in the absence of water being provisioned on the fire. The Incident Commander (IC) transmits an ‘all hands’ signal to the dispatcher, leading to the immediate assignment of additional companies. Almost simultaneously, members operating above the fire transmit an ‘URGENT’ message over the radio. Although the IC tries to establish communication and get more information about the difficulties encountered, he does not have uncommitted companies to assist the members. Within less than a minute, a back-draft-type explosion occurs in the on-fire apartment, engulfing the building’s staircase in flames and intense heat for several seconds and erupting through the roof. As the members operating on the second floor had not been able to get access to the apartment there due to various difficulties, they lacked both a refuge area (apartment) and an egress route (staircase). The second ladder company was directly exposed to life-threatening conditions.
The three basic patterns can all be seen at work in this case.
Decompensation
The situation deteriorated without companies being able to address the problem promptly. The IC recognised and signalled an ‘all hands’ situation, in order to inform dispatchers that all companies were operating and to promptly request additional resources. As there were no uncommitted resources available, the fire companies were unable to respond when an unexpected event occurred (the back-draft), which created dangers and hindered the ability of others to assist. As a result, team members were exposed to dangerous conditions.
Working at Cross-purposes
Companies were pursuing their tasks and experienced various challenges without the knowledge of other companies’ difficulties. Without this information, actions on the first floor worked against the actions and safety of operators on the second floor. Goal conflict arose (1) between the need to provide access to the fire and to contain it while water management was difficult, and (2) between the need to address a deteriorating situation and to rescue injured members while all operators were committed to their tasks.
Getting Stuck in Outdated Behaviour
The ladder companies continued to implement standard procedures that assumed another condition was met (water availability from the engine companies). They failed to adapt the normally relevant sequence of activities to fit the changing particulars of this situation: the first ladder company gained access to the apartment on fire; but in the absence of water, the opened door fuelled the fire and allowed flames and heat to spread to the rest of the building (exacerbating how the fire conditions were deteriorating). Similarly, the unit operating on the second floor executed its tasks normally, but the difficulty it encountered and the deteriorating situation required adaptation of normal routines to fit the changing risks.
Urban Fire-fighting and the Dynamics of Decompensation
During operations, it is especially important for the IC, constantly and correctly, to assess progress in terms of trends in whether the fire is in or out of control. To do this, the IC monitors (a) the operational environment including the evolution of the fire and the development of additional demands or threats (e.g., structural damages or trapped victims) and (b) the effort companies are exerting to try to accomplish their tasks as well as their capacity to respond to additional demands. Based on such assessments, the IC makes critical decisions related to the management of resources: redeploying companies in support of a particular task; requesting additional companies to address fire extensions or need to relieve members; requesting special units to add particular forms of expertise to handle unusual situations (e.g., presence of hazardous material).
ICs are particularly attentive to avoid risks of falling behind by exhausting the system’s capacity to respond to immediate demands as well as to new demands (Branlat et al., 2009). The ‘all-hands’ signal is a recognition that the situation is precarious because it is stretched close to its maximum capacity and that current operations therefore are vulnerable to any additional demands that may occur. The analysis of the IC role emphasised anticipating trends or potential trends in demands relative to how well operations were able to meet those demands (see also Cook’s analysis of resource crunches in intensive care units; Cook, 2006). For urban fire-fighting, given crucial time constraints, resources are likely to be available too late if they are requested only when the need is definitive. A critical task of the IC therefore corresponds to the regulation of adaptive capacity by providing ‘tactical reserves’ (Klaene and Sanders, 2008: 127), that is, an additional capacity promptly to adapt tactics to changing situations. Equivalent processes also play out (a) at the echelon of fire-fighters or fire teams, (b) in terms of the distributed activity (horizontal interactions) across roles at broader echelons of the emergency response system, and (c) vertically across echelons where information about difficulties at one level change decisions and responses at another echelon.
Urban Fire-fighting and Coordination over Multiple Groups and Goals
Fire-fighting exemplifies situations within which tasks and roles are highly distributed and interdependent, exposing work systems to the difficulty of maintaining synchronisation while providing flexibility to address ever-changing demands. Interdependencies also result from the fact that companies operate in a shared environment.
Several reports within the corpus described incidents where companies opened hose-lines and pushed fire and heat in the direction of others. These situations usually resulted from companies adapting their plan because of difficulties or opportunities. If the shift in activity by one group was not followed by a successful resynchronisation, it created conditions for a coordination breakdown where companies (and, importantly, the IC) temporarily lost track of each other’s position and actions. In this context one group could adapt to handle the conditions they face in ways that inadvertently created or exacerbated threats for other groups. Another example in the corpus was situations where companies’ capacity to fulfil their functions were impeded by actions of others. One group’s actions, though locally adaptive relative to their scope, introduced new constraints which reduced another company’s ‘margins of manoeuvre’ (Coutarel et al., 2003). This notion refers to the range of behaviours they are able to deploy in order to fulfil their functions and therefore to their capacity to adapt a course or plan of action in the face of new challenges. Such dynamics might directly compromise members’ safety, for example when the constrained functions were critical to egress route management. In one case, a company vented a window adjacent to a fire escape, which had the consequence of preventing the members of another company operating on the floor above from using the fire escape as a potential egress route, should it have been needed.
Goal conflicts arise when there are trade-offs between achieving the three fundamental purposes of urban fire-fighting: saving lives, protecting property and ensuring personnel’s safety. This occurs when, for example, a fire department forgoes the goal of protecting property in order to minimise risk to fire-fighters. Incidents in the corpus vividly illustrate the trade-offs that can arise during operations and require adaptations to on-going operations. Under limited resources (time, water, operators), the need to rescue a distressed fire-fighter introduces a difficult goal conflict between rescue and fire operations. If members pursue fire operations, the victim risks life-threatening exposure to the dangerous environment. Yet by abandoning fire operations, momentarily or partially, team members risk letting the situation degrade and the situation becomes more difficult and more dangerous to address. The analysis of the corpus of cases found that adaptations in such cases were driven by local concerns, for example, when members suspended their current operations to assist rescue operations nearby. The management of goal conflicts is difficult when operations are not clearly synchronised, since decisions that are only locally adapted risk further fragmenting operations.
Urban Fire-fighting and the Risk of Getting Stuck in Outdated Behaviours
As an instance of emergency response, urban fire-fighting is characterised by the need to make decisions at a high-tempo and from a position of uncertainty. As fire-fighters discover and assess the problem to be addressed during the course of operations, re-planning is a central process. It is critical that adaptations to the plan are made when elements of the situation indicate that previous knowledge (on which on-going strategy and tactics are based) is outdated. The capacity to adapt is therefore highly dependent on the capacity correctly to assess the situation at hand throughout the operations, especially at the level of the IC. Accident cases show that the capacity of the IC efficiently to supervise operations and modify the plan in progress is severely impaired when this person only has limited information about, and understanding of, the situation at hand and the level of control on the fire.
Given the level of uncertainty, this also suggests the need for response systems to be willing to devote resources to further assess ambiguous signals, a characteristic of resilient and high-reliability organisations (Woods, 2006a; Rochlin, 1999). This is nonetheless challenging in the context of limited resources and high tempo, and given the potential cost of re-planning (risk of fragmenting operations, cost of redeploying companies, coordination costs).
At a wider temporal and organisational scale, fire departments and organisations are confronted with the need to learn from situations in order to increase or maintain operations’ resilience in the face of evolving threats and demands. The reports we analysed resulted from thorough investigation processes that aimed at understanding limits in current practices and tools and represented process of learning and transformation. However, it is limiting to assume that the events that produce the worst outcomes are also the ones that will produce the most useful lessons. Instances where challenging and surprising situations are managed without leading to highly severe outcomes also reveal interesting and innovative forms of adaptations (Woods and Cook, 2006). As stated previously, many minor incidents also represent warning signals about the (in)adequacy of responses to the situations encountered. They are indicators of the system starting to stretch before it collapses in the form of a dramatic event (Woods and Wreathall, 2008). To be resilient, organisations must be willing to pursue these signals (Woods, 2009a). Unfortunately, selecting the experiences or events which will prove fruitful to investigate, and allocating the corresponding resources, is a difficult choice when it has to be made a priori (Hollnagel, 2007; Dekker, 2008: Chapter 3).
Recognising what is Maladaptive Depends on Perspective Contrasts
The chapter has presented three basic patterns in how adaptive systems fail. But it is difficult to understand how behaviours of people, groups and organisations are adapted to some factors and how those adaptations are weak or strong, well or poorly adapted. One reason for this is that what is well-adaptive, under-adaptive, or maladaptive is a matter of perspective. As a result, labelling a behaviour or process as maladapted is conditional on specifying a contrast across perspectives.
First, adaptive decision-making exhibits local (though bounded) rationality (regardless of scale). A human adaptive system uses its knowledge and the information available from its field of view/focus of attention to adapt its behaviour (given its scope of autonomy/authority) in pursuit of its goals. As a result, adaptive behaviour may be adequate when examined locally, even though the system can learn and change to become better adapted in the future (shifting temporal perspective).
Second, adaptive decision-making exists in a co-adaptive web where adaptive behaviour by other systems horizontally or vertically (at different echelons) influences (releases or constrains) the behaviour of the system of interest. Behaviour that is adaptive for one unit or system can produce constraints that lead to maladaptive behaviour in other systems or can combine to produce emergent behaviour that is maladaptive relative to criteria defined by a different perspective.
Working at cross-purposes happens when interdependent systems do things that are all locally adaptive (relative to the role/goals set up/pressured for each unit) but more globally maladaptive (relative to broader perspectives and goals). This can occur horizontally across units working at the same level as in urban fire-fighting (Branlat et al., 2009). It can occur upward, vertically, where local adaptation at the sharp end of a system is maladaptive when examined from a more regional perspective that encompasses higher level or total system goals. One example is ad hoc plan adaptation in the face of an impasse to a plan in progress; in this case the adaptation works around the impasse but fails to do so in a way that takes into account all of the relevant constraints as defined from a broader perspective on goals (Woods and Shattuck, 2000).
Working at cross-purposes can occur downward vertically too (Woods et al., 2010). Behaviour that is adaptive when considered regionally can be seen as maladaptive when examined locally as the regional actions undermine or create complexities that make it harder for the sharp end to meet the real demands of situations (for example, actions at a regional level can introduce complexities that force sharp end operations to develop workarounds and other forms of gap-filling adaptations).
This discussion points to the finding in adaptive system science that all systems face fundamental trade-offs. In particular, becoming more optimal with respect to some aspects of the environment inevitably leads that system to be less adapted to other aspects of the environment (Doyle, 2000; Zhou et al., 2005; Woods, 2006a; Hollnagel, 2009). This leads us to a non-intuitive but fundamental conclusion that all adaptive systems simultaneously are as follows (Woods, 2009b).
• Well-adapted to some aspects of its environment (e.g., the fluency law—‘well’-adapted’ cognitive work occurs with a facility that belies the difficulty of the demands resolved and the dilemmas balanced; see Woods and Hollnagel, 2006),
• Under-adapted in that the system has some degree of drive to learn and improve its fitness relative to variation in its environment. This is related in both intrinsic properties of that agent or system and to the external pressures the system faces from stakeholders.
• Maladapted or brittle in the face of events and changes that challenge its normal function.
This basic property of adaptive systems means that linear causal analyses are inadequate for modelling and predicting the behaviour of such systems. Adaptive systems’ sciences are developing the new tools needed to accurately model, explain and predict how adaptive systems will behave (e.g., Alderson and Doyle, 2010), for example, how to anticipate tipping points in complex systems (Scheffer et al., 2009).
Working organisations need to be able to see and avoid or recognise and escape when a system is moving toward one of the three basic adaptive traps. Being resilient means the organisation can monitor how it is working relative to changing demands and adapt in anticipation of crunches, just as incident command should be able to do in urban fire-fighting. Organisations can look at how they have adapted to disruptions in past situations to estimate whether their system’s ‘margins of manoeuvre’ in the future are expanding or contracting. Resilience Engineering is beginning to provide the tools to do this even as more sophisticated general models of adaptive systems are being developed.