Chapter 18

Is the Aviation Industry Ready for Resilience? Mapping Human Factors Assumptions across the Aviation Sector

Kyla Zimmermann, Jean Pariès, René Amalberti and Daniel H. Hummerdal

This research maps out differences in safety perspectives, the Traditional and Resilient, respectively, across the civil aviation industry in Europe and the Americas. We surveyed 705 aviation professionals to determine whether they agreed or disagreed with the human factors and safety assumptions currently dominant in the aviation industry’s tools and methods. The results show variations in perspectives according to the national culture and occupation of the respondent. We also discovered that non-experts in human factors (HF), ergonomics, or safety may indiscriminately and unconsciously use HF ideas from a variety of (sometimes conflicting) safety models or paradigms. The results of this study can help Resilience Engineering researchers and safety managers to better understand the point-of-view of practitioners who use their tools and models.

Paradigms in Safety and Human Factors

Resilience Engineering is the culmination of over 25 years of accumulating evidence supporting a different way of thinking about organisational behaviour and the performance of complex, high risk, socio-technical systems (Amalberti, 2001; Hollnagel, 1983; Hollnagel and Woods, 1983; Pariès, 1996, 1999; Perrow, 1984; Turner, 1978). This way of thinking is not only different; it directly conflicts with some of the fundamental assumptions which define human factors or ergonomics theories applied in industry today. Thus Resilience is the antithesis of the traditional and still prevailing, human factors and safety paradigm, referred to by Hollnagel as the ‘Traditional Safety Perspective’ (see Table 18.1).

Models and methods based on these assumptions may not meet the needs of ultra-safe, complex, modern industries such as aviation and may prevent further progress (Amalberti, 2001, 2006; Dekker, 2005; Hollnagel, 2004; Le Coze, 2005; Leveson, 2002; Marais et al., 2004; Pariès, 1999; Rasmussen, 1997; Woods, 2004). There is a lot of academic momentum building in favour of change yet 25 years on, the traditional ideas seem to remain entrenched in the perspectives and approaches of industry practitioners.

This is not a simple transition to make, particularly at a large scale (Steele and Pariès, 2007). Amalberti explains that a mature system such as commercial aviation may no longer have the flexibility for dramatic or profound change (2006).

Table 18.1  Contrasting system perspectives (adapted from Hollnagel, 2008)

Figure 18.1 is an overview of the evolution of the existing human factors and safety science paradigm as well as the aspirations of Resilience Engineering, adapted partly from the Resilience symposia and literature (Hollnagel et al., 2006). Normally this table is presented as three distinct columns (Hollnagel, 2004) but in fact the underlying scientific paradigm of the epidemiological model (the Swiss cheese model) is just an extension of the sequential model (dominoes). Resilience is more of a revolution than evolution, breaking with the past at an epistemological level.

Figure 18.1  The traditional safety and resilience engineering models as applied to aviation systems and safety

Diversity in Aviation

Commercial aviation is highly standardised and regulated at an international scale. However, as these different safety paradigms demonstrate, there is still room for interpretation and variation of how people perform, understand and manage the work. We undertook to characterise the variation in perspectives about human factors and safety across the aviation community, specifically the agreement with the Traditional Safety or Resilience Engineering perspectives.

The Safety Assumptions and Resilient Attitudes (SARA) Survey

The objective of the research was to map the differences in safety perspectives held by aviation professionals across the industry in Europe and the Americas. We were primarily interested in the variation across the aviation professions, but we looked at 20 factors, including national culture, HF background, and work experience.

The Business of (not) Measuring Resilience

We set out to ask questions to gauge who in the industry used Resilience Engineering as a theoretical framework. We were immediately confronted with practical problems: First, resilience is still more easily defined (even by the Resilience Engineering community itself) in terms of what it is not than what it is. Second, many Resilience Engineering ideas appear complex and do not lend themselves easily to survey methods. Finally, presenting Resilience ideas, like those in the right-hand column of Figure 18.1, in a survey would not be meaningful – they seem like common sense (even though they are far from the industry norm). So our solution was instead to gauge the level of agreement with the contested Traditional assumptions still so prevalent in the news media, accident reports, and safety media. These assumptions, like those in the left-hand column of Figure 18.1, help define Resilience indirectly. Thus our aim was to infer Resilience Engineering attitudes by the rejection/acceptance of the Traditional Safety perspective.

Mapping the variations in perspectives across the industry would allow safety and HF professionals interested in Resilience to better understand the practitioners – the target audience of their work. Assumptions and beliefs form the safety paradigm, which in turn lays the foundation for safety action through models, tools, etc. (Lundberg et al., 2009). There is more than one way of seeing ‘the facts’ and this is influenced by one’s paradigm, whether or not one is aware of it (Gergen, 1999; Simpson, 1996). Hence making this paradigm explicit is useful, especially as Resilience researchers are pushing for changes. Mapping Safety Assumptions and Resilient Attitudes (SARA) results across the industry would tell us whether aviation is ready to make the paradigm shift to Resilience.

Developing the SARA Survey

In the first phase of the research, we gathered examples of frequently contested assumptions from the literature, such as those in Figure 18.1 and in Dekker’s Ten Questions on Human Error (2005), which define the existing Traditional aviation safety paradigm (similar to Dekker’s ‘old view’).

We developed a series of questionnaire items based on these assumptions, and surveyed aviation professionals in different domains and countries to find out whether or not they agreed with the ‘Traditional Safety Perspective’ on a scale of 1 to 5. We also conducted an initial round of confidential interviews with aviation professionals representing different jobs and geographic regions to assist and inform the survey writing process.

The Survey Respondents and Interview Participants

Using the ‘snowball’ distribution technique, the anonymous online survey was disseminated in three languages to a convenience sample based on the researchers’ professional networks and the partners of the European aviation research project HILAS (Human Integration into the Lifecycle of Aviation Systems). Tables 18.2 and 18.3 show the distribution of the 705 survey respondents according to job and geographic region. We grouped countries into geographic regions based on culturally similar clusters identified in the GLOBE Study (House et al., 2004).

Table 18.2  Distribution of the 705 survey responses according to geographic region

Based on the results of the statistical analysis of the questionnaire, we carried out a second round of confidential interviews with participants from specific regions and occupations.

Analysis of the SARA Survey Results

The survey data were analysed using qualitative statistics (multiple correspondence analysis or MCA) to allow the significant factors to emerge on their own without the a priori influence of the chosen analysis criteria. We will present the two factors which predominantly explained the variation in questionnaire response using simple descriptive statistics (e.g., averages) since they reveal the same result as the MCA but are more familiar and simpler to interpret. In order to analyse the data quantitatively, a ‘score’ was calculated for each respondent by averaging their ratings on the five-point scale for the 33 questionnaire items. The score was used as a measure of the respondent’s agreement with the Traditional Perspective and is the basic unit of comparison for the analysis described in this chapter. The differences discussed here are all statistically significant (p< 0.05 or better).

Table 18.3  Distribution of the 705 survey responses according to job

While necessary for the analysis, the idea of a ‘score’ is inherently contradictory to our research philosophy: it is not our intention to judge people as wrong or misguided based on whether they agree or disagree with us. It is our hope that readers of this book share this philosophy and will not misuse the data. The real potential in these findings is to highlight the variations in opinions across the industry so that researchers and safety managers can better design and target safety interventions for these diverse populations.

Differences between National and Occupational Cultures

The largest variation in the survey results was according to the country of residence (Figure 18.2), followed by differences among the professions (Figure 18.3). In both figures, the centre of the bar represents the mean score while the total length of the bar represents 68.2 per cent of the scores (two standard deviations) for normally distributed data. The respondents who most strongly disagreed with the traditional perspective were those living in Northern Europe or working as HF researchers/specialists. The respondents who were most in agreement with the traditional perspective were those living in Latin America or Southern Europe (Italy, Spain, and Portugal), or aircraft mechanics (AMEs). There was no interaction of these two factors. Overall the averages of all scores appear close to neutral on the scale, however the differences shown on the graphs are statistically significant and show an interesting trend: disagreement with the assumptions is roughly correlated with latitude (Hofstede, 2001), with the usual exceptions of New Zealand, Australia, and South Africa (part of the Anglo sample).

The follow-up interviews were done with pilots, ATCos, and AMEs from countries in the Northern and Southern European regions in order to represent the largest gaps among practitioners. Although the professions exhibiting the largest differences were AMEs and HF/Ergo/Safety, we did not interview the latter because we had never actually intended to survey a significant number of our colleagues; their views are already a matter of public record. However, this proved fortuitous as a measure of the survey’s validity (many of the HF/Ergo/Safety respondents did not answer anonymously, and we know them to be opponents to the Traditional Perspective).

During the interviews in Southern Europe, we had to reexamine the logic of clustering Spanish and Italian responses after several Spanish participants claimed their perspective on aviation safety to be more similar to that of France than Italy. Closer examination of the data confirmed this to be true for the Spanish (n=19) SARA scores. When analysed alone, the responses from Italy (n = 48) were further to the Traditional end of the scale, more similar to the Latin American sample.

Figure 18.2  Average survey responses according to geographic region

Figure 18.3  Average survey responses according to job

Discussion of Differences between National/Societal Cultures

Given the difficult business of quantitatively measuring ‘safety’ and defining the limits of societal cultures (a more correct term, as culture does not necessarily correspond to national borders) it is not surprising that there is very little available material for comparing safety outcomes across nations and making conclusive links to culture; there is ‘no smoking gun’, as Hutchins et al. put it (2002: 6). What does exist is controversial and incomplete, showing only crude comparisons between continents (Civil Aviation Authority, 1998) and these data is highly contested due to confounds in the metrics. It is not informative for the purposes of our study, unless we compare only the continents of North and Latin America, Western and Eastern Europe. In that case we can see a correlation between higher SARA scores and quantity of aircraft hull losses.

There is non-aviation safety data available (e.g., road safety or occupational accidents) tending to support the stereotype that the North, West and Germanic European countries (considered more rule-based, law-abiding, orderly and stoic) have better safety outcomes than their Mediterranean or Latin European neighbours (considered more laissez-faire about rules and more passionate or emotional about life) or the new EU member states or CIS states (Zimmermann, 2009). Europe is the most culturally diverse region of the world. Comparing Scandinavia and Italy reveals a very different profile along Hofstede’s cultural dimensions. Yet politically things have changed so dramatically throughout the 20th century that cultural assessment is a moving target (e.g., the Spanish SARA results mentioned previously). It is difficult to rely even on recent, large-scale studies such as those by House et al. (2004) or Hofstede (2001), as they may already be out of date in some respects.

The Limitations of Understanding Culture

Comparisons using surveys and interviews are problematic in themselves since the semantics of terms is subjective and might have cultural or individually determined connotations. For example, we overheard a German colleague complaining that trains in Switzerland are ‘always late’. This is a salient example of the subjectivity of even concrete concepts like ‘late’ and ‘always’ – so how can we have meaningful discussions about ‘safety’ or ‘risk’, etc? Hutchins et al. explain that ‘at first glance, the effects of national culture appear pervasive and obvious, but when one seeks a theory … or when one looks for direct evidence of the effects of culture …, culture seems to vanish’ (2002: 6). This is one of the dilemmas of cultural research, and makes it difficult to explain the real meaning behind the different SARA scores. However, since our objective was mainly to map the differences for practical purposes, understanding the different interpretations of the questionnaire items is not essential (but it does provide interesting opportunities for future research).

Behaviour and Attitudes Understood Within their Context

During the follow-up interviews participants were quite moderate in their praise or criticism of other continents or regions within Europe in terms of safety or rule-following behaviour. However, there were some evocations of the North–South stereotype mentioned above. Interestingly most participants described their own culture as the ideal balance between rule following and creativity and considered those to the North as slightly too rigid, and those to the South as slightly too unpredictable (Zimmermann, 2009). There may be some truth to this; it could be a reflection of the clashes occurring when people encounter different operational cultures that do not match their expectations. The behaviours may be different but still appropriate for the local context.

While the orderly, rule-following cultures may seem logically safer, that depends on the scenarios, time-scale and measures chosen to assess ‘safety’. The Tenerife disaster is an example of a paradoxical negative side-effect of safety rules and how the pressure of a strict rule-following culture can pose a different sort of risk.

Resilience and/or Safety?

One participant from Southern Europe explained his point-of-view on the difference approaches as they relate to aircraft maintenance:

In Northern Europe they won’t do the work until they have all the equipment and tools and parts and things. That’s fine for them because they can get everything they need. We [in Southern Europe] don’t have everything we need – sometimes we don’t have anything – so we can’t check off the boxes … the way we’re supposed to, but we get the job done anyway, … use creativity … In the North they would just stop working. We can’t work that way or we could never get anything done. (Zimmermann, 2009)

This raises the question of Resilience. One might argue that the individuals who are used to coping in ‘adverse conditions’ like those described above may actually be more resilient; the challenging working conditions offer more opportunities to practice their skills and develop problem solving. The work environment may also be more loosely coupled by necessity, making it more flexible in the face of crises. However, if these resilient properties exist at the micro-level only because the macro-level system is stretching things too thin (e.g., not adequately supporting the work or not providing the needed tools and infrastructure) then the system as a whole would be unprepared to deal with problems for a multitude of other reasons.

An Italian interviewee recounted an ironic anecdote:

Of course Northern Italians feel that driving in Southern Italy is more dangerous, but the fact that no one stops at a red light means that everyone is paying attention. In the North, people assume if the light is green, they can go through without looking, so actually it is more dangerous to drive in the North. (Zimmermann, 2009)

Because of constant exposure to a dangerous environment this Southern Italian driver uses Simpson’s Cautious Cognitive Framework: everything is considered a hazard unless it is explicitly indicated otherwise (1996). The irony is evident here; that the driver himself presumably also ignores traffic signals – so the net result may not be objectively safer. We know from HRO theory that it is a challenge to maintain that Cautious framework (keeping the possibility of risk alive) in a system with few accidents or incidents.

These two examples from the interviews about national/societal culture illustrate a paradox of the relationship between Resilience and safety: An unsafe system may be more flexible, more cautious, and may inadvertently foster Resilience at the micro-level. Similarly, a stable, safe system would have difficulty maintaining it. As Rasmussen (1997) has shown, when things go well the natural tendency is to increase production levels. This could increase the inherent risk (e.g., more planes or passengers), reduce the flexibility, tighten the couplings, etc. Amalberti (2006) describes the different types of actors (e.g., pioneers, craftsmen, and equivalent actors) in systems with different levels of safety. As aviation keeps evolving towards higher levels of standardisation, automation, procedures, and stability we must recognise that this comes at the expense of Resilience (Holling, 1973).

Discussion of the Differences between Occupational Cultures

Professions often create and sustain their own specific cultures – the training, the environment, the expectations of others, as well as the nature of the work itself influences and is influenced by the traits, skills, attitudes and behaviours of its members. There are common stereotypes about professions (although these are to some extent specific to a the local culture) and at a dinner party one would probably respond differently upon meeting a primary school teacher or politician, a bank teller or a Rock ‘n Roll musician, a cashier or a university professor. Within the aviation community, like any other, there are stereotypes highlighting the differences between pilots, cabin crew, managers, inspectors, ATCos, etc.

Although Hofstede’s (2001) seminal work is most known for his characterisation of national culture, he points out that the differences between professions in his results are also very as significant. For example, his sample of senior managers across the world had more in common with each other than senior managers and secretaries from the same country. This evidence supports the concept of occupational culture, demonstrating its significant influence on values and way of thinking.

Helmreich and Merritt (1998) found that pilots do score differently than their country averages on some Hofstede’s cultural dimensions, and Hutchins et al. (2002) use pilots as an example of professional culture. Lumpé (2008) did a study on airlines in Europe and explored the idea even more deeply, explaining that the various professional cultures within an airline were so different as to merit unique management and leadership strategies.

Considering the results of our study combined with this evidence from previously published work, HF researchers and safety managers should be able to appreciate that their way of thinking about safety and HF topics differs from that of practitioners, and in particular AMEs. Additionally, Safety and HF interventions may need to be tailored to take into account not only the target national/societal culture but also the profession, since a standard ‘one size fits all’ approach will not be as effective.

Cultural Bias in Culture Research

Human Factors, like many aspects of aviation, is a cultural artefact based on the Western way of thinking. It is possible that the definition of the Resilience paradigm itself is culturally bound and may be incompatible with certain cultures. In this study Northern Europeans and HF specialists most strongly rejected the Traditional Safety Perspective. This raises the question of whether our tool is biased because our background and approach are rooted in these two areas to a large extent. We offer this deliberate misquotation of Dekker and Woods (2002) as food for thought: If Resilience gets to pick the battlefield, Resilience will win.

Ambiguity and Contradictions

Three other results are worth mentioning here and analysed together they may offer more insight to HF experts and researchers on how to improve resilience in the aviation industry.

The first result was that the means of all the groups within the sample were near the centre of the scale, expressing a neutral opinion towards the assumptions in the survey. Second, there was a large variation within the results of each individual respondent (but our analysis showed the results were not random).

Third, during the interviews we had hoped to reveal the safety perspective of the participant, but instead we observed that in terms of a safety philosophy or paradigm, participants radically contradicted themselves throughout the interview. For example, one respondent claimed that ‘following the rules assures total safety’, then gave an example of a problematic company rule and explained that ‘following the rules does not mean you’ll be safe’. It was not unusual for a single participant to express attitudes and beliefs representing the range of paradigms derived from the HF literature.

During the course of our research we also observed an airline revising their internal incident investigation process. Their new manual proscribed the application of several different, independent investigation methods in spite of some inconsistencies between them. All of the methods were well known industry tools. Although this approach seems illogical to us, from the company’s point of view each of the methods must be ‘right’ because it was published somewhere by an expert, thus applying multiple methods will give results that are ‘even more right’. They never questioned the underlying model or why traditional incident investigation techniques were falling short of the airline’s needs.

Discussion: Integrating and Interpreting Ambiguity and Contradictions

Considering these findings together indicates to us that practitioners (i.e., non-experts in human factors or ergonomics, etc.) may not have a coherent, consistent, complete framework guiding how they view and understand safety. They may call up individual ideas from different paradigms or frameworks depending on the situation or the cognitive availability of the idea. There are many possible explanations for this, among them that practitioners may not have or need a coherent framework and may not even be aware when they express contradictory ideas. Or they may realise that there are frameworks but may apply different ones to different situations. As Hollnagel (2004) and Amalberti (2006) have suggested, different accident models or approaches may be appropriate in different contexts.

The Limitations of Attitude Measurement

These results also raise the problem of whether it is even possible to ‘measure’ beliefs or attitudes in this way. Attitudes are assumed consistent beliefs held by an individual. From the literature we learn that people’s attitudes in response to a certain question may change quite a lot depending on the context, how a question is phrased, or even the order of questions in a survey. Repeated measures of attitudes shows low stability in how people respond in surveys (Bertrand and Mullainathan, 2001; Zaller and Feldman, 1992). Burr (2003) argues that attitudes are neither stable nor coherent. It appears rather as if ideas (and attitudes) are in the service of action (Schwarz, 2007). Talk and behaviour are some of the tools people use to bring about different effects and points in their social encounters. That means that people may change their responses depending on, for example, how they want to portray themselves in the survey or interview (known as ‘social desirability bias’), or depending on what other ideas or desired actions were triggered by the question (e.g., to distance themselves from/identify with a colleague who ‘made a mistake’).

For researchers the paradigm or scientific framework is explicit as part of our professional identity. We normally need to have a well defined, theoretically consistent, stable perspective in order to do our work. Whereas researchers may see competing and/or contradictory ideas, practitioners may not. From the field of conversation analysis it has been shown that an individual may use completely contradictory ideas depending on what point he or she is trying to make (Potter and Wetherell, 1987). For example, in our interview data, it was common for a participant to express beliefs that accidents may arise from normal daily operations and environmental constraints, as well as claiming that accidents stem from some remarkable human error or failure, deserving of inquiry and sanctions.

Different discourses about what is safe and what are sources of risks are available for practitioners to position themselves along different dimensions or define their identities. They may draw from any convenient paradigm or popular folk wisdom that seems applicable to justify an action or explain a phenomenon. Hence, variances in the local context of culture, groups and individuals may introduce conflicting responses. As people become aware that there are competing models (as HF experts and researchers supposedly are) they may be less likely to express such contradictory ideas.

Two Explanations, One Conclusion

The neutral overall response combined with a large internal variation in scores could have another obvious explanation: the survey may not do its job, it might lack validity and not measure what it claims to. However, even if this is the case the implications of the result are the same since the questionnaire items represent common assumptions prevalent in accident reports and safety literature taken out of context (such as ‘human error is the largest threat to flight safety’). Thus if the survey is measuring what we intended, it indicates that practitioners do not have a strong allegiance to the Traditional Perspective. If the survey’s validity is in doubt, this demonstrates how the common HF assumptions or ‘folk wisdom’ used indiscriminately in aviation safety are ambiguous and even unconvincing when presented out of context.

In either case, the bottom line is that the predominant HF paradigm does not act as a successful framework for practitioners to understand non-technical issues in a consistent manner. Although not loyal to any single perspective, at least practitioners appear open to plausible-sounding argument. If Resilience hopes to do better we have our work cut out for us to create models and tools which are clear enough for users to recall and apply in their daily work.

Is Resilience Ready for the Aviation Industry?

We started out by asking whether the aviation industry is ready for Resilience. We were curious whether they are prepared and motivated to accept the Resilience perspective mainstream. During our inquiry of the safety perspectives of a sample of aviation professionals from around the Western world we discovered that the answer lies partly in the question: Is Resilience ready for the aviation industry? The insight gained from this study can help Resilience researchers and safety managers bring about the much-needed paradigm shift in aviation.

First, we saw that HF researchers and safety leaders (who develop the systems, model, rules and safety management tools) need to consider their target user population, since our survey results revealed variations in perspectives according to national culture and occupation. Second, the aviation practitioners in our study were not loyal to the Traditional Safety Perspective – in fact they may not think about HF and safety using any coherent perspective the way HF specialists do. This implies both a problem and an opportunity for Resilience advocates: It may be easy to convince people that Resilience makes sense, but they may not necessarily apply it consistently or exclusively.

The variations in perspectives identified in this study also help to dispel the myth that aviation is a purely technical domain in which standardisation has eliminated all variations in how people do the work. The human contribution is critical since there is always room for interpretation, no matter how standardised and regulated it becomes. Flying, controlling, and maintaining aircraft involves more than just checklists, radio frequencies, and torque settings. It took tragedy for the world to recognise how national culture influences communication on the flight deck and with ATC. Likewise, cultural differences (of any type) cannot be ignored, as they exist throughout the commercial aviation industry.

Acknowledgements

This work was supported by Dédale, the Foundation for an Industrial Safety Culture (FonCSI), and HILAS ‘Human Integration into the Lifecycle of Aviation Systems’ (a European Commission 6th Framework project).