Keywords

trustworthy; design; software; NFR Approach; Phoenix; Air Force

• Trustworthy software systems

• Case study: Phoenix System

• Understanding the NFR Approach

• Applying the NFR Approach for trustworthy software design

Introduction

The National Institute of Standards and Technology defines trustworthy information systems as reliable, usable, interoperable, and secure [1]. Trustworthy systems are essential for critical operations and are expected to deliver when they are most needed. There are environments in military, government, and civil domains where trustworthiness is an essential property. Examples include missile deployment control systems, the tax submission system of the federal government, and nuclear safety control systems. Improving software trustworthiness is considered the most important goal in cybersecurity by 2015 [2], and trustworthiness in cyberspace is a national priority in the United States [3]. Trustworthiness is also an important emerging requirement for software systems deployed by the US Air Force.

Trustworthiness has been defined differently by different sources, based on their approach to determining trust in a system. For example, in [4], trustworthiness is defined as the degree of confidence that the system meets its requirements; [5] defines trustworthiness as a level of confidence in using software engineering techniques to reduce failure rates and enhance testing, reviews, and inspections; and in [6, p. 114], a trusted computer system is defined as a system that employs sufficient hardware and software integrity measures to allow its use for processing simultaneously a range of sensitive or classified information. Another definition of trustworthiness [7] includes correctness, safety, availability, reliability, performance, security, and privacy. A discussion of software trustworthiness among stakeholders often invokes numerous non-functional attributes like reliability, safety, usability, portability, and maintainability, which together ensure non-interference with the normal operation of the system.

However, there seems to be a lack of systematic processes for developing trustworthy systems. For example, [8] proposes a technique to develop a trustworthy operating system for software applications, and [9] proposes a trustworthy computing platform for a voting application; however, each follows its own approach to develop those specific target systems. While trustworthy systems evaluation criteria have been proposed [6], these criteria are oriented mostly toward incorporating security requirements into systems being developed. A trustworthiness model for open-source software has been proposed in [10] wherein factors that impact trustworthiness in a product are identified and then evaluated for the extent to which the factors have been achieved. A process for improving trustworthiness has been suggested in [11] with the belief that a trustworthy process improves trustworthiness in the product.

In this chapter we employ the goal-oriented framework called the NFR Approach [12] for engineering trustworthy software systems. The NFR Approach, where NFR stands for “non-functional requirements,” treats trustworthiness as a goal to be achieved during the process of software development. NFRs are global characteristics of systems such as flexibility, security, and performance, and they synergistically or conflictingly interact with each other. The NFR Approach is well suited to reason about these NFRs and their interactions. The NFR Approach adapts the NFR Framework [13] to ensure that the system being developed satisfies the goals for which the system is being developed during the process of system development. The NFR Approach uses a structure called the Softgoal Interdependency Graph (SIG) to analyze and reason about trustworthiness. The SIG captures the definition(s) for trustworthiness, depicts design elements as softgoals, and records design rationale. The SIG also provides a platform for arriving at consensus among alternate definitions of trustworthiness from stakeholders and for qualitatively evaluating the extent of trustworthiness in a design; it also maintains an archive of design decisions. The NFR Approach has been used to develop adaptable [14], testable [15], and maintainable [16] systems, and we extend it to develop trustworthy software systems.

It is widely accepted that NFRs such as trustworthiness must be incorporated into designs from the earliest stages so that the final system is trustworthy. Since software architecture [17,18] is the first step in developing a software system after requirements have been elicited, the software architecture should itself be trustworthy for the final system to be trustworthy. In this chapter we develop trustworthy software architecture for the Phoenix system [19] used by the US Air Force. The Phoenix system is a middleware system developed by the air force for use in tactical and enterprise systems. Phoenix belongs to the class of message-oriented middleware and provides transparent message transport facilities over multiple operating systems and network protocols. Applications can communicate with Phoenix using well-defined interfaces and exchange real-time data. Phoenix operates using a producer-consumer paradigm and uses a store-and-forward technique for increased reliability of message transport over any unreliable communication links. The code base of Phoenix is over 100,000 lines of code and consists of fifteen subsystems.

During the process of architecture design, multiple views for the software architecture [20] are developed, including component and connector view, detailed structural view, logical deployment view, physical deployment view, and use-case scenarios. These views follow from the functional and non-functional requirements for the Phoenix system, where the functional requirements specify what is to be done, and non-functional requirements specify the characteristics of the software system such as reliability, maintainability, and flexibility. In this chapter we re-engineer trustworthy designs for the Phoenix system using the NFR Approach. This chapter is a significant extension of the presentation in [21]; detailed discussion of our results is included in this chapter.

The remainder of the chapter is organized into four sections and a summary, as follows: the first section describes the NFR Approach briefly; the second section describes four use-case scenarios that exemplify the lack of trustworthiness in the current Phoenix system; the third applies the NFR Approach to design the trustworthy Phoenix system. The fourth section discusses validation and lessons learned, and the summary concludes the paper and provides directions for future work.

The NFR approach

The NFR Approach is a goal-oriented approach that can be applied to determine the extent to which objectives are achieved by a process or product [22]. NFR stands for “non-functional requirements,” which represent properties of a system such as reliability, maintainability, and flexibility, and could equally well represent functional objectives and constraints for a system. In this chapter we apply the NFR Approach to design a trustworthy software system by evaluating whether a specific design element satisfied trustworthy requirements for the system. The NFR Approach uses a well-defined ontology for this purpose that includes NFR softgoals, operationalizing softgoals, claim softgoals, contributions, and propagation rules; each of these elements is described briefly below (details may be seen in [13]). Furthermore, the NFR Approach uses the concept of satisficing, a term borrowed from economics, which indicates satisfaction within limits instead of absolute satisfaction, since absolute satisfaction of NFRs is usually difficult.

NFR softgoals represent NFRs and their decompositions. Elements that have physical equivalents (process or product elements) are represented by operationalizing softgoals and their decompositions. Each softgoal is named using the convention

where Type is the name of the softgoal and Topic (could be zero or more) is the context where the softgoal is used; Topic is optional for a softgoal; for a claim softgoal, which is a softgoal capturing a design decision, the name may be the justification itself.

Following decompositions of either the NFR softgoals or the operationalizing softgoals are possible:

Contributions (MAKE, HELP, HURT, and BREAK) are made by operationalizing softgoals to the NFR softgoals. Reasons for contributions are captured by claim softgoals, and claim softgoals may form a chain of evidence where one claim satisfices another, which in turn satisfices another, and so on. Each of the four types of contributions has a specific semantic significance: MAKE contribution refers to a strongly positive degree of satisficing of the objectives (represented by NFR softgoals) by artifacts (represented by operationalizing softgoals) under consideration, HELP contribution refers to a positive degree of satisficing, HURT contribution refers to a negative degree of satisficing, and BREAK contribution refers to a strongly negative degree of satisficing. Due to these contributions, some of the softgoals acquire labels that capture the extent to which a softgoal is satisficed: satisficed, weakly satisficed, weakly denied (or weakly not satisficed), denied (or not satisficed), or unknown (indicated by an absence of any label attribute). Moreover, high priority softgoals, decompositions, and contributions may be indicated using the criticality symbol. The graph that captures the softgoals, their decompositions, and the contributions is called the Softgoal Interdependency Graph (SIG). A partial ontology of the NFR Approach is shown in Figure 13.1.

Propagation rules propagate labels from child softgoal to the parent across decompositions, from operationalizing softgoals to NFR softgoals across contributions, and from claim softgoals to contributions; propagation rules aid in the rationalization process of the NFR Approach. In a SIG represented graphically, the NFR softgoals and their decompositions are shown at the top of the figure, the operationalizing softgoals and their decompositions are shown at the bottom of the figure, while the contributions between the operationalizing softgoals and the NFR softgoals are shown in the middle. Therefore, contributions are usually received by the leaf NFR softgoals that are at the bottom of the NFR softgoal decomposition hierarchy. While detailed propagation rules may be seen in [13], a simplified list is provided here:

Upon applying these propagation rules, if the root NFR softgoals are satisficed, then the goals for the domain of interest have been met, to a large extent. In this chapter, the root NFR softgoals are related to trustworthiness, and therefore the SIG helps determine the extent to which a particular design is trustworthy.

The NFR Approach requires the following interleaving tasks, which are iterative:

The Phoenix system and trustworthiness deficit

In this section we describe the lack of trustworthiness in the current Phoenix system. The Phoenix system is a middleware, and as such, it serves as a transparent transport layer for its users. Publishers and subscribers send messages to each other over the Phoenix system. Figure 13.2 shows four use-case scenarios for the Phoenix system as it exists now.

As can be seen in Figure 13.2 (a), the use-case scenario describes the lack of confidentiality in the current Phoenix system since messages are sent in plain-text. In Figure 13.2 (b), the scenario describes the lack of availability of the system when large messages are sent between the publisher and the subscriber—the system simply crashes. Likewise, the scenario of Figure 13.2 (c) describes how accumulation of non-dropped connections can also make the system available after a period of time. The scenario of Figure 13.2 (d) describes how the lack of sufficient authentication and authorization can reduce the integrity of the data in the Phoenix system due to an attack. For these and other reasons, Phoenix is not considered trustworthy enough for use in future applications that have more stringent trustworthy requirements. In subsequent sections we elicit these trustworthy requirements, analyze the shortcomings using the NFR Approach, and suggest modifications to the Phoenix system for the new environment.

Application of the NFR approach for designing a trustworthy Phoenix system

In this section, we describe the application of the five tasks of the NFR Approach for generating trustworthy designs for the Phoenix system. To obtain trustworthiness requirements, we interviewed stakeholders—users who plan to use the Phoenix system in a trustworthy environment. As mentioned earlier, trustworthiness, being an NFR, can have multiple definitions and each stakeholder can have a different definition for trustworthiness; we apply the five tasks of the NFR Approach to one definition and discuss how the NFR Approach can be used for building consensus among stakeholders. In order to automate the evaluation process, we used the StarUML tool [23] for developing the SIGs. This tool is free to use and comes with a toolset for capturing all aspects of a SIG; moreover, all propagation rules are built into StarUML, which makes its use for the evaluation and analysis task convenient.

Validation and lessons learned

The architectural modifications for improved trustworthiness were validated by three of the original Phoenix development engineers. We presented the modified designs along with a questionnaire that required them to evaluate the artifacts by comparing them with the system documents for Phoenix (which we did not have access to) in the spirit of the Delphi technique [30]. Developers answered the questionnaires independently of each other without having recourse to answers provided by others. The developers were provided the updated architectural models and a modified version of Table 13.3 as part of their questionnaire; their responses are tabulated in Table 13.4.

The development engineers were in agreement that techniques such as passwords, biometrics, and Kerberos will help the Authorization Service component satisfy the requirement for Verifiability of Data Source; one of them pointed out that two-factor authorization will also be a preferred technique. Likewise, the techniques identified from catalogs for improving integrity of the Phoenix system, namely, access control lists and feedback control, were also perceived by the engineers as helpful to achieve the goal; however, it was felt that integrity can still be impacted by bypassing Authorization Service and attacking other components of the system directly. In row 3 of Table 13.4, the technique neighborhood group data caching in a deployed physical system was expected to improve consistency over constrained communication links. However, synchronization of fetched data with that of the central store was important so that data viewed by users were not stale; a differential data pre-fetching (where only differences were obtained from the central store) was felt to be more useful, and this technique is planned for incorporation in a future version of the Phoenix system. In row 4 of Table 13.4, the common techniques of increasing bandwidth/processing power/buffers in a deployed physical system were expected to improve performance of the system; however, it was pointed out that scalability of such an increase is not linear in general and incurs costs of time and effort to realize improved performance. Row 5 of Table 13.4 was a control row. It was used to ensure that engineers were actually focusing on the questionnaire; the technique “noncryption” was a made-up word for plain text transmission, and all three engineers were unanimous in their opinion that this will not improve security.

Table 13.4 conveys the point that recommendations from the NFR Approach are largely acceptable to the industry practitioners from a trustworthy system development standpoint. An interesting observation is the comment in row 3 of Table 13.4: the feature identified as important using the NFR Approach has been under consideration by the development team for some time, but we arrived at this point using the systematic process of the NFR Approach, while the development team identified this issue from user feedback. The suggestions from the SIG analysis could be used to improve the trustworthiness of the system being designed with a high confidence that these suggestions will find acceptance from the designer community.

An important aspect of the NFR Approach is that it helps build consensus among stakeholders: we obtained three differing definitions of trustworthiness from stakeholders. Figure 13.7 shows another definition for trustworthiness from a different stakeholder—trustworthiness includes reliability and dependability attributes. The oval shapes in Figure 13.7 are the constraints imposed on a design to satisfy certain trustworthiness requirements. The architectural models of Figure 13.6 may not satisfy all the requirements in Figure 13.7; however, the NFR Approach provides a mechanism to reason about the “gaps,” if any. For example, one constraint (oval shape) is that all messages be encrypted to satisfice the NFR softgoal Security [Messages]. However, the connector Phoenix Channel in Figure 13.6 does not encrypt messages and therefore, after applying the propagation rules of the NFR Approach, we find the models of Figure 13.6 are not trustworthy by this definition (Figure 13.7), either. Therefore, if an Augmented Phoenix Channel connector is used in the updated models of Figure 13.6, then these updated models will satisfy to some extent the definition in Figure 13.7 besides meeting all requirements in Figure 13.3. This way, consensus between definitions may be reached using the NFR Approach.

As mentioned in the introduction, the SIG captures trustworthiness definitions, architectural or design elements, and the design rationale in the form of claim softgoals. The propagation rules help qualitatively evaluate the extent to which designs satisfy trustworthiness requirements, and the historical record of design decisions are captured by different versions of the SIG. If any change occurs to the SIG due to requirements changes, design changes, or rationale changes, the resulting change to trustworthiness can be quickly recomputed; therefore, impact analysis can be performed. Also, “what if” scenarios can be studied by deliberate changes to the SIG and by analyzing the resulting impact on trustworthiness.

One aspect that we did not discuss is the process of converting interview notes into a SIG, such as in Figure 13.3 and Figure 13.6. We have discussed this aspect in another paper [31]; however, this process involves creating elements of SIG from interview notes and is a predominantly manual process. There does not seem to be a one-to-one correspondence between a given interview note and the SIG developed; however, based on our experience, the results of the reasoning process seem to be highly synchronous between different SIGs generated from an interview note. Therefore, we believe that the NFR Approach’s process is repeatable across projects and organizations.

Certain aspects of the NFR Approach have not been described here; for example, the propagation rules in the section “The NFR Approach” are simplified versions (as mentioned earlier). For example, rule R6 assumes if even one child softgoal is not satisficed in an AND-decomposition, then the parent softgoal is denied. However, what if, as we observe in Table 13.2, rows 8 and 14, one of the child softgoals is weakly denied? The NFR Approach actually defines rules that allow the parent to the weakly denied instead of denied, but we have assumed simpler rules in this chapter to convey the spirit of the NFR Approach without overwhelming the reader with details. Another aspect of the NFR approach not discussed here is the fact that claim softgoals can have labels as well, which allows claims (or rationales) to change with time and their impact(s) to be quickly reflected in the SIG.

Summary

Trustworthiness is an important emerging requirement for software systems deployed by the US Air Force. While NIST [1] has defined trustworthiness to be reliable, usable, interoperable, and secure, there seems to be no uniformity in the definition of this requirement within the Air Force. Moreover, there seems to be no systematic method to develop trustworthy systems. In this chapter, we employ the goal-oriented framework called the NFR Approach [12,13] for engineering trustworthy software systems. The NFR Approach treats trustworthiness as a goal to be achieved during the process of software development. Since software architecture is the first step in the development of a solution, we employ the NFR Approach to develop a trustworthy architecture for a software system. The software system considered for validation is the Phoenix system [19] used by the US Air Force; the Phoenix system is a middleware system developed by the Air Force for use in tactical and enterprise systems. We applied the NFR Approach, developed trustworthy architectural models for the Phoenix system, and validated the models by having them evaluated by Phoenix’s engineers. The results are encouraging, with many design updates for improved trustworthiness being approved by developers as being not only feasible but actually useful for a trustworthy Phoenix system.

In the future we plan to partially automate this process by incorporating natural language processing to identify elements of SIGs from interview notes, which should speed up the process considerably. Furthermore, integrating natural language processing with tools available for handling NFR approach, for example, StarUML [23], will enable automating the processing of knowledge recovery from users. However, we believe that the NFR Approach provides a systematic technique for developing trustworthy software systems.

Acknowledgments

This research was sponsored by Air Force Research Laboratory/Information Directorate, Rome, New York, USA. In the summers of 2011 and 2012, the authors spent many months on this project, and we thank several engineers at the lab for their help, including Mark Lindermann, James Hanna, Vaughn Combs, James Milligan, Chris Schuck, Tim Blocher, Dawn Nelson, Mark Mowers, and Brian Lipa. We also thank the reviewers of the initial version of this chapter for their valuable suggestions and comments.

References

1. [Internet]. Retrieved from: <www.nist.gov/itl/tis/index.cfm> [accessed 22.05.12].

2. Software 2015: A national software strategy to ensure U. S. security and competitiveness. Report of 2nd National Software Summit. [Internet]. Retrieved from: <www.cnsoftware.org/NSS2>; 2005 [accessed 06.06.12].

3. Trustworthy cyberspace: Strategic plan for the federal cybersecurity research and development program. Executive Office of the President, National Science and Technology Council; 2011 Dec.

4. Parnas DL, van Schouwen AJ, Kwan SP. Evaluation of safety-critical software. Communications of ACM. 1990;33(6):636–648.

5. Amoroso E, Taylor C, Watson J, Weiss J. A process-oriented methodology for assessing and improving software trustworthiness. Proceedings of the 2nd ACM Conference On Computer and Communication Security 1994;39–50.

6. DoD 5200.28-STD. Department of Defense standard: Department of Defense trusted computer system evaluation criteria. [Internet]. Retrieved from: <csrc.nist.gov/publications/history/dod85.pdf>; 1985 [accessed 22.05.12].

7. Hasselbring W, Reussner R. Toward trustworthy software systems. IEEE Computer. 2006;39(4):91–92.

8. Garfinkel T, Rosenblum M, Boneh D. Flexible OS support and applications for trusted computing. [Internet]. Retrieved from: <suif.stanford.edu/papers/trusted-hotos03.pdf>; [Accessed 22.05.12].

9. Fink RA, Sherman AT. Combining end-to-end voting with trustworthy computing for greater privacy, trust, accessibility, and usability. [Internet]. Retrieved from: <csrc.nist.gov/groups/ST/e2evoting/documents/papers/SHERMAN_trustworthye2e-NISTrevised9-25-09a.pdf>; [accessed 22.05.12].

10. Taibi D. Defining an open source software trustworthiness model. Proceedings of 3rd International Doctoral Symposium on Emperical Software Engineering 2008.

11. Yang Y, Wang Q, Li M. Process trustworthiness as a capability indicator for measuring and improving software trustworthiness. ICSP '09 Proceedings of the International Conference on Software Process: Trustworthy Software Development Processes 2009;389–401.

12. Subramanian N, Chung L. Software architecture adaptability: An NFR approach. Proceedings of the International Workshop on Principles of Software Evolution (IWPSE 2001) Vienna: ACM Press; 2001; p. 52–61.

13. Chung L, Nixon BA, Yu E, Mylopoulos J. Non-functional requirements in software engineering Boston: Kluwer Academic Publishers; 2000.

14. Chung L, Subramanian N. Adaptable architecture generation of embedded systems. Journal of Systems and Software. 2004;71(3):271–295.

15. Chung L, Subramanian N. Testable and adaptable architectures for embedded systems. University of Texas at Dallas Technical Report No. UTDCS-22-01; 2001 Nov.

16. Subramanian N, Puerzer R, Chung L. A comparative evaluation of maintainability: A study of engineering department’s website maintainability. Proceedings of the International Conference on Software Maintenance; 2005 Sep; Budapest, IEEE Computer Society, Hungary. p. 669–672.

17. Shaw M, Garlan D. Software architecture: Perspectives on an emerging discipline Prentice Hall 1996.

18. Bass L, Clements P, Kazman R. Software architecture in practice. SEI Series in Software Engineering 2nd ed. Addison-Wesley 2003.

19. Combs VT, Hillman RG, Muccio MT, McKeel RW. Joint battlespace infosphere: Information management within a C2 enterprise. 10th International Command and Control Research and Technology Symposium 2005.

20. Eeles P, Cripps P. The process of software architecting Upper Saddle River, NJ: Addison-Wesley; 2010.

21. Subramanian N, Drager S, McKeever W. Engineering a trustworthy software system using the NFR approach. Presented at the Systems & Software Technology Conference; Salt Lake City, UT. 2012 Apr.

22. Chung L, Supakkul S, Subramanian N, et al. Goal-oriented software architecting. In: Avgeriou P, ed. Relating Software Requirements and Architectures. Springer 2011;91–109.

23. [Internet]. Retrieved from: <http://staruml.sourceforge.net/en/modules.php>; [accessed 11.10.11].

24. Ware WH. Perspectives on trusted computer systems. Rand Library Collection; 1988 Sep. p. 7478.

25. Linderman M, et. al. A reference model for information management to support coalition information sharing needs. 10th International Command and Control Research and Technology Symposium. Verginia; 2005 Oct.

26. Stallings W, Brown L. Computer security: Principles and practice 2nd ed. Prentice Hall 2011.

27. Mandke VV, Nayar MK. Information integrity: A structure for its definition. In: Strong DM, Kahn BK, editors. Proceedings of the 1997 Conference on Information Quality; MIT, Cambridge, MA, 1997.

28. Shanmugavadivu K, Madheswaran M. Caching technique for improving data retrieval performance in mobile ad hoc networks. International Journal of Computer Science and Information Technologies. 2010;1(4):249–255.

29. Hennessy JL, Patterson DA. Computer architecture: A quantitative approach 5th ed. Morgan Kaufmann Publication 2011.

30. Hsu C-C, Sandford BA. The delphi technique: making sense of consensus. Journal of Practical Assessment, Research, and Evaluation. 2007 Aug;12:10 ISSN: 1531-7714.

31. Subramanian N, Drager S, McKeever W. Eliciting software design from interviews using the NFR approach. Submitted for publication.