Acquiring software is very different from other types of consumer purchases. However, to better understand the need for a different approach for acquiring software, it is useful to consider the following scenarios for customer requirements in other consumer-oriented businesses. I encourage you to ask yourself what you would do in the following scenarios if you were the consumer:

Now, I’m willing to bet that in Scenario 1, you would bring the cold Big Mac back to the counter and demand one that was piping hot. And I suspect that 9 times out of 10 you will get exactly what you request. The probability of anyone receiving a Big Mac served at or below room temperature out of the 550 million Big Macs served each year is relatively low, given McDonald’s focus on quality and customer satisfaction. I wonder how many of those orders for Big Macs include an explicit request from the customer to deliver it to the counter hot. It is an implied requirement, but one that managers are anxious to fulfill. There is no need to specify your desire for a hot Big Mac upon ordering to ensure that the temperature is well above room temperature.

In Scenario 2, both you and the representatives at Amazon.com would likely agree immediately that 16 missing pages is a significant defect. I suspect that, given Amazon.com’s reputation for excellent customer service, you would receive a new book with all of the pages within a day or so without paying for additional shipping, after bringing this defect to their attention. In this scenario, you never specified that the book you ordered needed to have all of the pages. But no service representative from Amazon.com would point out that you did not specify that the book needed all of the pages. Once again, the requirement that the book contain all of the pages is implied. The inability of a bookseller to sustain a profitable operation while offering only partial pages in its books reinforces the industry’s common practices of offering all of the pages for every book.

Scenario 3 is much less likely to result in a satisfying outcome. The reason for this is that, although security may be an implicit part of your requirements when you purchase software, it is certainly not explicit. Furthermore, security rarely figures in the requirements when the software is developed, unfortunately. The majority of commercial software providers do very little to identify security vulnerabilities and address them during the development process.

As I said at the beginning of this chapter, the gap between adherence to explicit and implicit requirements is substantial. Commercial software providers focus on delivering functionality to meet the time-to-market demands of customers, and they do it well. New features and capabilities make it easier for them to differentiate their products from other commercial software developers and encourage their customers to purchase new software or upgrade old packages.

Clearly, the omission of security from explicit requirements is no reason to believe that customers don’t care about it, or don’t understand its importance. But customers don’t explicitly specify security requirements for two reasons:

Determining whether a Big Mac is hot enough is a relatively trivial exercise for a customer. Determining whether a software product has security vulnerabilities requires an entirely different level of effort and expressiveness. There is no standard measure to determine the presence of software vulnerabilities or a secure approach to software development, which would resemble using a thermometer to determine the temperature of a Big Mac.

I have worked for several highly regulated financial service firms with core business operations that rely heavily on the resiliency of their software. One of these companies, which I’ll give the fictitious name of Acme, launched an effort costing substantial money over several years and requiring intense involvement at the CIO level to reduce its security risks. The rest of this chapter explains how we started, championed, and extended these efforts.

At the time Acme committed itself to the long-term project described here, it had never made any previous attempts to improve software security. However, it had an impressive quality record, having made investments in the use of a consistent methodology for systems development and achieved a level 3 status in the Carnegie Mellon Capability Maturity Model Integration (CMMI) process. To address security, at first, Acme considered bringing in a few security vulnerability detection tools to use on selected projects in a “proof of concept” mode. This is a common approach that companies take when starting a secure software program.

But Acme’s chief information security officer (yours truly) read a book (Software Security: Building Security In, published by Addison-Wesley) on software security written by a recognized authority, Gary McGraw. Gary clearly points out that software security is more than identifying and removing defects in software code. In fact, over 50% of software vulnerabilities are based on defects in the architecture, not the code itself. I also recognized from previous experience in software development that identifying and addressing software defects early in the lifecycle is far more cost effective than fixing defects once the code is in production. I believed that emphasizing the identification and remediation of software defects early in the lifecycle was the best approach for creating an economic incentive for more secure software that the CIO was more likely to appreciate.

I knew from my previous experience in software development that encouraging software developers to think of security vulnerabilities as defects would be a significant challenge, requiring both a comprehensive education program and organizational development techniques for managing behavior change. I decided to put our focus on teaching developers to create secure code as opposed to spending a lot of money on pen testing services once the code was developed, tested, and implemented.

With a relatively mature application security program in place, Acme decided to tackle commercial off-the-shelf software, or COTS. Like most firms, Acme’s reliance on purchased software for business and workstation software was growing, and we had no security review prior to implementation of the software.

Within Acme there was a great deal of skepticism about the possibility of adopting any security controls for commercial software. The IT leaders were reluctant to support anything that added hurdles and time to the process of evaluating and selecting software packages. Acme business leaders were uncertain how software companies were going to react to customers requiring information on security controls in the development process.

But I was anxious to push for the adoption of security controls for at least a subset of commercial software applications. I recognized that software was playing an increasing role in commercial infrastructure, and that economics was driving companies such as Acme to depend increasingly on purchases from commercial software vendors. My rationale also took into account that over time, regulatory requirements were likely to include application security and that Acme needed to be ahead of the curve by adopting controls to avoid paying more for a “knee-jerk” reaction to a new regulation. For example, in April 2008 the Office of the Comptroller of the Currency (OCC) published comprehensive guidance on software security requirements for the financial service firms they regulate. It’s the first comprehensive regulatory requirement in the U.S. that is explicitly for software security. Attempting to adopt and implement a program to fully comply with the OCC requirements is a significant undertaking for firms without an established and mature program in place for software security. The cost alone for immediate compliance would be significant, in addition to the time and resource effort required. In addition, any control that we could implement would represent a significant improvement over the current situation, where we had no way to identify the security risks of using purchased software and no leverage over the providers. From my perspective, commercial software was simply one more important dimension of the IT supply chain that required appropriate controls.

When I presented the concept of commercial software controls to the Security Committee, there was significant debate. They ultimately agreed to try out this new process in a “proof-of-concept” project. I defined the evaluation criteria for the project, and I promised to provide the results to the committee.

I recognized that software companies were likely to resist the new requirements and that added delays in software procurement would be poorly received by business stakeholders. So I developed an approach where I would intervene in the negotiations with a software vendor prior to making a buying decision and simply request whatever artifacts the software vendor could provide to demonstrate effective controls in their software development process.

This request mirrored the information Acme was prepared to share with other customers who requested similar information from Acme. Acme had artifacts from the requirements, design, development, and testing phases of our software development lifecycle that demonstrated our security controls. The artifacts requested by Acme are identified in Figure 11-1. Each of the four boxes represents a core phase of a conventional waterfall methodology, identified by the label at the top of the box. The bottom of each box lists the categories of vulnerability detection tools and services we use.

In short, Acme laid out a conventional software development process and the types of tools and services we used for quality control and security checking, and asked vendors for similar information.

Figure 11-1. Acme’s security controls, as shown to customers

In addition, I decided to test out a new service from a young start-up firm that developed an innovative way to scan code binaries; for this article I’ll call it Verify. This new method did not require access to source code, a significant obstacle for commercial software vendors who go to great lengths to protect their proprietary intellectual capital. Acme offered the use of Verify to its suppliers as an alternative to providing Acme with the information previously mentioned about the four stages of development. If vendors opted to use Verify, Acme would use it to scan the binaries from their latest release of software. The software vendors receive the detailed scan results so they could identify and remediate the vulnerabilities based on risk and their priorities. I could then influence their priorities based on Acme’s needs. The Verify service provides a risk score using a consistent framework, and Acme is able to establish the required risk score for each software product or type of software product. So the level of vulnerabilities detected are comparable with other software products, applications developed by third parties, and systems that Acme has internally developed.

My proof-of-concept project included three mid-sized software firms. Two out of the three software vendors resisted the initial request for artifacts and reluctantly agreed to the Verify service. The third firm had coincidentally been asked to use the Verify service by another financial services firm as part of its own vendor evaluation process. This software vendor actually scored lower than its competitor in vulnerabilities discovered by Verify, but was selected by the financial services firm anyway because it showed a commitment to address the vulnerabilities and improve the software. This software firm hired Verify too and incorporated the service into its controls on a consistent basis, ultimately improving its risk score and the resiliency of its software. Several other firms, including Acme, asked for and received the vulnerability summary information from the software firm.

The results for the three software vendors that we asked for security controls were generally positive. Two of the three firms responded well to the vulnerability information and used the results to prioritize their remediation work effort. One of these vendors actually received a relatively lower risk score than its competitor in a separate evaluation by a financial services firm that was using the risk rating as an evaluation criteria, but the vendor’s response to the vulnerability risk information was so proactive and effective that it won the business anyway and later improved its risk score for Acme.

Acme does not expect perfect risk scores; it is more encouraged by vendors that demonstrate a willingness to learn how to improve their security controls over time.

The third vendor received a relatively low vulnerability score and then challenged the accuracy of the binary scanning technology. This firm had invested in the Common Criteria for Software from BITS, a financial services industry organization. This is a rigorous documentation exercise for controls and vulnerabilities that is labor intensive and whose value is tied directly to the specific version of software reviewed. Acme agreed to accept this as an artifact from this vendor but encouraged the vendor to engage with Verify. The ensuing dialogue with the vendor provided an effective assessment of security controls within the vendor’s environment and a significantly better understanding of the security risks that would go with the adoption of the vendor’s product into Acme’s environment.

The proof of concept demonstrated enough success for Acme to move forward and enforce security requirements for all future business application software purchases. I analyzed the existing inventory of desktop applications currently offered by Acme and selected products where we should require a security assessment prior to the annual license renewal. The Vendor Management Office contacted each vendor, explained the new security requirements, and gave them our choices of options for compliance.

Once again, Acme decided to take a gradual approach to improving software security by applying new security requirements only to a subset of the software purchased, business application software, which is typically web-based and offers the highest risk of exploit. Our plan was to evolve security assessment until we felt it was mature, and then apply similar requirements to all software procured by Acme.

Software, whether commercially produced or internally developed by IT organizations, has penetrated to the core infrastructure of the global economy. Very few commercial products or services can be provided effectively today without the contribution of software. The planes we fly for air travel were designed, manufactured, operated, and maintained by processes dependent on software functionality. The food we eat, the cars we buy, and the goods we acquire are enabled by software. Yet with all of this demand for functionality from software, companies and consumers still have a limited understanding of processes for creating resilient software with control of security vulnerabilities.