When Microsoft first embarked on the journey called Trustworthy Computing in 2002, the first major foray into changing the software development process was the security push. The goal of the push was simple: to hunt for security bugs, triage them, and fix them once the push was complete. The problem with doing security this way is that security pushes are not a sustainable way to produce secure software because a push misses the point of building secure systems. Building secure software requires you to reduce the chance that security bugs are created in the first place, and hunting for bugs late in the process is anathema to “getting it right” early in the process. Just after Bill Gates announced Trustworthy Computing, all major Microsoft products went through a security push. The products included Microsoft Windows Server, Exchange Server, SQL Server 2000, and Office.
But the security push has a place in the Software Development Lifecycle (SDL), which requires that teams use the push primarily to focus on legacy code—in other words, code created before the current development cycle. However, a security push is not a “quick fix” for insecure code. Do not for one moment think you can put off security until the security push.
Important
Do not put off focusing on security until the security push. Doing so is an egregious violation of the SDL.
A security push occurs after a product has entered the verification stage and has reached code and feature complete (often around the beta time frame). Because the results of the security push might alter the default configuration and behavior of a product, a final beta test release should be planned after the security push is complete and all bugs and changes required based on the push have been resolved.
Note
Anecdotally, product team members have told us the push is hard work but that it gives the team a valuable chance to focus on “nothing but security.”
All software that is not brand-new uses some form of legacy code. Invariably, this code was written when less was known about building secure software or, in many cases, when security was not a main project objective. The push targets mainly this older code. We expect the need for security pushes to dwindle over time as old code is removed or upgraded to the quality of newly created code. Eventually, the need for a security push will diminish because new and earlier code is more secure. In fact, the authors believe the push might eventually be no longer needed.
Important
The goal of a security push is to find bugs, not to fix them. Fixing is done after the push.
The security push is not restricted to code. The main tasks during the push are as follows:
Training
Code reviews
Threat-model updates
Security testing
Documentation scrub
We’ll look at each of these in detail shortly, but first we want to outline the preparation process for a successful security push.
A successful push requires plenty of planning, and time for the push must be built into the schedule from the outset. If you tack the security push onto the schedule at a late date or as a “surprise event,” you’ll lose the benefits of the push. The person responsible for security of the product should drive the push, which includes setting up the push leadership team. In our experience, this team includes a small number of people representing all product design and development disciplines, such as development, testing, documentation, design, and project management, as well as one or more people from the team that owns the schedule and ultimately delivers the software to customers. At Microsoft, this team is usually the release management team, with assistance from a lead program manager.
At the outset, you should set up a security push Web site with push details, including the goals of the push, expected time frames, and a list of tasks per engineering discipline. The rest of this chapter outlines the recommended tasks.
Also, the Web site should have links to bug databases, incoming bug counts, and other up-to-date information. Make sure to keep this site current.
On the subject of bug databases, we’ve found that it’s useful to store security-push bug information in a separate bug-tracking database, if possible, owing to the possible high bug count. At the end of the push, this separate database is triaged for bugs that should be included in the main project bug-tracking database. If it’s infeasible to maintain a separate bug database, make sure you mark all push-related security bugs as “HowFound = Security Push”.
The push is complete when the required tasks are complete, not when some arbitrary amount of time elapses. Common exit criteria follow:
All personnel are up to date on security training.
All high-priority (Pri1) source code has been reviewed and signed off.
All Pri1 executable code has been signed off by the test owners.
All threat models have been reevaluated and updated (or, for very old components or products, threat models have been created for all components).
The attack surface has been reanalyzed, and the appropriateness of the default attack surface has been confirmed.
All documentation has been reviewed for correct security guidance.
Best Practices
Make the exit criteria crisp and actionable, and then broadly communicate the criteria to the team.
Obviously, you can save time spent on the push by making sure the threat models and documentation are up to date. Also, as the quality of older code improves over time, the push duration will diminish.
For large products at Microsoft such as Microsoft Windows, Office, and SQL Server, a push typically lasts no less than six weeks. For smaller products, the duration is typically no less than three weeks.
At an absolute minimum, the software development team will need training on what to expect during the security push as well as on push logistics. Some employees might also need to attend technical security-related training, so holding extra technical classes is worthwhile. These technical classes are especially useful if a high percentage of the team members are approaching their training anniversary.
Best Practices
Our group performs a good deal of security training for teams about to start a security push. We usually fine-tune existing class material so that it’s completely relevant to the team being trained.
The push logistics explanation should start with a short (perhaps 15-minute) introduction by a senior and well-respected individual, who emphasizes the importance of the security push.
Note
During the famed Windows Security Push of 2002, we provided a series of training events for each development discipline, and each training event was launched by a vice president. This provided a great deal of much-needed gravitas and emphasized Microsoft’s commitment in the early days of Trustworthy Computing.
No one really likes to review code for bugs, especially code written 10 or more years ago. Given the opportunity to review old code or work on a new, cool feature, developers will lean toward the latter. It’s just human nature. Reviewing code is slow, tedious, and mind-numbingly boring. Unfortunately, all code that runs on computers and is used as part of your software must be reviewed, regardless of age. Attackers don’t target only new functionality; they target all functionality. Make no mistake: the attackers will attack code regardless of its age. In fact, in some cases, it’s worth the attacker’s time finding bugs in older code because more people will have older versions of the software installed. The situation is compounded if the code is common to multiple versions of the product.
Important
Attackers attack all code, so waiting to make the code more secure in the next version of the product is not a good solution for protecting customers. You must secure the currently supported versions, too.
The first step in security code review is to build a database of all the files and assign an owner to each source code file. At this stage, don’t exclude any code from the review; include everything. Many source-code repositories don’t store the source code file “owner.” In this case, you can usually use the name of the last person who updated the file. Create a table in a database with the following fields:
File name. The name of the file, including the directory where it’s stored.
File owner. The name of the file owner.
Priority. The priority for reviewing the code. Ratings range from 1 to 3, with 1 being the highest priority.
Reviewed by. The name of the person who will review the code. If you plan to have multiple people review the code, you might want to make this a separate table.
Reviewed. A field with only three possible values: Yes, No, and Partial.
Comments. Any comments about the file that might be of interest to people reviewing the code.
Populate the table with the file and owner information. Of course, you should use a tool to do this; don’t do it manually. If you are using Microsoft Visual Studio 2005 Team Foundation Server, you can query the source code repository from within the system itself or create a Microsoft Office Excel PivotTable of the data you want. If you are more development-inclined, you can access the source code database programmatically.
The next step is to determine review priority. Much of this identification should be driven by the threat model. The highest-risk components that the threat model identifies are the high-priority code segments for review. That said, you can follow some simple rules to help determine priority:
Code running by default, listening on the Internet, or connecting to the Internet is Pri1.
Code with numerous prior vulnerabilities is Pri1.
Code executing with high privilege (for example, SYSTEM, administrator, root) is Pri1.
Security-related code (for example, authentication, authorization, cryptographic, and firewall code) is Pri1.
For .NET code, unverifiable code is Pri1. You can use the PEVerify tool (Microsoft 2006) to determine whether Microsoft intermediate language code is verifiable.
Code that parses data structures from potentially untrusted sources is Pri1.
Optionally installed code that runs with user privilege, or code that is installed by default that doesn’t meet any Pri1 criteria, is Pri2.
Setup code is generally Pri3 except for the code portions that set access controls or handle encryption keys or passwords. Consider these latter components Pri1 because of the sensitivity of the data and the possibility of getting default permissions settings wrong or leaving a password that’s supplied at installation time in the wrong place. (We’ve been surprised at how many times we have seen this in the past [Secunia 2004].)
The Q/A or test lead, not the development team, must agree with the priority list.
Code reviews should also cover sample code you make available to others who build on your code. Prioritizing sample code is a little harder than shipping code, but a good approach is to consider how your users will apply the sample code. If it is template code that requires minor modification to use in production environments, and it fits any of the Pri1 criteria noted in the previous list, the sample code is Pri1.
More Info
For more information on the tasks involved in performing a code review, refer to Appendix D, “A Developer’s Security Checklist,” in Writing Secure Code, Second Edition, as a minimum checklist of security issues to be aware of during code reviews (Howard and LeBlanc 2003). Also, 19 Deadly Sins of Software Security (Howard, LeBlanc, and Viega 2005) and “Expert Tips for Finding Security Defects in Your Code” (Howard 2003) provide guidelines for reviewing code for security bugs.
Note that the owner does not review the code but, rather, nominates someone else to review the code. One best practice is to simply have two owners swap their source code files and then review each other’s code. You might sometimes need to balance the workload equitably among code reviewers. If a reviewer finishes her assigned code, she should help others review their assigned code. But again, a reviewer should not review her own code.
Finally, you must build a list of all the executable files (.exe files, dynamic-link libraries, COM objects, assemblies, script files, and so on) that make up the product, and you must assign a test owner to each component. Again, create a table with the following columns:
Executable file name. The full path of the executable file
Test owner. The name of the test or Q/A person in charge of testing this component
Priority. Priority for reviewing the component, on a scale of 1 to 3, with 1 being the highest priority
Signed Off. Yes or No
Comments. Any comments about the component or the sign-off procedure
The goal of this task is to have the test people sign off on all high-risk executable components within the product. A successful sign-off means that the test team has agreed with the following:
The code that makes up that component has been reviewed.
The component has all appropriate security tests in place.
The component’s threat models are up to date and accurate.
The component’s attack-surface documentation is up to date and accurate.
Make no mistake, assigning executable-file owners is a very important substep in the SDL process and should not be taken lightly.
The architects and program managers need to review the threat models one more time. We know that it might seem like there has been a great deal of focus on threat models, but they are very important. As we write this chapter, Windows Vista starts its verification phase, in which threat models are reviewed to make sure that they are complete and correct. The beauty of doing this is that you can ascertain which areas the various component groups might have missed, if indeed they have missed anything. The critical portions of the threat model to look at are described in Chapter 9. To recap, here are some of the authors’ favorite things to look for in a threat model during the security push:
Determine whether the data flow diagram (DFD) needs to be changed since its last review. Software design changes that happen between the design phase and verification phase should be reflected in the DFD and, hence, the threat model as a whole.
Make sure all DFD elements are mapped to appropriate STRIDE threat categories.
Look at all the entry points into the system. Make sure the list is complete and has not changed since the last review.
Look at all the anonymous network-facing interfaces to the system. Should they be authenticated or restricted to a local subnet or list of trusted Internet Protocol (IP) addresses?
Make sure all the sensitive data stores are correctly protected. This protection often includes an access control list (ACL) review.
Make sure all data flows carrying security-sensitive data are adequately protected. This includes protection from disclosure (using encryption) and tamper detection (using message authentication codes or digital signatures).
You might be thinking, “Didn’t the prior chapter discuss security testing?” The answer is a resounding yes, but security-push testing has a slightly different focus. You can still employ many of the testing techniques discussed in the previous chapter, but testing during the push focuses on the highest-risk components only. It’s also good to do one last validation to make sure that you have a list of all file formats parsed by your code and that fuzz tests are in place for all of those file formats.
Program management drives the task of reevaluating your attack surface during the security push, providing two major benefits. The first is general “good security hygiene”: is your product exposing just the right amount of functionality to the correct users without putting them unnecessarily at risk? The simple list that follows will help you drive your attack-surface scrub:
Count all the open ports, sockets, SOAP interfaces, remote procedure call (RPC) end points, and pipes. Are they all needed by default?
Verify that all unauthenticated network entry points are needed. Can they be authenticated by default?
Verify that all network entry points are restricted to the correct subnet or set of trusted source addresses.
Verify that every process you run uses the right level of privilege to get the tasks done. Can you shed some privileges to protect customers?
Verify that the ACL is correct for every object you create. If you inherit base operating system ACLs in Windows, your code is probably correct because the operating system ACLs are likely to be good.
If you use datagram protocols, must your code listen on datagram protocols by default? Can you use stream protocols by default and allow users to opt in to datagram support if they need it and understand the risks?
The second benefit of reanalyzing the product’s attack surface is that it helps drive the priority ranking for the code review task. A review of attack surface will allow teams to understand which components have direct exposure to attack and, hence, the highest risk of damage if a vulnerability occurs in those components. Obviously, the highest-risk code must be reviewed the most thoroughly.
After the attack surface has been reevaluated, the attack-surface document should be updated as appropriate. The attack-surface document explains your attack-surface rationale by asking questions such as Just why is that port open? and Why is a specific ACL set this way?
Finally, the writers and editors involved in building the end-user documentation should review all their draft documentation to verify that the security best practices are correct and that the documentation does not recommend potentially bad practices. If the documentation provides code examples, they should show secure coding practices.
Seriously consider adding “Security Best Practice” and “Security Alert” sections to each topic outlined in the documentation if this has not already been done. Better yet, add a security icon to highlight such sections.
Your users should be informed of the security ramifications when enabling or changing functionality, but never lose sight of who the customer is. For example, if your product is used by network administrators and a setting is changed, the security ramifications of that change should be made known to the network administrators. You might include text like this:
Security Issue: Enabling this functionality opens port TCP/1067 to anonymous and authenticated users on the local subnet.
We have already touched on this topic a little in this chapter, but it’s worth repeating: The security push is finished when the approved security push tasks are complete.
The amount of time and energy and the degree of teamwide focus required for a security push will differ for individual teams based on the state of the code base and the team’s attention to security during the development process. Teams that have performed the following tasks will have a shorter security push:
Rigorously kept all threat models up to date
Actively and completely tested the threat model through penetrations testing
Accurately tracked and documented attack surfaces and any changes made to them during the development process
Completed security code reviews for all high-priority code identified and documented development and testing contacts for all code released with the product
Adhered to stringent coding standards
Rigorously brought all earlier code in the project up to current security standards
Validated the security documentation plan
The amount of time needed to complete the requirements of a security push is most influenced by the size and complexity of the product, coupled with how much effort has gone into creating the exit criteria requirements earlier in the development process.
There is no easy way to predict the duration of a security push. The push duration is ultimately determined by the amount of code that needs to be reviewed for security because all pushes to date have been gated by code quantity. Teams are strongly encouraged to conduct security code reviews throughout the development process, once the code is fairly stable, because the quality of code reviews will suffer if too many code reviews are condensed into too short of a time period during the push.
Again, the push is finished when the tasks are complete, not when an arbitrary date arrives.
The focus of a security push is primarily code-review of legacy code, but it also includes updating threat models and attack-surface documentation as well as pointed security testing and an end-user documentation scrub. The push allows the team to focus on nothing but security, and the length of time needed to complete a security push is task driven, not date driven: the security push is finished when the exit criteria have been met.
Make sure that time is built into the master schedule for the security push. It’s a relatively lengthy process, but we see these push durations diminishing over time as the older code is brought up to the quality dictated by current threats. If your threat models and attack-surface documentation are up to date before you enter the push and the amount of unreviewed former code is small, the security push duration will also be short.
The threat models and attack surface help drive the code-review priority. All code in the source-code tree should be assigned an owner. Then all high-priority code should be reviewed by someone other than the code owner. Bugs should be filed but not fixed at this point; fixing bugs is done after the push.