RMF Phase 4

Assess Security Controls

Abstract

This chapter introduces the fourth phase of the RMF, which is when the implemented security controls are assessed. Phase 4 includes developing an assessment strategy, assessing the controls, and producing the security assessment report (SAR).

Table of Contents

Chapter Overview and Key Learning Points

This chapter focuses on the assessment of the security controls that have been implemented and documented in the information system. The chapter begins with the creation of a security assessment plan that will be used to conduct the security controls assessment. The chapter continues explaining how to conduct an assessment and how to document the assessment in a security assessment report (SAR). The chapter concludes describing how the information system program office will remediate the deficient security controls and exercises that will re-enforce the information contained in the chapter.

Assessing Security Controls

This phase focuses on assessing, or testing, how each of the security controls is implemented in the information system or by the common control provider. There are four tasks that must be accomplished to successfully complete this phase: develop, review, and approve the test plan; assess the security controls; prepare the assessment report; and remediate deficient controls. Each of these sequential tasks leads to the next with the purpose of ensuring that the security controls are implemented as designed and are providing the required level of security. The status of the security control implementation is reported to the system owner for correction and to the authorizing official to assist in making an authorization decision.

Many organizations conduct security control assessments as an event at the end of the system design and development phase to obtain an approval to operate (ATO); it is more cost effective and efficient to conduct security control assessments in parallel with the system design process. Weaknesses and corrections can then be identified as early as possible in the system development.

Phase 4, Task 1: Security Control Assessment Plan

The security control assessor develops a detailed assessment plan to be used as a map for conducting the independent security controls assessment. This plan specifies which components of the system will be assessed, what automated tools or manual processes will be used, and what the boundaries of the test will be. The test plan should also include agreed-upon rules of engagement (ROE) that have been approved by the information system owner and authorizing official. These REO should define the scope and depth of the security assessment, points of contact (POC) for events or incidents that occur during the test event, acceptable tools and techniques approved for conducting the assessment, and levels of access required for completing a successful assessment or test event. The test plan is developed from the information system’s existing body of evidence (BOE) that is presented by the information systems owner to the security control assessor, including the systems security plan (SSP), engineering, architectural and policy documents, user and administrator guides, and previous test plans and test results. A complete BOE at this point ensures that the assessor understands the information system as it has been built, in the required level of detail that will allow the SCA to develop a test plan that adequately tests the system’s security posture and the system’s implementation of the required security controls. Controls that have been assessed and verified by an assessor that has been deemed independent by the authorizing official (during earlier steps in the RMF) to be providing sufficient protections and compliance for the system or common control provider will not need to be reassessed; the initial independent assessment results can be used as proof of compliance with that control’s requirement. In cases where the assessor was not deemed independent, the assessment plan and results can be used by the independent control assessor to form a baseline of the information system’s compliance to speed up the assessment, if the previous test plan is deemed adequate. If the previous plan is not adequate, it will need to be enhanced, rewritten, or developed from scratch by the independent control assessor.

The assessment plan provides the objectives for the independent security control assessors’ test event. This blueprint for the control assessment identifies who will be conducting the test event and the procedures that will be used to validate that the security controls are in place and working as designed to provide security and compliance. A fully developed plan serves two purposes for the organization. First, it establishes the appropriate expectations for the security control assessment by defining the procedures that will be used to assess the system or control provider, as well as the boundaries and scope of the test. Next, it binds the assessors to a defined level of effort, ensuring that resources are not squandered in overly complex testing, and results in the correct assessment of the security controls requirements. In some cases, a complex test strategy is needed based on the criticality of the system or the information it contains. In all cases, the level of complexity and detail in the security control assessment should be commiserate with the criticality of the system or common control set.

There are three major types of assessments used to test the security controls required to be implemented in the system: the developmental test and evaluation assessment, the independent verification and validation (IV&V), and a third type, one that supports any of the following: assessments supporting authorization or reauthorization; the continuous monitoring assessment; and remediation, or regression, assessments. An independent assessment team, as defined by the authorizing official, is required for authorization or reauthorization assessments. This level of independence is also needed for IV&V assessments, including assessments designed to have their findings used by IV&V or assessment/reassessment testing, such as tests conducted early in the system’s development. Often, it is more advantageous to have a dedicated team of independent assessors assigned to conduct all of these types of assessments to ensure that the results can be used to support authorization and reauthorization events as well as IV&V. Independence implies that the assessor is free from any actual or perceived conflicts of interest with respect to the design and development of the system’s security controls. To achieve this level of independence, the authorizing official may look to an organization that is separated from the design and development team, including other organizations or a contracted team, for independent and security control assessments. If contracted resources are used, it is important that the system owner is not directly involved in the contracting process to ensure independence of the control assessors.

The security test plan should define the types of manual and automated tools that will be used in the test event; maximum effectiveness can be obtained by utilizing automated tests and test procedures when conducting security control assessments, whenever possible. When using an automated assessment tool or testing application to validate the control, the test plan should indicate the settings, profiles, plugins, and other configuration settings for the automated tool used. There are many cases where automated tools cannot assess a security control or will need to be reinforced by manual processes and procedures. In these cases, the manual procedures need to be fully documented, including steps taken, commands entered, and menu items selected. Detailed documentation ensures that the test can be accurately repeated, and in the case of reciprocity, it allows individuals inspecting the test plan as part of the body of evidence to understand how each control was assessed.

In addition to documenting the steps taken to complete the specific test task for each security control, it is important to document the expected result of the test event. This is usually listed as a portion of the tested event for each control, typically after the detailed test steps. This documentation allows the SCA and other interested parties to know the expected output from the system or device, which will result in successfully passing that specific test event. In some cases, there are multiple results that are acceptable; for these events, it may be more prudent to list outcomes that would result in failing the test event; then the test plan should indicate that any result except those listed indicates the control is implemented correctly.

In addition to being independent the security control assessor should have a good deal of technical expertise in the implementation and assessment of technical, operational and management controls. These three types of controls can be implemented as common controls, be system specific controls or implemented as hybrid controls, based on the way the system was developed and engineered. For this reason the assessor must have experience in assessing a wide range of control implementations in a number of different environments and technologies.

When developing the security control test plan, the SCA should reference NIST SP 800-53A often, as this document defines the way the control must be evaluated. SP 800-53A is often referred to by the Greek term for the character A, the “alpha.” The alpha defines three different ways to evaluate a security control: examine, interview, and test. NIST defines these three events in SP 800-53A as follows:

Generally, test events are technical; however, some test events involve reviewing documentation, while some examine and interview tasks require a form of technical evaluation—most commonly system settings review or output. While not an official definition, a general rule is that examination events focus on reviewing documentation or system output, interview events focus on talking with different individuals, and test events are technical evaluations. The SCA will use these requirements to ensure that the test plan they are developing addresses the specific way the control should be assessed. The alpha also contains three appendices that may be helpful at this point. Appendix D defines assessment methods, applicable objects, and attributes; appendix G explains how to develop a SAR and a SAR template; and appendix G details how to create assessment cases for a test plan, as well as examples of assessment cases that can be used as templates.

Determining the depth and coverage of testing is important when developing a security controls assessment or test plan, as the assessment or plan sets the level of effort used for testing each control. These factors define the rigor and scope of the testing required for each specific control and is hierarchical, providing increased assessment requirements for the increased assurances needed for some information systems. The depth of the assessment determines the level of detail required for complete testing of the security controls and can be one of three attributes values: basic, focused, or comprehensive. Coverage defines the scope or breadth of the assessment and includes the same attributes of basic, focused, and comprehensive. These depth and coverage attributes are assigned by the organization when defining an organization-wide risk management program that will support the RMF. Normally, as the assurance requirements for information systems increase, the requirements for the scope and rigor of the security controls assessment increase, as well. Appendix D of NIST SP 800-53A defines the depth and coverage requirements for each of the assessment methods of examine, interview, and test. Portions of this document have been reproduced in Appendix D for convenience; however, before completing the development of the assessment plan, the security controls assessor should verify that this information is up to date; the latest approved version is on the NIST website.

Armed with this information, the SCA can develop the test plan. Using SP 800-53A, the SCA can identify the methods of assessment that will be used for each test event, for each control. As an example, the assessment objective for AC-3(6), Access Enforcement, which is listed in SP 800-53A, is quoted here:

This is an enhancement of the main security control AC-3. It verifies two key enhancements that must be implemented in systems where this control is mandated. As indicated by assessment objective (i), the enhancement verifies that the organization has identified the system information to be encrypted or stored off-line, and as indicated by part (ii), that the organization does in fact encrypt or store this information off-line,. The assessment methods and objects define what is required to assess these two components. In this case, all three methods—examine, interview and test—are required for a complete assessment of the enhancement.

When developing the security controls assessment plan, the security control assessor first develops a method to examine the system. The SCA evaluates the methods in the “select from” section of the examine method. The SCA can examine any or all of the following documents: access control policy, procedures addressing access enforcement, information system design documentation, information system configuration settings and associated documentation, information system audit records, and other relevant documents or records. The assessor evaluates the test method defined for this enhancement, then builds a test plan to assess the automated mechanisms, implementing access enforcement functions. Next, the assessor reviews the system’s submitted BOE to determine the technologies that will be used to implement either encryption or off-line storage for the required information types, including test procedure output or expected assessment results. Finally, the assessor notes that this enhancement has an interview requirement so that, during the assessment, the interview portion is conducted. While not required, some assessors develop a script to conduct the interview and others only use the results from the assessment’s other methods (examine and test) to determine the interview questions in an ad hoc manner during the test.

In either case, the results of the interview should support the identification and evaluation of a system with a correct implementation of this enhancement, in line with the documented requirements uncovered in the examine component and the test results. This ensures that the system’s supporting staff understand the requirements for this enhancement as defined by the organization and system owners. If the assessment results in a failure of any of the methods, the control will be listed as a failure or partial-pass/partial-fail, depending on the assessment’s rules of engagement.

The output of this task is a fully developed, reviewed, and approved test plan that defines the security control assessors, the assessment process, and the boundaries of the assessment, and includes approved ROE. The independence of the assessment team is defined, and in certain cases, independent assessors, as determined by the authorizing official, are required to be used. The test plan, once developed and reviewed, is approved by the AO.

A section of an example security control assessment plan is included in Appendix G

Phase 4, Task 2: Security Control Assessment

The test plan that was developed, reviewed, and approved in task 1 of this phase is the critical input document in task 2. Without it, effective and accurate assessment of the information system’s required security controls can’t be completed. The independent security control assessor uses this test plan to conduct the assessment of the security controls. This assessment determines the effectiveness of the security controls and ensures that they are implemented correctly and providing the desired level of security protection as well as required compliance with policy, regulation, and law.

During the security controls assessment, a test director should be identified. This individual should have a full understanding of the assessment’s rules of engagement and be able to guide and direct members of the assessment team. The test director serves as a liaison between various members of the information systems development team, the authorizing official and the AO’s office, the CIO’s office, and other stakeholders. Once the assessment is complete, the test director is responsible for developing the security assessment report (SAR) and presenting it to the authorizing official.

Once the security control assessment task begins, it is important that no changes to the information system occur. This practice of freezing the system’s configuration ensures that the system can be evaluated as accurately as possible at a single point in time. If this does not occur, it would be possible for the changes made to the system to impact the validity of the information system’s security control posture, requiring security controls that had already been assessed to require reassessment. While making changes may improve the capabilities or even security posture of the information system, these changes invalidate any testing done on the system, as controls that may have passed the assessment to this point are now in question, as the information system change could have modified the controls’ required and assessed settings. For this reason, the restrictions on making changes to the information system during security control testing should be communicated to all information system users and stakeholders.

The use of automated tools in this phase should be maximized, as defined by the test plan, and enhanced by manual checks where applicable to increase accuracy, efficiency, repeatability, and cost effectiveness. Automated tools used in this phase of the assessment can also be used in ongoing, or continuous, monitoring of the required security controls. In cases where automated tools are not available to validate that the security controls are in place, the security assessor uses various manual checks following the procedures outlined in the assessment or test plan.

As stated earlier, initial security assessments should be conducted as early as possible in the system development to ensure that security control integration and corrective actions are done in the most efficient and cost-effective manner. If these assessments are conducted by independent assessors, those controls determined to be in place and effective will not require reevaluation in IV&V or authorization/reauthorization assessments, based on review of the test plan and documented results. Authorization and reauthorization testing is a specific type of testing that requires the security control assessors be validated as independent by the authorizing official. This assessment is conducted after the system has been developed and user functional testing has been completed, to ensure user requests for features do not change the information system in a way that impacts the security posture of the system or the implementation of the required security controls. Successful authorization or reauthorization testing can lead to a system receiving an authorization to operate from the organization’s authorizing official. If authorized, the authorizing official accepts the risk of placing the system or common control into the production environment.

In almost all cases, incremental security control assessments throughout the information system’s development are effective ways to increase the speed, efficiency, and effectiveness of the security assessment. Incremental assessments allow parts of the system to be assessed by available independent assessors as the system components are completed and become available. Many times, organizational policies, procedures, and standards are developed long before the system’s development is complete. Controls that are satisfied by these policies, procedures, and standards can be assessed by an independent assessor before the official authorization test begins. This is true for any components that are complete and can be assessed before the official test event begins. The test director evaluates each of the controls that has been assessed as compliant by independent security control assessors before the start of the test event to ensure that modifications during the system’s development did not change the status of the security control. Finally, the systems security plan, developed and approved in phase 2, can be reviewed and any controls that are satisfied by the SSP can be noted as compliant. If, however, the SSP changes before the approval security control assessment begins, these controls need to be reassessed. This way, a number of controls can be assessed before the official test begins, allowing the assessors to focus on specific, and often technical, controls implemented in the information system. This assessment, paired with the establishment of organizational common control providers that offer approved controls for inheritance, dramatically reduces the number of controls and enhancements that need to be assessed during the official test event.

Before the control assessment begins, the organization must ensure that the assessors have the required access to both the system and the system’s documentation. They must be provided with information on the system’s operating environment, where each of the required security controls are implemented, and required documentation such as system artifacts and records; test results; architectural documents that contain information flows, network diagrams and access points; and any other documentation needed to accurately assess the security controls required by the SCTM. These documents should be provided to the security assessor as early as possible in the development cycle while ensuring that they are complete and accurate.

The authorizing official and information owner rely on the expertise of the security control assessors to evaluate the system against documented requirements of regulations, laws, and organizational policy, as indicated by published documentation such as the systems security plan or the common control provider’s security plan. The assessors evaluate the common controls inherited by the system to ensure that they are authorized for inheritance and provide the levels of protection needed by the system. Common controls that are not part of a common control set, and do not have a valid authorization from the appropriate authorizing official, cannot be inherited and must be implemented by each information system owner.

Independent and competent security control assessors ensure that the authorizing official and system owner have an accurate picture of how the system’s security controls are implemented and, by extension, have a picture of the security status of the system. Complete and accurate documentation in this phase ensures that the test results can support reuse and reciprocity. Reciprocity is enhanced by complete documentation of the system, documentation of the process used to assess the required security controls, and the results of this assessment.

Phase 4, Task 3: Security Assessment Report

The security assessment report, or SAR, is one of the three key required documents for a system, or common control set, authorization package. The SAR accurately reflects the results of the security control assessment for the authorizing official and system owner. This document is also extensively used for determining reciprocity of the system’s authorization—assuming it is granted—by other organizations. This document describes the effectiveness of the security controls implemented by the system and identifies controls that are not implemented, functioning as required, or are not providing an adequate level of protection for the system or organization. The SAR is critical in determining the level of risk that will be introduced to the organization if the system, or common control set, is placed into production. The risk executive (function) and authorizing official use the SAR to determine how the resultant risks to the organization may impact it if the system is approved to operate in the organization’s production environment.

Results from interim security assessment reports obtained during system development or incremental assessments can be brought forward and included in the final authorization SAR. Once the system is authorized, it enters a state of continuous monitoring, discussed in more detail later. The controls continue to be assessed and the SAR is updated based on the results of these assessments. According to NIST, the SAR should, at a minimum, contain the following items:

• Information system name

• Security categorization

• Site(s) assessed and assessment date(s)

• Assessor’s name/identification

• Previous assessment results (if reused)

• Security control or control enhancement designator

• Selected assessment methods and objects

• Depth and coverage attributes values

• Assessment finding summary (indicating satisfied or other than satisfied)

• Assessor comments (weaknesses or deficiencies noted)

• Assessor recommendations (priorities, remediation, corrective actions, or improvements)

Many organizations develop an executive summary of the SAR and include it in the beginning of the document or in a separate document. The executive summary highlights and summarizes the assessment results, providing key information and recommendations based on the weaknesses discovered during the assessment. The authorizing official can use this summary to quickly understand the security status of the system and use the detailed SAR to provide full details for those items that require a more detailed explanation. Appendix G includes examples of components of the security assessment report.

Phase 4, Task 4: Remediation Actions

The system owner uses the SAR to develop a plan to remediate those weaknesses and deficiencies discovered during the security controls assessment. These flaws are often the result of a failure to implement a control, misconfigurations, or inadequate implementation of the required security controls during system development for a number of reasons. Some findings may be deemed inconsequential by the system owner or common control provider after consulting with appropriate organizational officials, including the information systems security officer, the authorizing official, the chief information officer, the senior information security officer, and the information owner. Identification of controls that are deemed inconsequential, and the supporting rational, are documented in the systems security plan.

Findings that are not determined to be inconsequential are prioritized by the security impact they have, or may have, to the overall organization. Some items may be significantly important and vulnerable enough that they may require immediate remediation before the system can be placed into production operation. The SAR, including the prioritization of the findings, is used by the risk executive (function) to update the assessment of risk for the system and the organization. The risk executive (function) can also assist in determining the initial prioritization of findings to support remediation actions. This ensures that the organization’s resources are assigned to the items that have the greatest impact to the overall organization, not just the system or common control set.

Once deficient controls are remediated, the independent assessor again tests each control during a remediation assessment. Controls that are implemented correctly and are providing the required level of protection after this assessment are noted in the SAR, ensuring that the original finding’s documentation is not altered or removed. It is important that the systems security plan is updated with details of the current state of the system, including the change to the security control. The systems security plan should, at this point, accurately detail how each security control is implemented, all compensating controls, and a listing of residual vulnerabilities.

In some organizations, the system or common control provider, develops an addendum to the SAR in response to the initial findings of the assessors. This addendum often details initial remediation actions enacted to respond to the assessor’s findings. The addendum does not, however, change or impact the findings documented in the SAR or change the SAR itself in any way.

Organizations may also develop a resolution process that helps system owners and common control providers to determine appropriate actions to take when addressing common control weaknesses and deficiencies discovered in the control assessment. This process can help with understanding vulnerabilities and the linked risk, as well as how to identify and resolve false positives. False positives are cases when assessment tools or manual processes inaccurately discover an issue that appears to be a vulnerability, weakness, or deficient control. This process is quite helpful to the authorizing official when making an informed authorization decision.

Chapter 12 Lab Exercises: Assessing Security Controls

The lab exercises in this chapter use NIST SP 800-53A, which is available on the book’s accompanying disk, downloadable at http://www.cyber-recon.com or at the NIST website, http://csrc.nist.gov/publications/PubsSPs.html. SP 800-53A provides guidance for developing test procedures and plans for assessing security controls and preparing security assessment reports (SAR).

1. When developing a test plan, a security control assessor (SCA) reviews 800-53A to develop test plans and assessment cases. The system’s security controls tractability matrix (SCTM) defines the controls that will be implemented by the system’s developers. One of these controls is MP-3, media marking. What type of assessment (examine, interview, or test) is required for this control? According to SP 800-53A, what are the assessment methods and objectives?

2. When conducting a security control assessment of the system, the SCA was able to review documentation that supports MP-3. The examination of these records met the requirements of the test plan as developed from SP 800-53A. Unfortunately, every administrator or support technical working the system was unable to answer the SCA’s interview questions. Would this control be documented in the SAR as compliant for this assessment?

3. The organization conducts re-training for the findings discovered in the SCA, including detailing requirements for media marking. The SCA was able to return after the SAR was published and again conducted the interview required for the control MP-3. Can this finding now be deleted from the SAR?