The purpose of Acquisition Technical Management (ATM) is to evaluate the supplier’s technical solution and to manage selected interfaces of that solution.
The Acquisition Technical Management process area focuses on the following:
• Conducting technical reviews of the supplier’s technical solution
• Analyzing the development and implementation of the supplier’s technical solution to confirm technical progress criteria or contractual requirements are satisfied
• Managing selected interfaces
Tip
Many problems encountered in the integration of product or service components or the product or service transition to operations is due to interface incompatibilities. Therefore, ATM covers interface management.
Typically, these activities interactively support one another to gauge technical progress and allow effective management of project technical risks. Different levels of detailed analysis, depending on the development progress and insight required, may be needed to conduct technical reviews to the acquirer’s satisfaction. Prototypes, simulations, and technology demonstrations created by the supplier may be used as a means of gaining knowledge to manage selected interfaces.
Hint
It is not enough to consider product functionality and behavior in the intended operational environment when evaluating a solution. Ask questions about other phases in the life of the product, including whether the solution can be manufactured; whether it is easy to test, install, repair, migrate to new versions or platforms, and support; and what the costs and legal implications will be.
In some acquisitions, the acquirer assumes the role of overall systems engineer, architect, or integrator for the product. In these acquisitions, the Technical Solution process area of CMMI-DEV should be used. Technical Solution in CMMI-DEV includes additional information about designing, developing, and implementing solutions, including the design approaches, design concepts, and alternative solutions for which an acquirer may have varying degrees of responsibility.
Acquisition Technical Management activities involve measuring technical progress and the effectiveness of plans and requirements. Activities include those associated with technical performance measurement and the conduct of technical reviews. A structured review process should demonstrate and confirm completion of required accomplishments and exit criteria as defined in project planning and technical plans (e.g., the Systems Engineering Management Plan). Acquisition Technical Management activities discover deficiencies or anomalies that often result in corrective action.
X-Ref
ATM is driven by the contractual requirements established by ARD, which are managed by REQM. The technical management in ATM has to coordinate with the supplier agreement management in AM. The processes associated with these process areas interact significantly to accomplish their purposes.
Acquisition Technical Management should be performed with other technical and agreement management processes, such as requirements management, risk management, configuration management, data management, and agreement management.
Refer to the Acquisition Requirements Development process area for more information about allocating requirements, establishing an operational concept, specifying technical performance measures, and defining interface requirements.
Refer to the Decision Analysis and Resolution process area for more information about establishing evaluation criteria and selecting alternatives based on those criteria.
Refer to the Requirements Management process area for more information about managing requirements.
Refer to the Risk Management process area for more information about managing risks.
Refer to the Configuration Management process area for more information about managing configuration items.
Refer to the Plan Data Management specific practice of the Project Planning process area for more information about managing data.
Refer to the Agreement Management process area for more information about managing the supplier agreement.
Supplier technical solutions are evaluated to confirm that contractual requirements continue to be met.
Technical reviews (or architectural evaluations) are performed throughout the project lifecycle to gain confidence that the requirements, architecture, and supplier technical solutions are capable of guiding a development that results in a product or service that provides the required capability. This activity should be integrated with risk management activities. Mature organizations typically perform technical reviews using different proven techniques depending on the type of review. They broaden the basis of the review to include other stakeholder needs, expectations, and constraints.
Refer to the Establish the Acquisition Strategy specific practice in the Project Planning process area for more information about specifying technical performance measures and their threshold values.
This specific goal focuses on the following:
• Selecting supplier technical solutions (i.e., preliminary designs, detailed designs, and design implementations) based on sound decision-making criteria
• Analyzing selected supplier technical solutions
• Conducting technical reviews using results of the analysis
Select supplier technical solutions to be analyzed and analysis methods to be used.
The supplier technical solutions are typically in one of the following three stages:
• Candidate solutions (i.e., design approaches,design concepts, or preliminary designs) that potentially satisfy an appropriate set of allocated requirements
• Detailed designs for selected solutions (i.e., containing all the information needed to manufacture, code, or otherwise implement the design as a product or product component)
• Implemented designs (i.e., the product or service)
Depending on where in the acquisition lifecycle the highest risks occur, the acquirer selects supplier technical solutions for analysis to reduce those risks. Analysis methods are selected based on the type of technical solution being analyzed.
Tip
It is best to evaluate a range of candidate solutions. Input from stakeholders with diverse skills and backgrounds can help teams identify and address assumptions, constraints, and biases that may be exhibited in the supplier’s candidate designs. Also, consider the “ilities” addressed in the contractual requirements (e.g., usability, reliability, maintainability, interoperability, and security).
When you consider the successful products or services you have acquired, ask what made them successful. Was it features, usability, cost, customer support, response time, reliability, and so forth? These characteristics were achieved through careful evaluation of technical solutions.
The acquirer may want to select interfaces for analysis to help decide which interfaces require acquirer management. (See specific goal 2 of this process area.)
Tip
Complexity is a “double-edged sword.” Sometimes today’s high-performance solution becomes tomorrow’s high-maintenance solution.
Typical Work Products
- Criteria used for selecting supplier technical solutions for analysis
- Lists of supplier technical solutions selected for analysis
- Analysis methods for each selected supplier solution
Typical Supplier Deliverables
- List of supplier deliverables
Subpractices
- Select criteria for determining which supplier technical solutions to analyze.
Refer to the Decision Analysis and Resolution process area for more information about establishing evaluation criteria used in making decisions.
- Identify supplier technical solutions for analysis.
Hint
When evaluating a supplier’s alternative solution, treat its components together, not individually. For example, do not rush to evaluate a promising COTS component or new technology without considering the impacts and risks.
- Identify the requirements to be satisfied by each selected technical solution.
A traceability matrix is a useful tool for identifying requirements for each selected technical solution, as it typically includes information that relates requirements to work products. When identifying requirements for each selected technical solution, consult the appropriate traceability matrix.
Refer to the Maintain Bidirectional Traceability of Requirements specific practice in the Requirements Management process area for more information about tracing requirements to work products.
Tip
Criteria for selection can include the impact of the technical solution on cost, schedule, performance, and risk. How these criteria are defined in detail, however, depends on the requirements.
Screening criteria may involve setting thresholds for selected quality attributes (e.g., response time) that must be met by technical solutions.
- Identify the analysis methods to be used for each selected technical solution.
- Include analysis methods and review activities in the project plan.
Refer to the Project Planning process area for more information about planning the technical aspects of the project.
Hint
Explore the use of COTS (or open source or new technology) by the supplier early in technical evaluations because to use COTS effectively, you may need to consider changes to requirements. Fully understand the tradeoffs of such requirements and designs early, before committing to (and putting under contract) a particular development approach.
Analyze selected supplier technical solutions.
Depending on the type of technical solution being analyzed (i.e., preliminary design, detailed design, or design implementation), the results of the analysis are provided to the technical review described in the next specific practice.
X-Ref
For more information regarding principles, methods, and techniques for creating systems from COTS products, see the SEI Web site at www.sei.cmu.edu/cbs/.
The acquirer should select a supplier’s design to analyze by exploring the adequacy and completeness of that design by reviewing product representations (e.g., prototypes, simulations, models, scenarios, and storyboards) and by obtaining feedback about them from relevant stakeholders.
The acquirer should confirm the following.
• The selected design adheres to applicable design standards and criteria.
• The design adheres to allocated requirements.
• The resulting product will perform appropriately in its intended-use environment.
Hint
To gain the insight needed to fully evaluate supplier solutions, you may need to refine operational concepts and scenarios to understand the implications of each alternative solution. Also, you may need to develop scenarios for other phases of the product lifecycle.
During design implementation, the supplier implements the design reviewed and analyzed by the acquirer by developing product components, integrating those components, conducting unit and integration testing of the product, and developing end-user documentation.
A successful analysis of the supplier’s implementation is predicated on the acquirer’s determination that the requirements are fully met in the final production configuration, that the production capability forms a satisfactory basis for proceeding into pilots or full-rate production, and that the product is ready to be brought into the acquirer environment for further integration and acceptance testing.
Hint
A design is a document used by stakeholders over the life of the product, and thus must communicate clearly and accommodate change. Consider this when selecting criteria to be used in evaluating the value of a design.
The acquirer may require delivery of verification results from the supplier of the technical solution, as applicable. The suppliers may conduct verifications in an iterative fashion, concurrently with the acquirer’s technical analyses, or the supplier may be required to conduct follow-on verifications of technical solutions.
Tip
Documentation can be treated as a type of product component for which a solution may be selected, designed, and implemented. There are design and implementation methods and standards for documentation.
The acquirer should also confirm that sufficient end-user documentation has been developed and is in alignment with the tested implementation. The supplier may develop preliminary versions of the installation, operations, and maintenance documentation in early phases of the project lifecycle for review by acquirer and relevant stakeholders.
Tip
Installers, operators, end users, and maintainers may have different documentation needs that may be addressed in different documents.
Documentation assists product maintenance and support later in the life of the product.
Typical Work Products
- Record of analysis
- Results of analysis
Typical Supplier Deliverables
- Alternative solutions
- Product architecture
- Product component designs
- Unit and integration test results
- Verification results
Subpractices
- Confirm that the selected technical solution adheres to applicable standards and criteria.
- Confirm that the selected technical solution adheres to allocated requirements.
- Use analysis results to compare actual performance measurements to specified thresholds of technical performance measures.
Refer to the Measurement and Analysis process area for more information about technical performance measures.
- Conduct technical interchange meetings as necessary.
Technical interchange meetings are scheduled meetings between the supplier and acquirer to discuss technical progress. These meetings are less formal than the event-driven technical reviews in the next specific practice.
- Confirm that the selected technical solution is sufficiently analyzed and meets entrance criteria to begin technical review.
- Review critical verification results and data from verifications conducted by the supplier.
Hint
No solution will rank best for every criterion used. Instead, analyze a solution’s ability to provide a balanced approach to cost, schedule, performance, and risks across the product lifecycle.
Following the evaluation, you may conclude that the selection criteria were not complete or detailed enough to adequately assess the ability of the solution to meet the entrance criteria for the technical review. If so, you may need to iterate through SP 1.1 and 1.2.
Conduct technical reviews with the supplier as defined in the supplier agreement.
Technical reviews are used by the acquirer to confirm that products and services being developed or produced by suppliers meet user needs and requirements.
Technical reviews should have the following characteristics:
• Conducted when the technical solution under development satisfies review entry criteria (i.e., event-driven, not schedule-driven)
• At a minimum, conducted at the transition from one acquisition phase to the next and at major transition points of technical effort
• Have their processes and requirements addressed in and required by the supplier agreement
Tip
The purpose of a technical review is to review the supplier’s technical progress and identify and resolve issues.
Refer to the Project Planning process area for more information about planning the technical aspects of the project.
Typical Work Products
- Review schedule
- Entry and exit criteria
- Review results
- Documented issues (e.g., issues with customer requirements, product and product component requirements, product architecture, and product design)
Tip
Involving relevant stakeholders (engineers, technical writers, QA personnel, etc.) in evaluating a supplier’s technical solution can reduce the number of serious issues that must be resolved by participating in reviews of the supplier’s technical solutions. In these reviews, issues affecting installation, operation, and so on can be identified and resolved.
Typical Supplier Deliverables
- Progress reports and process, product, and service-level measurements
- Technical performance measurements
- Review materials and reports
- Action items tracked to closure
- Documentation of product and document deliveries
Hint
You may want to examine the rationale for a particular technical review result when you later learn that a promising technology or COTS component is now available. It may be unnecessary to interrupt product development to explore the implications if they were already explored earlier and records were maintained.
Subpractices
- Identify participants for the technical review.
- Conduct the technical review.
- Analyze and record results of the review.
- Use the results of technical reviews to improve the supplier’s technical solution.
The results of some reviews may require changes to the supplier agreement.
Refer to the Solicitation and Supplier Agreement Development process area for more information about establishing and maintaining the supplier agreement.
Tip
Overlooked or incompletely described interfaces are risks to be identified and managed (RSKM). The realization of these risks may lead to safety recalls and can prove very costly.
Interface management may sometimes follow a formal evaluation (DAR) process: Establish criteria for correctness of an interface (based in part on interface requirements), evaluate alternatives, and so on.
Selected interfaces are managed.
Many integration and transition problems arise from unknown or uncontrolled aspects of both internal and external interfaces. Effective management of interface requirements, specifications, and designs helps to ensure implemented interfaces are complete and compatible.
The supplier is responsible for managing the interfaces of the product or service it is developing. However, the acquirer identifies those interfaces, particularly external interfaces, that it will manage as well.
Hint
Where possible, reach early agreement with the supplier on interface design. For some external interfaces, agreement must also be reached by the acquirers or owners of the other interfacing products or services. Lack of agreement leads to wasted time, as assumptions made by the teams on each side of an interface must be validated. Project costs may increase and the schedule may slip.
Select interfaces to manage.
The interfaces considered for selection include all interfaces with other products and services in the operations and support environment as well as environments for verification and validation and services that support those environments. The acquirer should review all supplier interface data for completeness to substantiate the complete coverage of all interfaces when making the selection.
Tip
You can use a formal evaluation (DAR) process to select the interfaces to manage. Criteria for selection can include the impact of the interface on cost, schedule, performance, and risk.
Typical Work Products
- Criteria to be used in selecting acquirer-managed interfaces
- Categories of interfaces
- List of interfaces per category
Typical Supplier Deliverables
- Interface description documents
- Categories of interfaces
- List of interfaces per category
- Mapping of interfaces to product components and the product integration environment
- Interface design specifications
- Interface control documents
- Interface specification criteria
Tip
Interface control documents define interfaces in terms of data items passed, protocols used for interaction, and so on. These documents are particularly useful in controlling products and product components being built by different teams.
- Select the criteria to be used for determining which interfaces the acquirer should manage.
Refer to the Decision Analysis and Resolution process area for more information about establishing evaluation criteria used in making decisions.
- Identify interfaces that are candidates for acquirer management.
X-Ref
It may be prudent to mitigate risks by establishing integrated teams across related acquisition programs when final capabilities require effective interfaces between or among programs. IPM SP 1.6 provides guidance for such teams.
- Review identified interfaces against the selection criteria.
- Include acquirer-managed interfaces in the project plan.
Manage selected interfaces.
Managing interfaces includes the maintenance of the consistency of the interfaces throughout the life of the product and the resolution of conflict, noncompliance, and change issues. In a system of systems environment, the management of interfaces between products or services acquired from suppliers and other systems within the system of systems is critical for success of the project.
Hint
To know which interfaces to manage, the acquirer should include in the supplier agreement a relationship table, prepared by the supplier, which identifies interfaces among product components (and among product components and the environment).
The acquirer should also prepare a relationship table for the external interfaces if the acquired product or service will become part of a system of systems. The ultimate objective is to achieve coverage of all interfaces and ensure completeness.
Refer to the Acquisition Requirements Development process area for more information about establishing and maintaining interface requirements.
Refer to the Requirements Management process area for more information about managing changes to interface requirements.
Refer to the Configuration Management process area for more information about managing changes to interface descriptions (i.e., specifications) as configuration items so that stakeholders can be made aware of the current state of the interfaces.
Interface changes are documented, maintained, and readily accessible.
Tip
An API is the interface that a computer system, library, or application provides to allow other computer programs to make requests for service and/or exchange data.
- Table of relationships among the supplier’s product or service and the external environment
- Updated interface description or agreement
Tip
A repository for interface data provides access to interface descriptions so that deviations from these definitions are less likely.
Typical Supplier Deliverables
- Table of relationships among the product components and the external environment (e.g., main power supply, fastening product, and computer bus system)
- Reports from interface control working group meetings
- Action items for updating interfaces
- Application program interface (API)
X-Ref
Interface descriptions are typically placed under configuration management (see GP 2.6) so that changes in status are recorded and communicated (see CM SP 3.1).
Subpractices
- Review and analyze selected interface definitions and designs.
- Confirm that interface descriptions adhere to allocated requirements.
- Confirm the compatibility of selected interfaces throughout the life of the product or service.
Confirm that interface descriptions adhere to applicable standards, criteria, and interface requirements between the supplier’s product and acquirer’s intended environment.
- Verify that interfaces have been sufficiently tested by the supplier.
- Verify that issues identified during testing have been resolved appropriately, with product revisions, if necessary.
- Resolve conflict, noncompliance, and change issues for the selected interfaces.
- Periodically review the adequacy of interface descriptions.
Once established, interface descriptions must be periodically reviewed to ensure there is no deviation between existing descriptions and the products being developed, processed, produced, or bought.
The interface descriptions should be reviewed with relevant stakeholders to avoid misinterpretations, reduce delays, and prevent the development of interfaces that do not work properly.
Tip
These tests may include tests for interface compatibility, tests of how well the assembled product components interoperate, and tests involving end users.
Interfaces with environments such as product integration or assembly, verification, and validation environments, as well as operational, maintenance, and support environments, should be addressed.