Chapter 3. The Role of IT Governance

IT governance is a subset of corporate governance that focuses on the belief that the managers, directors, and others in charge of an organization must establish key roles and responsibilities to control IT risks. Management must implement rules and policies to control the IT infrastructure and develop practices to distribute responsibilities. Not only does this prevent a single person or department from shouldering responsibility, it also sets up a framework of control.

IT governance is established by creating an IT strategy committee, developing policies and procedures, defining job roles, executing good HR practices, and performing risk assessments and periodic audits. This chapter discusses each of these topics.

This chapter discusses IT governance, which involves control, including items that are strategic in nature. Senior management and the IT steering committee help provide the long-term vision so that control can be implemented on a more tactical level.

1. Which of the following is not a typical IT steering committee group or member?

a. Business management

b. Chief technology officer

c. Human resources

d. Chief information officer

2. Which of the following is a funding strategy related to paying for an information system’s services and requires individual departments to directly be charged for the specific services they use?

a. Shared cost

b. Guarantor cost

c. Chargeback

d. Sponsor pays

3. Which of the following IT governance frameworks is best suited to define the quality, control, and reliability of information systems by establishing IT governance and management structure and objectives?

a. ITIL

b. COBIT

c. COSO

d. All the above

4. Which role is typically not aligned to the Three Lines of Defense model?

a. End user

b. Business unit leadership

c. Risk and compliance teams

d. Auditor

5. Which category would be the best fit to classify data-related IT process documentation in a simple data classification model?

a. Public

b. Business confidential

c. Customer confidential

d. Proprietary

6. Which of the following control(s) should be used during the hiring process?

a. Confidentiality agreements

b. Non-compete agreements

c. Background check

d. All the above

7. Which of the following key performance terms refers to how well a process is performing?

a. Balanced scorecard

b. Target value

c. Key performance indicator

d. Key goal indicator

8. Consider the following list of management and control framework examples. Which of them provides guidance on how to assess and improve the ability to prevent, detect, and respond to cyberattacks?

a. COBIT

b. COSO

c. NIST CSF

d. ISO

9. Which of the following capability maturity model levels best fits an ad hoc process with no documentation?

a. Initial

b. Repeatable

c. Defined

d. Optimized

10. Which of the following laws requires accurate financial and accounting disclosure for U.S. public companies?

a. U.S. Federal Information Security Management Act (FISMA)

b. U.S. Fair and Accurate Credit Transaction ACT of 2003 (FACTA)

c. U.S. Health Insurance Portability and Accountability Act (HIPAA)

d. Sarbanes-Oxley Act (SOX)

11. During which stage of the Plan-Do-Check-Act (PDCA) approach does the auditor measure actual process outcome against objectives?

a. Plan

b. Do

c. Check

d. Act

12. Which location would not be best suited for third-party outsourcing?

a. Corporate HQ functions

b. Data center functions

c. Help desk functions

d. Payroll processing functions

The exact makeup of the IT steering committee will vary by organization based on size, industry, regulatory mandates, and leadership strength. In general, the IT steering committee needs to be made up of senior leaders from IT, corporate functions, and lines of business. The following are typical IT steering committee members:

Business management: The committee is managed by the chief executive officer (CEO) or by another person who is appointed, such as the chief information officer (CIO).

IT management: IT management is represented by the CIO or a CIO representative.

Legal: The legal group is represented by an executive from the legal department.

Finance: A representative from finance is needed to provide financial guidance.

Marketing: A representative from marketing should also be on the committee.

Sales: A senior manager for sales should be on the committee to make sure the organization has the technology needed to convert shoppers into buyers.

Quality control: Quality control ensures that consumers view products and services favorably and that products meet required standards. Therefore, quality control should be represented on the committee.

Research and development (R&D): Because R&D focuses on developing new products, this department should be represented on the committee. IT must meet the needs of new product development.

Human resources (HR): Managing employees is as complex as the technology needed to be successful. HR should be represented on the committee.

Notice that the IT steering committee does not typically consist of technologists such as the chief technology officer (CTO) because this is primarily viewed as a business committee. The chief information officer (CIO) typically is a member and acts as the bridge between the IT steering committee and the technology department. The IT steering meeting provides an opportunity for exchange of views where the business communicates its business goals and IT discusses how it can align and enable the business’ goals through the use of technology. Often this includes IT leadership educating the business on the limits and risks of technology.

Once an understanding is reached between the business and IT leadership on goals, the CTO and other technologies engage in implementation planning. Although membership might vary, the goal of the committee should be consistent. In additional to goal setting, the committee is responsible for reviewing major IT projects, budgets, and plans.

The duties and responsibilities of the IT steering committee should be defined in a formal charter. If an organization lacks a charter or doesn’t have a steering committee, this should be a clear warning that IT and the business may not be closely aligned. Although the charter gives the committee the power to provide strategic guidance, it should not be involved in the day-to-day activities of the IT department. Evidence that indicates otherwise should alert auditors that the committee has strayed from its charter or that the charter is not clear on the committee’s responsibilities. A steering committee is just one of three items needed to build a framework of success. The other two are performance measurement and risk management.

The IT steering committee is a good place to start understanding the separation between governance and management. Senior management’s role in the IT steering committee process is at a strategic level, not a tactical one. Consider eBay, for example. Although eBay’s senior management is very concerned about merchandise being listed for the duration of an auction and about bidding and closing occurring seamlessly, they should have little concern about the operating system and platform. As long as the technology can meet the stated business goal and budget constraints, the choice of Windows, Linux, or UNIX should be left up to the IT department. Senior management’s goal is to ensure that goals are aligned, IT is tasked with meeting those business needs, and the auditor is responsible for ensuring that controls are present and operating effectively.

Answers to the “Do I Know This Already?” Quiz:

1. B;

2. C;

3. B;

4. A;

5. D;

6. D;

7. C;

8. C;

9. A;

10. D;

11. C;

12. A

Shared cost: With this method, all departments in the organization share the cost. The advantage of this method is that it is relatively easy to implement and for accounting to handle. Its disadvantage is that some departments might feel that they are paying for something they do not use.

Chargeback: With this method, individual departments are directly charged for the services they use. This is a type of pay-as-you-go system. Proponents of this system believe that it shifts costs to the users of services. Those opposing the chargeback system believe that it is not so clear-cut. For example, what if your city of 1,000 people decided to divide electrical bills evenly so that everyone pays the same? Many might complain, as not everyone uses the same amount of electricity. Opponents of the chargeback system make the same argument, as end users don’t consume IT resources evenly.

Sponsor pays: With this method, project sponsors pay all costs. Therefore, if sales asks for a new system to be implemented, sales is responsible for paying the bills. Although this gives the sponsor more control over the project, it might lead to the feeling that some departments are getting a free ride, which can cause conflicts.

Align the goals of IT to the goals of the organization: Both must be focused on and working for the common goal.

Establish accountability: Accountability requires that individuals be held responsible for their actions. Accountability can be seen as a pyramid of responsibility that starts with the lowest level of employees and builds up to top management.

Define supporting policies and processes: It is important to establish the rules of the road and expected behavior.

Not all IT governance frameworks are created equal. Each IT governance framework was designed to meet a specific industry or regulatory guidance need. While many frameworks overlap to some degree, they are also often complementary, building on the strengths and weaknesses of others. Because of this natural synergy, many organizations adopt multiple governance frameworks. Let’s examine this synergy by examining COBIT and ITIL.

COBIT essentially defines what is needed to achieve the organization’s goals and defines the high-level organizational structure and control requirements needed to reduce IT risks.

COBIT 5 is the newest version of COBIT, released in 2012. It outlines five core governance principles:

1. Meeting stakeholder needs

2. Covering the enterprise end to end

3. Applying a single integrated framework

4. Enabling a holistic approach

5. Separating governance from management

COBIT 5 describes these principles in terms of enabler requirements that support an enterprise in meeting stakeholder needs related to the use of IT assets and resources across the enterprise. There is a significant emphasis on governance, responsibilities, and accountability. COBIT requires management to understand and manage the business risk.

There are two types of processes in COBIT 5: governance processes (evaluate, direct, and monitor) and management processes (plan, build, run, and monitor). COBIT 5 is a broad framework that can be applied to any industry to organizations of all sizes.

ITIL essentially defines how to achieve the organization’s goals and defines the low-level organizational structure and process requirements needed to reduce IT risks.

ITIL provides a set of interrelated best practices that provide detailed guidance for developing, delivering, and managing enterprise IT services. There are five stages in the ITIL service life cycle:

1. Service strategy

2. Service design

3. Service transition

4. Service operation

5. Continual service improvement

Think of COBIT as defining the coaching staff for an NFL team. In this analogy, COBIT would define the need for an offensive coordinator and a defensive coordinator. COBIT would define their roles and accountabilities, what type of records they should keep, how often the player health should be checked, and so on. What’s missing? The plays—the specific drill routines and much more. Whereas COBIT talks about the running of the team, ITIL talks about all the details of how to win each game.

The NFL team analogy illustrates the synergy between the frameworks; without both a well-run team and effective execution on the field, the team cannot win games.

Governance frameworks can and do overlap. Typically, they overlap in how they define certain functions or placement of those functions within an organization. For example, ITIL calls out IT risk management as a unique topic and chooses to integrate the practice across its services. COBIT calls out both IT risk management as a topic with separate and unique process requirements for its management. While accommodation may be needed so they coexist, in the end they are complementary. In fact, ITIL provides detailed advice on how to carry out several COBIT processes. Change management is an example where ITIL can define a structure and a process to achieve COBIT control objectives.

Ultimately, IT governance frameworks are often adjusted to accommodate the organizational, industry, and technology environment in which they are to be implemented. These accommodations make each IT governance framework implementation unique.

While auditors may have a firm grasp on any framework at an academic level, they need to understand the accommodations made before they can effectively audit the environment. Here is a high-level list of what an auditor needs to consider as part of an IT governance framework audit:

Familiarize yourself with the implemented frameworks.

Understand the business goals and objectives from the IT steering committee.

Focus on the strengths and weaknesses of each of the applicable frameworks to ensure coverage of goals and business objectives.

Ensure that accommodations between frameworks have not resulted in conflicting definition or redundant processes.

Ensure that measurement systems are complementary.

The first step in the risk management process is to identify and classify the organization’s assets. Information and systems must be assessed to determine their worth. When asset identification and valuation are complete, the organization can start the risk-identification process to identify potential risks and threats to the organization’s assets.

A risk management team is tasked with identifying these threats. The team can then examine the impact of the identified threats. This process can be based on real monetary amounts or a reasonable estimate based on experience.

Chapter 2, “The Information Systems Audit,” discusses types of threats and how to manage the associated risks. It covers recognizing different types of risk: inherent, control, detection, and residual. Chapter 2 also discusses the fact that, after identifying high-risk, high-impact concerns, the risk management team can move on to the risk mitigation or risk disposition phase. Risk can be disposed of in the following ways:

Avoiding risk

Reducing risk

Accepting risk

Transferring risk

The same tools and methods discussed in Chapter 2 also apply to ERM. The difference is that ERM applies these tools to the entire end-to-end population of risk. For example, consider weather forecasting. Every day we can use tools to measure the weather. But there is also value in looking at the pattern of weather for the month, year, decade, and century. ERM is the processes that take the aggregate view of risk.

Teams of specialists may be formed to address emerging or high-profile risks. These teams are not created in a void but are developed within a risk management program with a purpose. For example, a program might be developed to look at ways to decrease insurance costs, reduce attacks against the company’s website, or verify compliance with privacy laws.

After the purpose of the team is established, the team can be assigned responsibility for developing, modifying, and/or implementing a more comprehensive risk management program. This is a huge responsibility because it requires not only identification of risk but also implementation of the team’s recommendations.

Hardware

Software

Employees

Services

Reputation

Documentation

When looking at an asset, the team must first think about the replacement cost of the item before assigning its value. The team should consider the value brought by an asset more than just the cost to create or purchase it. These considerations are key:

What did the asset cost to acquire or create?

What is the liability if the asset is compromised?

What is the production cost if the asset is made unavailable?

What is the value of the asset to competitors and foreign governments?

How critical is the asset, and how would its loss affect the company?

Business owners and senior managers

Legal counsel

HR representatives

IS auditors

Network administrators

Security administrators

Operations

Facility records

Government records and watchdog groups, such as CERT

A threat is any circumstance or event that has the potential to negatively impact an asset by means of unauthorized access, destruction, disclosure, or modification. Identifying all potential threats is a huge responsibility. A somewhat easier approach is to categorize the common types of threats:

Physical threat/theft

Human error

Application error/buffer overflow

Equipment malfunction

Environmental hazards

Malicious software/covert channels

A threat coupled with a vulnerability can lead to a loss. Vulnerabilities are flaws or weaknesses in security systems, software, or procedures. An example of a vulnerability is human error. This vulnerability might lead an improperly trained help desk employee to unknowingly give a password to a potential hacker, resulting in a loss. Examples of losses or impacts include the following:

Financial loss

Loss of reputation

Danger or injury to staff, clients, or customers

Loss of business opportunity

Breach of confidence or violation of law

Losses can be immediate or delayed. A delayed loss is not immediate; it has a negative effect on the organization after some period of time—in a few days, months, or years. For example, an organization could have its website hacked and thus suffer an immediate loss. No e-commerce transactions occur, technical support has to be brought in to rebuild the web server, and normal processing halts. All these are immediate losses. Later, when the local news channel reports that the company was hacked and that personal information was lost, the company loses the goodwill of its customers. Some might remember this event for years to come and choose to use a competitor. This is a delayed loss.

Thus far, we have discussed building a risk management team that has the support of senior management, identifying tangible and nontangible assets, and performing threat identification.

1. Determine the asset value (AV) for each information asset.

2. Identify threats to the asset.

3. Determine the exposure factor (EF) for each information asset in relation to each threat.

4. Calculate the single loss expectancy (SLE).

5. Calculate the annualized rate of occurrence (ARO).

6. Calculate the annualized loss expectancy (ALE).

The advantage of a quantitative risk assessment is that it assigns monetary values, which are easy for management to work with and understand. However, a disadvantage of a quantitative risk assessment is that it is also based on monetary amounts. Consider that it’s difficult, if not impossible, to assign monetary values to all elements. Therefore, some qualitative measures must be applied to quantitative elements. Even then, this is a huge responsibility; therefore, a quantitative assessment is usually performed with the help of automated software tools.

If asset values have been determined as previously discussed and threats have been identified, the next steps in the process for quantitative risk assessment are as follows:

1. Determine the exposure factor: This is a subjective potential percentage of loss to a specific asset if a specific threat is realized. This is usually in the form of a percentage, similar to how weather reports predict the likelihood of rainy conditions.

2. Calculate the single loss expectancy (SLE): The SLE value is a monetary figure that represents the organization’s loss from a single loss or the loss of this particular information asset. SLE is calculated as follows:

Single loss expectancy = Asset value × Exposure factor

Items to consider when calculating SLE include the physical destruction or theft of assets, loss of data, theft of information, and threats that might delay processing.

3. Assign a value for the annualized rate of occurrence (ARO): The ARO represents the estimated frequency at which a given threat is expected to occur. Simply stated, how many times is this expected to happen in one year?

4. Assign a value for the annualized loss expectancy (ALE): The ALE is an annual expected financial loss to an organization’s information asset because of a particular threat occurring within that same calendar year. ALE is calculated as follows:

Annualized loss expectancy (ALE) =

Single loss expectancy (SLE) × Annualized rate of occurrence (ARO)

The ALE is typically the value that senior management needs to assess to prioritize resources and determine what threats should receive the most attention.

5. Analyze the risk to the organization: The final step is to evaluate the data and decide whether to accept, reduce, or transfer the risk.

Much of the process of quantitative risk assessment is built on determining the exposure factor and the annualized loss expectancy, which rely heavily on probability and expectancy. When looking at events such as storms or other natural phenomena, it can be difficult to predict their actual behavior. Yet over time, a trend can be established. These events can be considered stochastic. A stochastic event is based on random behavior because the occurrence of individual events cannot be predicted, yet measuring the distribution of all observations usually follows a predictable pattern. In the end, however, quantitative risk management faces challenges when estimating risk, and it must therefore rely on some elements of the qualitative approach.

Another item that is sometimes overlooked in quantitative risk assessment is the total cost of a loss. The team should review these items as it’s assessing costs:

Lost productivity

Cost of repair

Value of the damaged equipment or lost data

Cost to replace the equipment or reload the data

When these costs are accumulated and specific threats are determined, the true picture of annualized loss expectancy can be assessed. Now the team can build a complete picture of the organization’s risks. Table 3-2 shows sample results.

Although automated tools are available to minimize the effort of the manual process, these programs should not become a crutch to prevent businesses from using common sense or practicing due diligence. Care should also be taken when examining high-impact events, even for the probability. Many of us witnessed the 100-year storm that would supposedly never occur in our lifetime and that hit the Gulf coast and severely damaged the city of New Orleans. Organizations must be realistic when examining such potential events and must openly discuss how such a situation should be dealt with. Just because an event is rated as a one-in-100-year probability does not mean that it can’t happen again next year.

Low: Minor inconvenience; can be tolerated for a short period of time but will not result in financial loss

Medium: Can result in damage to the organization, cost a moderate amount of money to repair, and result in negative publicity

High: Will result in a loss of goodwill between the company and a client or an employee; may result in a large legal action or fine; and may cause the company to significantly lose revenue or earnings

The downside of performing a qualitative assessment is that you are not working with monetary values; therefore, this type of assessment lacks the rigor that accounting teams and management typically prefer.

Other types of qualitative assessment techniques include these:

The Delphi technique: This group assessment process allows individuals to contribute anonymous opinions.

Facilitated Risk Assessment Process (FRAP): This subjective process obtains results by asking a series of questions. It places each risk into one of 26 categories. FRAP is designed to be completed in a matter of hours, making it a quick process to perform.

Without a cohesive and coordinated approach to managing risk, the number of defects and problems can increase. The Three Lines of Defense model is one method to continually assess the environment to ensure that people, process, and technology are meeting the organization’s goals. The Three Lines of Defense model provides a simple and effective way to ensure that risk is identified and reported to leadership. This model works for all industries and organizations of all sizes.

The Three Lines of Defense model identifies the key roles and responsibilities for managing risk in layers. The idea is that while one or two layers may miss a material risk, it is highly unlikely that all three layers would miss identifying a major risk. The roles and responsibilities in this model are as follows:

Business unit leadership: These business leaders have primary and ultimate accountability to ensure that appropriate management and internal controls are in place to manage risk. Key responsibilities include the following:

Day-to-day risk management of defects and process problems

Following policies and risk management process

Promptly remediating and reporting risk

Risk and compliance teams: These teams vary from one organization to another and generally advise and verify that management and internal controls are working as designed. Compliance and operational risk teams are typical examples of this internal control function. Key responsibilities include the following:

Advising and educating management on required controls and emerging risks

Managing key ERM processes

Testing to ensure that management and internal controls are working

Reporting to senior leadership on enterprise aggregated risk

Auditor: An auditor provides the risk governance committees and senior management with comprehensive assurance that risk is being appropriately managed across the enterprise. Key responsibilities include the following:

Reviewing the first and second lines of defense

Providing an independent opinion to senior leadership and the board of directors on the state of risk in the enterprise

Promptly remediating and reporting risk

The CISA exam expects a candidate to have a deep understanding of the auditor’s role. A key fact is understanding the auditor’s independence in the reporting structure. Audit teams generally report directly to a board of directors audit committee. In addition, for publicly traded companies, the head of the internal audit department (sometimes referred to as the General Auditor) is required to meet with the full board of directors several times each year.

Because the audit department reports directly to the board of directors, auditors’ opinions are considered the highest level of independence and objectivity in the organization. This high level of independence is not available in the second line of defense. Because the first and second lines of defense are subject to management oversight (including annual performance reviews), they cannot be considered completely independent.

Auditors play a big role in the success of an organization. Auditors must be independent of management and have the authority to cross departmental boundaries. Auditors must also have the proper skills. If in-house individuals do not have the skills required to lead an audit, an external independent third-party auditor should be hired. This situation requires careful attention. It’s natural to develop relationships with those we work with. Internal auditors interact extensively with their clients. This can lead to problems because the level of closeness between management and internal auditors might affect the results of an audit.

Finally, both external and internal auditors can burn out as a result of staleness and repetition, and they may thus start to lose attention to detail, which is very important.

An auditor is expected to be free to provide guidance and recommendations to senior management. The objective of providing recommendations is to improve quality and effectiveness. The first step of this process is to review the following:

Learn the organization: Know the company’s goals and objectives. Start by reviewing the mission statement.

Review the IT strategic plan: Strategic plans provide details for the next three to five years.

Analyze organizational charts: Become familiar with the roles and responsibilities of individuals in the company.

Study job descriptions: Job descriptions detail the level of responsibility and accountability for employees’ actions.

Evaluate existing policies and procedures: These documents detail the approved activities of employees.

For example, for a pizza shop that sells 12-inch hand-tossed pizzas, if the pizzas turn out to be between 11.5 inches and 12.5 inches, that may be well within the tolerance set by policy. However, for an airplane engine manufacturer, the parts design tolerance must be within 1 to 5 microns. These examples show the need to establish tolerance and the amount of risk that management is willing to accept.

Always remember that policies reflect leadership perception of priorities. An auditor has two main roles related to policies: (1) ensure that a policy is complete and reasonable, given industry norms, and (2) identify any misalignment between stated policies and actual practice.

An auditor can learn a great deal about an organization by simply reviewing the strategic plan and examining the company’s policies. These documents reflect management’s view of the company. Some might even say that policies are only as good as the management team that created them. Policies should exist to cover almost every aspect of organizational control because companies have legal and business requirements to achieve organizational goals.

Management is responsible for dividing the company into smaller subgroups so that control can be managed effectively. Policies will dictate how activities occur in each of the functional areas. One of the first steps in an audit is for the auditor to examine these critical documents. Any finding an auditor makes should be referenced back to the policy. This allows the auditor to specify how to rectify identified problems according to management views on risk.

Policies don’t last forever. Like most other things in life, they need to be reviewed periodically to make sure they stay current. Technology becomes obsolete, new technology becomes affordable, and business processes change. Although it’s sometimes easy to see that low-level procedures need to be updated, this also applies to high-level policies.

We defined standards, procedures, guidelines, and baselines in Chapter 2. In this chapter, we discuss the broad policy environment and how each artifact is developed, including the terms defined in Chapter 2.

For the purpose of this chapter, we use the term policy to reflect the policy environment. In this context, not all policies are created in the same way. The policy process can be driven from the top or from the bottom of the organization.

Top-down policy development means that policies are pushed down from the top of the company. The advantage of a top-down policy development approach is that it ensures that policy is aligned with the strategy of the company. It lacks speed, however, and may not reflect a complete understanding of how detailed processes actually work. This lack of understanding of detail could lead to confusion and unrealistic expectations. It’s a time-consuming process that requires a substantial amount of time to implement.

A second approach is bottom-up policy development. Bottom-up policy development addresses the concerns of operational employees because it starts with their input and concerns and builds on known risk. This is faster than a top-down approach but has a huge disadvantage in that it risks lack of senior management support.

No matter what the development type, policies are designed to address specific concerns, including the following:

Regulatory: Regulatory policies ensure that the organization’s standards are in accordance with local, state, and federal laws. Industries that frequently use these documents include health care, public utilities, refining, and the federal government.

Advisory: Advisory policies ensure that all employees know the consequences of certain behaviors and actions. An example of an advisory policy is one covering acceptable use of the Internet. This policy, called an acceptable use policy (AUP), might state how employees can use the Internet during the course of business; violating the policy could lead to disciplinary action or dismissal.

Informative: Informative policies are designed not for strict enforcement but for teaching. Their goal is to inform employees and/or customers. An example of an informative policy is a return policy on goods purchased on the business’s website.

Let’s discuss policy as a strategic document. A policy document outlines broad and strategic goals. A policy document is typically approved by a board of directors–level committee. The policy document outlines accountabilities and broad risk tolerance statements in the form of a business document. For example, a policy document may authorize the chief information security officer (CISO) to be accountable for setting and enforcing information and cybersecurity standards and procedures across the enterprise. The intent can ensure that the CISO has the authority to stop a cybersecurity attack, which may include taking some business systems offline. The policy may also outline the business’s priority for IT, such as stating opening up of operations in Europe is a strategic goal and holding the CIO accountable to ensure that appropriate technology is in place to control the cross-border movement of data.

Standards, in contrast to policy, describe how control should be deployed to achieve the policy and IT steering committee goals. Standards are much more specific than policies. A standard reflects industry-accepted norms and specifications for hardware, software, or human behavior. Standards should always point to the policies to which they relate. Standards are often technology agnostic. For example, a standard may say that “database administrators must use dual-factor authentication.” In this case, the standard does not specify which technology would be used to satisfy this requirement.

A procedure is an operational document that lays out specific steps or processes required to meet the requirements within the standards. Procedures also identify roles and accountabilities. To extend our dual-factor authentication example, procedures might say that to obtain a hardware token, an individual must request the device from a specific internal website and then get the device activated by the individual’s manager.

During an audit, an auditor must review all relevant procedures and map them to employee behavior through direct observation or interview. Misalignment can mean that there are no existing procedures, that procedures don’t map well to existing practices, or employees have not had the proper or adequate training on the procedures they are tasked with following.

Baselines procedures are operational documents that define the minimum configuration settings to achieve the standards requirements and support the procedure steps. This is the absolute minimum level that a system, network, or device must adhere to. To extend our dual-factor authentication example, a baseline may describe how to configure a Windows OS and Oracle database to accept only the approved hardware tokens for authentication. The Windows OS and Oracle database configuration setting for dual-factor authentication would be quite different, and thus two separate baselines would be created.

Risk management documents, especially the identification and inventory of risks

Human resources documents

Quality assurance procedures

Process and operation manuals

Change management documentation

IT forecasts and budgets

Security policies and procedures

Organizational charts and functional diagrams

Job details and descriptions

Steering committee reports

Documents that deal with external entities (sometimes referred to as third parties) should also be reviewed. A company might have contracts with vendors or suppliers for an array of products and services. How vendors are chosen, how the bidding process functions, what factors are used to determine the best bid, and what process is used to verify contract completion should all be reviewed. During the review process with policies, procedures, and documentation, any of the following might indicate potential problems:

Lack of guidance on what policies are to be followed

Excessive costs

Budget overruns

Late projects

A large number of aborted projects

Unsupported hardware changes or unauthorized purchases

Lack of documentation

Out-of-date documentation

Employees unaware of or not knowledgeable about documentation

Policies related to external entities (that is, third parties) is a complicated topic and often a point of interest for regulators. The reason the topic can get complicated is that an organization is ultimately accountable for how an external entity conducts business on its behalf. Yet often the organization has no direct control over how the external entity operates—no direct control but ultimately accountable for someone else’s actions. Confusing, right? For example, assume that a company makes loans and, in the process, collects all kinds of personal and private information. The organization then hires an external entity (typically referred to as a vendor) to obtain a credit report on each applicant and sort the results by credit scores and demographics by region. Now let’s assume that there is a data breach of the external entity’s computer, and someone steals all your customers’ personal information.

Let’s examine who may be legally accountable for a breach at the external entity or vendor site, given our example. As an auditor, you would be expected to sort through the complexities and determine internal accountabilities—that is, what went wrong and why. An auditor does not determine legal accountability but can determine whether the actions taken by the organization meet the requirements and rules set by the regulators. Only the courts and a judge can determine legal accountability. In this example, here are a few assessment areas that may be of interest to an auditor:

The quality of the vendor: How were the vendor selection and the vendor’s capability assessed by the organization? Did the vendor have the resources to properly protect the organization’s data? An organization should never select a vendor exclusively based on cost.

Expectations on the vendor: Were expectations clearly conveyed to the vendor through contract, policies, standards, and so on? How were those expectations monitored by the organization? An organization has an obligation to monitor whether vendors are living up to their expectations. This may include onsite inspections of the vendor’s facilities.

Expectations on the organization: Did the organizational policies and controls contribute to the vendor’s breach? Let’s assume that to obtain credit scores and determine demographics, the vendor needed a tax ID, name, and ZIP Code. Let’s also assume that the organization passes all the personal information obtained during the loan application process, such as address, salary information, mother’s maiden name, and so on. While the organization did not contribute to the failure to protect the customer’s information effectively, the organization did contribute to the impact of the breach by sending too much personal information to the vendor. In other words, an organization should send to the vendor only the information needed to perform the contracted service. The unnecessary exposure of customer information to a vendor can create legal accountability in the form of fines for the organization.

This is an important concept related to business accountability and drives many organizations’ policies related to external entities such as vendors. Regulators are consistent in requiring that once an organization collects personal information, it has an obligation to ensure that it is properly handled—including by third-party vendors.

Most organizations prefer to use three to five data classifications. This way, handling rules and controls can be simplified and standardized. In addition, the smaller the number of data classifications, the easier it is to train personnel. Data and information assets are classified with respect to the risk of unauthorized disclosure, such as lost, stolen, and inadvertently disclosed. A simple data classification scheme is illustrated in Table 3-5.

A data classification process typically separates information into distinct classes, which are then aligned to various standards, procedures, and baselines. It is also important to align these policies with regulatory requirements. For example, the Health Insurance Portability and Accountability Act (HIPAA), a U.S. law designed to provide privacy standards to protect patients’ medical records, requires that patient information be stored securely. Electronic health records could be classified as customer confidential, and the hospital standards could require such data to be stored in encrypted form.

From time to time, information must be destroyed. To facilitate the destruction of data, an organization could classify data into different groupings. For example, emails could be classified as a group of data that must be deleted after 90 days. Such a policy allows an organization to consistently purge obsolete data against a consistent set of classification rules.

Given the explosion of data collected in recent years, data classification has become increasingly important to managing the dizzying volume of information. Keep in mind the cost of classifying data. Data that is more valuable requires more controls. The more controls applied to data, the higher the cost to securely collect, store, and manage the data.

The first step to take before classifying any information is to define the levels of classification and what controls should be applied to each classification. Consider the overall costs of the controls, based on the volume and value of data.

Once classifications are defined, an organization faces the costs of inventorying existing data against the classification types and of implementing the supporting controls. Automation can help. For example, data loss prevention (DLP) technology can help automate the protection of data such as blocking any attempt to email documents labeled “business confidential.” Automation can be used to manage data leakage and generate reports that support these policies. In addition, automation can support records retention schedules by identifying the types of data specified and their location, allowing for proper archiving or destruction to occur.

DLP systems can also be incorporated in baseline and configuration settings that block the transfer of data onto a USB drive. Another action could result in the system encrypting the sensitive data in such a way that only authorized users can decrypt it. The key point is that data classification is a powerful tool that can support the policies of an organization.

An audit of data classification processes is important to gain an accurate view of the nature of the data, including how data is valued and types of risks perceived by leadership if that data was compromised.

An audit can start with the existing metadata information, as well as the details of where and how the information has been stored, to give the richest possible view of the content. It’s important for an audit to sample data based on the metadata definitions and standards. For example, a payroll clerk might, out of convenience, create a spreadsheet to balance a department budget. If that spreadsheet is stored on the clerk’s laptop, it may be more susceptible to a data breach, which may violate the organization’s security standards.

An auditor must look closely at security policies during the audit process and should review them to get a better idea of how specific access controls should function. Often security requirements are added to many different types of policies. For example, an auditor should examine policies that have been developed for disaster recovery and business continuity. Some questions to consider are what kind of hardware and software backup are used; whether the software backup media is stored offsite; and, if so, what kind of security the offsite location has and what type of access is available. These are just a few security-related items an auditor needs to review.

It’s fairly common to see the principle of least privilege in security policies. The idea is that you can improve security by limiting access to just the functions that are consistent with the individual’s job function. That way, if an account is compromised, the amount of harm that can be performed is contained or limited to that job’s role.

The concept is simple, but the implementation quickly becomes challenging as the size of an organization grows. Assume that an organization has thousands or hundreds of thousands of accounts. The idea of going through each account one at a time and customizing security may not be practical. Grouping the accounts into roles and assigning access permissions by roles is much simpler. The challenge is that two users may be almost identical except in terms of a few functions that are different. What do you do? Create two roles with lots of duplication? Put both users in the same role, knowing they may have slightly more access than they need?

Many organizations adopt the principle of least privilege but make compromises to balance the need to reduce access to the least amount practically possible. In other words, least privilege is a concept, not a hard rule.

Most security policies make a distinction between privileged and non-privileged accounts. Think of privileged accounts as administrative accounts and accounts with higher risk privileges, such as the ability to transfer money. The privileged accounts are sometimes referred to as superusers, or users with the “keys to the kingdom;” if these accounts are compromised, the risk of significant impact to the organization rises. Think of non-privileged accounts as standard users whose access is limited under least privilege to a single job function and typically a specific set of transactions. If these accounts are compromised, the risk of significant impact to the organization is reduced compared with a privileged account.

Everyone wants to get the right person for the job, but good HR practices require more than just matching a resume to an open position. Depending on the position to be filled, company officials need to perform due diligence in verifying that they have matched the right person to the right job. For example, Kevin might be the best security expert around, but if it is discovered that he served a 10-year sentence for extortion and racketeering, his chances of being hired by an interested company will be slim. Some basic common controls should be used during the hiring practice:

Background checks

Educational checks

Reference checks

Confidentiality agreements

Non-compete agreements

Conflict-of-interest agreements

Hiring practices should be performed with due diligence. References can be checked, education verified, military records reviewed, and even drug tests performed, if necessary. When an employee is hired, he brings not only his skills but also his background, history, attitude, and behavior.

Once hired, employees should be provided with an employee handbook detailing the employee code of conduct, acceptable use of company assets, and employee responsibilities to the company. Per ISACA, the handbook should address the following issues:

Use of social media while at work

Use of company-owned devices (assets and technology)

Employee package of benefits

Paid holiday and vacation policy

Work schedule and overtime policy

Moonlighting and outside employment

Employee evaluations

Disaster response and emergency procedures

Disciplinary action process for noncompliance

Hiring is just the first step in good employee management. Employees can follow policies only if they understand them. Auditors should verify that HR has a written, well-defined performance evaluation process. Performance assessments should occur on a predetermined schedule and should be based on known goals and results. A fair and objective process should be used. Pay raises and bonuses should be based strictly on performance.

Training is another area that falls under the responsibility of HR and the business unit. Employees might not know proper policies and procedures if they are not informed and trained. Training increases effectiveness and efficiency. When a new process or technology is introduced in the organization, employees should be trained for proper operation. Training is also beneficial because it increases morale; it makes people feel better, so they strive to do a better job. Training categories include those for technical, personnel management, project management, and security needs.

Training can range from lunchtime programs to learning programs, multiday events, or degree programs. Common training methods include the following:

In-house training

Classroom training

Vendor training

On-the-job training

Apprenticeship programs

Degree programs

Continuing education programs

Required vacations are just one of the employee controls that can be used. Another control is rotation of assignment, which allows more than one person to perform a specific task. This not only helps ensure a backup if an employee is unavailable but also can reduce fraud or misuse by preventing an individual from having too much control over an area.

One other closely related control worth mentioning is dual control. Dual control requires two individuals to provide input or approval before a transaction or an activity can take place. In banking, moving large sums of money is often under dual control. For example, sending a large wire transfer from one account to another typically requires the manager and supervisor to sign off on the transaction. This prevents a manager from wiring herself a large sum of money and disappearing.

A separation event may not be voluntary, and there needs to be a process to handle the situation properly. The applicable policy must cover issues such as escorting the employee out of the facility, exit interviews, review of non-disclosure agreements (NDAs), and suspension of network access.

Help desk

End-user support manager

Quality assurance manager

Data manager

Rank-and-file employees

Systems development manager

Software development manager

Most organizations have clearly defined controls that specify what each job role is responsible for. An auditor should be concerned with these common roles in the IS structure:

Data-entry employees: Although most data-entry activities are now outsourced, in the not-too-distant past, these activities were performed in-house at an information processing facility (IPF). A full-time data-entry person was assigned the task of entering all data. Barcodes, scanning, and web entry forms have also reduced the demand for these services. If this role is still used, key verification is one of the primary means of control.

Systems administrators: This employee is responsible for the operation and maintenance of the LAN and associated components, such as midrange or mainframe systems. Although small organizations might have only one systems administrator, larger organizations have many.

Quality assurance employees: Employees in a quality assurance role can fill one of two roles: quality assurance or quality control. Quality assurance employees make sure programs and documentation adhere to standards; quality control employees perform tests at various stages of product development to make sure products are free of defects.

Database administrators: This employee is responsible for the organization’s data and maintains the data structure. The database administrator has control over all the data; therefore, detective controls and supervision of duties must be observed closely. This is usually a role filled by a senior information systems employee because these employees have control over the physical data definition, implementing data definition controls, and defining and initiating backup and recovery.

Systems analysts: These employees are involved in the system development life cycle (SDLC) process. They are responsible for determining the needs of users and developing requirements and specifications for the design of needed software programs.

Network administrators: These employees are responsible for the maintenance and configuration of network equipment, such as routers, switches, firewalls, wireless access points, and so on.

Security architects: These employees examine the security infrastructure of the organization’s network.

Authorization: Verifying cash, approving purchases, and approving changes

Custody: Accessing cash, merchandise, or inventories

Record keeping: Preparing receipts, maintaining records, and posting payments

Reconciliation: Comparing monetary amounts, counts, reports, and payroll summaries

An individual having excessive access privileges beyond those needed for his or her role may lead to malicious, negligent, or accidental misuse of access. The more dangerous combinations of access that could cause the greatest harm are sometimes referred to as toxic combinations. Table 3-6 lists some of the duties (that is, toxic combinations) that should not be combined because they can result in control weaknesses.

Job rotation: The concept is to not have one person in one position for too long a period of time. This prevents a single employee from having too much control.

Audit trail: Although audit trails are popular after security breaches, they should be examined more frequently. Audit trails enable an auditor to determine what actions specific individuals performed; they provide accountability.

Reconciliation: This is a specific type of audit in which records are compared to make sure they balance. Although it is primarily used in financial audits, reconciliation can also be used for computer batch processing and other areas in which totals should be compared.

Exception report: This type of report notes errors or exceptions. Exception reports should be made available to managers and supervisors so that they can track errors and other problems.

Transaction log: This type of report tracks transactions and the time of occurrence. Managers should use transaction reports to track specific activities.

Supervisor review: Supervisor reviews can be performed through observation or inquiry, or they can be done remotely, using software tools and applications.

Let’s consider a few examples of what IT performance management should measure. Does measuring the number of technology changes implemented over the past month seem important? Or does measuring the number of business service requests seem more appropriate? These measurements certainly have value, but they do not tell leadership whether the services are effective, cost-efficient, or aligned to strategic goals.

When we think about performance management, we need to think broader than the processes we run. Let’s consider the following perspectives:

The customer perspective: Includes the importance the company places on meeting customer needs. Even if financial indicators are good, poor customer ratings will eventually lead to financial decline.

Internal operations: Includes the metrics managers use to measure how well the organization is performing and how closely its products meet customer needs.

Innovation and learning: Includes corporate culture and its attitudes toward learning, growth, and training.

Financial evaluation: Includes timely and accurate financial data. Typically focuses on profit and market share.

We put these broader perspectives into performance management, which helps us understand not only what we produce but also what we consume to produce our products and services. The pitfall of performance management measurements is taking the easy way out and only measuring quantitative waypoints that are readily available, such as the number of widgets produced, cost, speed, and quality. These readily available metrics have operational value but by themselves do not tell management if they are headed in the right direction.

When we add the broader perspective just discussed, we force performance management to align measurements to business objectives. For example, rather than just measuring speed to delivery in the abstract, you might measure customer satisfaction. Measuring customers who are highly satisfied with the product or service will tell you if the speed and quality are meeting their expectation. Conversely, customers who are less satisfied will have an issue with quality, the speed of delivery, and/or cost.

Let’s explore each of the terms in Table 3-8 in the context of the malware example mentioned in the table. Say that management is trying to understand the effectiveness of the malware controls. The key to preventing operational disruptions is the capability to detect and cleanse malware. Knowing that cleansing is automated based on detection, management chooses the rate at which it can detect malware as its KPI. A high detection rate means less malware can cause disruptions. Knowing the level of redundancy in the processes, assume that management is comfortable that they can successfully manage one malware event per quarter. This threshold (typically referred to as a risk threshold) may indicate for the business the level at which unacceptable disruptions occur for products or services.

Sounds like a lot of moving parts? Yes. Performance management is all about what needs to be accomplished, the business goals, and key measurements. Once a goal is set, it’s a matter of comparing actual measurements against targets. In our example, the risk threshold is to have no more than one malware event per quarter, and given that the total number of malware events was four for the year, that threshold was achieved. Then why is the KGI a –400 percent? While the risk threshold was achieved, the business target value goal was to have only one malware event per year. The KGI is a broader indication of the business goal to be achieved. There is a close relationship between KPI and KGI: as the KPI changes, so does the KGI. In our example, if the malware detection rate is raised (as represented by the KPI), then we would expect to see a business goal being achieved, as represented by the KGI.

A steering committee needs to measure performance and align business strategy with IT objectives. A steering committee can be flooded with metrics. Selecting the metrics that are most insightful and can foster consensus among different organizational departments and groups to take action is essential in promoting healthy change. This is where a balanced scorecard (BSC) comes in. The information gathered using the balanced scorecard should be passed down the organizational structure to supervisors, teams, and employees. Managers can use the information to align employees’ performance plans with organizational goals.

There is no set format for a balanced scorecard. The measurements should reflect business goals and targets compared to actual performance. There should be a direct or implied relationship between the measurements. That is, as one performance measure changes, related indicators should also change, as in the example that increased malware detection capability will have a positive effect on achieving a business goal.

An organization will adopt multiple management and control frameworks, based on the risk being controlled. For example, an enterprise architecture framework is adopted to control the risks related to software and system deployments. A security framework is adopted to control risks related to information and cybersecurity, and a quality management framework is adopted to ensure that products and services are maintained within acceptable risk thresholds.

Think of management and control frameworks as best practices rules for unique disciplines in an organization. A larger organization with more diverse disciplines will have a greater number of frameworks adopted. This concept of organizational disciplines is important and explains many of the origins of the frameworks. This is especially true for the information systems disciplines. A finance department will have very different risks and challenges than an information security department. Both are important disciplines, and both have industry groups and associations promoting industry best practices. These industry groups and associations eventually create what we know as management and control frameworks. Table 3-8 reviews commonly adopted frameworks.

Table 3-9 is not an exhaustive list of management and control frameworks but is intended to illustrate the diverse disciplines and highlight common frameworks an IS auditor will encounter.

One way to enhance security and governance is to implement components of the NIST framework as requirements in an enterprise architecture (EA) plan. Such a plan organizes and documents a company’s IT assets to enhance planning, management, and expansion. The primary purpose of using EA is to ensure that business strategy and IT investments are aligned. The benefit of EA is that it provides traceability that extends from the highest level of business strategy down to the fundamental technology. EA has grown since John Zachman, the originator of the Framework for Enterprise Architecture, first developed it in the 1980s; companies such as Intel, BP, and the U.S. government now use this methodology.

Federal law requires government agencies to set up EA and a structure for its governance. This process is guided by the Federal Enterprise Architecture Framework (FEAF) reference model, which is designed to use six models:

Performance reference model (PRM): A framework used to measure performance of major IT investments

Business reference model (BRM): A framework used to provide an organized, hierarchical model for day-to-day business operations

Infrastructure reference model (IRM): A framework used to classify service components with respect to how technology supports the business through hardware, hosting, data centers, cloud, and virtualization

Application reference model (ARM): A framework used to categorize the standards, specifications, and applications that support and enable the delivery of service components and capabilities

Data reference model (DRM): A framework used to provide a standard means by which data may be described, categorized, and shared

Security reference model (SRM): A framework used to provide a standard means to describe information security and cybersecurity controls and how to adjust the risk and protect individuals’ privacy

Management is tasked with the guidance and control of the organization; managers are the individuals who are responsible for the organization. Although companies are heavily dependent on technology, a large part of management’s duties still involves people, processes, and related technology. People are key to making a company successful. Therefore, a large portion of management’s duties depends on people skills, including interaction with staff and with those outside the traditional organizational boundaries.

Outsourcing might not be a term that some people like, but it’s a fact of life that companies depend on an array of components and services from around the world. For example, consider Dell Computer. Dell might be based in Round Rock, Texas, but its distribution hub is in Memphis, Tennessee; Dell assembles PCs in Malaysia and has customer support in India. Many other parts come from the far corners of the globe. The controls that a company places on its employees and contracts, as well as its agreements with business partners and suppliers, must be examined and reviewed. The next several sections focus on good management practices. More outsourcing examples are discussed later in this chapter, in the section “Management of IT Suppliers.”

Technologists and IS auditors are tasked with ensuring that all changes are documented, accounted for, and controlled. Companies should have a well-structured process for change requests (CRs). The following steps provide a generic overview of the change management process:

1. Request a change.

2. Approve the request.

3. Document the proposed change.

4. Test the proposed change.

5. Implement the change.

CRs are typically examined by a subject matter expert (SME) before being implemented. CRs must also be assessed to ensure that no change poses a risk for the organization. If an application or code is being examined for a potential change, other issues must be addressed, including how the new code will move from the coding to a production environment and how the code will be tested, as well as an examination of user training. Change management ensures that proper governance and control are maintained.

Hardware and software requisitioning

Software development

Information systems operations

Human resources management

Security

Why are so many quality management controls and change management methods needed? Most companies move data among multiple business groups, divisions, and IT systems. Auditors must verify the controls and attest to their accuracy. ISO 9001 is one quality management standard that is receiving widespread support and attention. ISO 9001 describes how production processes are to be managed and reviewed. It is not a standard of quality but covers how well a system or process is documented. Companies that want to obtain an ISO 9001 certification must perform a gap analysis to determine what areas need improvement. The ISO 9001 consists of six procedure documents that specify the following:

Control of documents

Control of records

Control of nonconforming product

Corrective action

Preventive action

Internal audits

Being ISO certified means that the organization has the capability to provide products that meet specific requirements; this includes the process of continual improvement. Being ISO certified can also have a direct bearing on an IS audit because it places strong controls on documented procedures.

Another ISO document that an auditor should be aware of is ISO 27000 series, which is considered a code of practice for information security. These documents are written for individuals who are responsible for initiating, implementing, or maintaining information security management systems. Its goal is to help protect confidentiality, integrity, and availability, and it includes the following:

Risk assessment and treatment

Security policy

Organization of information security

Asset management

Human resources security

Physical and environmental security

Communications and operations management

Access control

Information systems acquisition, development, and maintenance

Information security incident management

Business continuity management

Compliance

For more information on the ISO, see www.iso.org/isoiec-27001-information-security.html and www.iso.org/iso-9001-quality-management.html.

A final control framework worth mentioning is the Committee of Sponsoring Organizations of the Treadway Commission (COSO), which was designed to improve the quality of financial reporting. The COSO framework sets specifications for the following:

Defining internal control

Categories of objectives

Components and principles of internal control

Requirements for financial control effectiveness

The COSO framework is a series of documents that illustrates approaches and examples of how principles are applied in preparing financial statements. These components constitute a viable framework for describing and analyzing an organization’s internal control system in a way that conforms to financial regulations. The framework considers changes in business and operating environments and demonstrates how a variety of entities should operate, including public, private, not-for-profit, and government organizations. COSO framework definitions and principles include the following core areas:

Control Environment

Risk Assessment

Control Activities

Information & Communications

Monitoring Activities

For more information on COSO, visit www.coso.org/Pages/default.aspx.

The underlying premise of all these management and control frameworks is that an organization exists to provide value for its stakeholders. All organizations face uncertainty, and the challenge for management is to determine how much uncertainty to accept as it strives to grow its services to its customers and drive stakeholder value. Uncertainty presents both risk and opportunity. Effective management of risk can bolster confidence or enhance value. Management and control frameworks can maximize value when management sets strategy and objectives to strike an optimal balance between delivery, growth, and risks. Effective management and control frameworks will achieve the following:

Align strategy and risk appetite

Implement effective processes to enable risk response decisions

Reduce operational surprises and losses

There are many CMMs on the market, focused on different industries and addressing different risks. Most CMMs align to five maturity levels, as described in Table 3-10.

Carnegie Mellon University provided one of the first major CMM models adopted by the industry in 1990. In 2006 Carnegie Mellon University released a major upgrade, referred to as the capability maturity model integration (CMMI) model. The COBIT 5 Capability Maturity Model references the same five maturity levels and is based on the ISO/IEC 15504 Capability Determination Model.

A CMM is an activity-based model. It focuses on the completion of a process and does not care about the desired result and, hence, does not motivate the organization to make the necessary changes. In contrast, CMMI is a result-oriented model based on key performance areas and, therefore, represents best practice for a given knowledge area. The idea is that establishing and continually improving knowledge areas will help organizations decrease costs and improve quality and speed of delivery. The core CMMI bodies of knowledge are illustrated in Figure 3-1.

The COBIT 5 CMM is outcome based. The difference between COBIT 5 CMM and CMMI is that COBIT 5 is applied broadly against five domains that include 37 processes, covering all aspects of managing and delivering technology solutions, from the board level to the developer. These are the five COBIT 5 domains:

Evaluate, Direct and Monitor (EDM)

Align, Plan and Organize (APO)

Build, Acquire and Implement (BAI)

Deliver, Service and Support (DSS)

Monitor, Evaluate and Assess (MEA)

While both CMMI and COBIT 5 CMM are outcomes based, CMMI can be viewed as more industry and specific process focused. The CMMI knowledge areas tend to be more prescriptive and detailed. In contrast, COBIT 5 CMM has broader application across multiple industries and aligns to specific control objectives across 37 well-defined processes.

Think about maturity levels the same way you think about school. Assume that your local school requires Algebra I for eighth grade and Algebra II for ninth grade. An individual will be eligible to graduate from eighth grade to ninth grade only when she demonstrates that she has achieved proficiency in Algebra I. In addition, the proficiency in Algebra I is foundational for meeting the next requirements for Algebra II.

Maturity models work much like the algebra graduation analogy. To graduate between maturity levels, an individual must demonstrate having met all the prescriptive requirements, as defined by whichever framework has been chosen. In addition, each subsequent layer will build on the previous layer as the maturity level increases. We can illustrate this point with a simplified example related to project management process maturity requirements:

Level 2 requires the following:

Establish cost estimates.

Establish a plan.

Obtain approval.

Level 3 requires the following:

Coordinate and collaborate with stakeholders.

Establish a back out plan.

The project management process maturity requirements shown here illustrate the set of simplified requirements needed to graduate from Level 2 to Level 3 maturity. This is not to suggest that the project would not be successful at maturity Level 2. As maturity level rises, risk is taken out of the process. In this case, two risks would be eliminated in moving from maturity Level 2 to Level 3. The first risk is reduced by formally engaging the stakeholders in the development and deployment of the project. The second risk is reduced by ensuring that a formal backout plan is established in the event that the project does not function as expected. Neither of these risks may occur at Level 2. Nonetheless, having a formal plan to deal with both instances will increase the projected likelihood of success.

Achieving maturity Level 5 is generally accepted as applying a higher level of automation to reducing defects and driving consistency. Should all organizations strive for maturity Level 5? No. Each progressive maturity level comes at a cost. Applying maturity Level 5 to every process would be cost-prohibitive, and the introduction of automation can make simple tasks more complex. For example, updating a monthly price table may be ideal for humans, while scanning for malware requires a high degree of automation.

The determination of what maturity level is required is driven by balancing risk, cost, industry best practices, and what’s needed to achieve regulatory compliance. An IS auditor needs to ensure that an appropriate set of tools and criteria have been used within management’s risk decision process.

U.S. Health Insurance Portability and Accountability Act (HIPAA): U.S. standards on management of health care data

Sarbanes-Oxley Act (SOX): U.S. financial and accounting disclosure and accountability for public companies

Payment Card Industry (PCI) standards: Handling and processing of credit cards

U.S. Federal Information Security Management Act (FISMA): Security standards for U.S. government systems

U.S. Fair and Accurate Credit Transaction Act of 2003 (FACTA): Legislation to reduce fraud and identity theft

You should see two themes emerging from these regulations related to the importance of protecting privacy and maintaining effective information security controls. Laws are often enacted after a major event or data breach. After such an event that broadly impacts markets or millions of customers, lawmakers often feel pressure to do something to ensure that such events do not reoccur. That something often takes the form of passing new laws or regulations. Laws and regulations have the benefit of being mandatory, which is a strong motivator for the market to move in a certain direction. The inherent weakness of laws and regulations is that they take a long time to enact and often are not put into place until well after the initial event occurred. Consequently, laws and regulations are typically considered lagging indicators of risk.

Management and control frameworks created by industry groups and associations are much better leading indicators of risk. These frameworks have the benefit of direct support and updates from industry leaders. In addition, industry framework updates are released on a much shorter timeline than laws and regulations. The inherent weakness of these frameworks is that they are optional and not enforceable in the same way as laws and regulations. The scope and level of adoption of industry frameworks are dependent on leadership’s commitment, regulatory inquiries, and peer pressure. Consequently, organizations will comply with both regulations and industry frameworks to control technology risks. Industries that are highly regulated generally tend to have more formal adoption programs related to industry frameworks and regulatory mandates.

Management must demonstrate and evidence compliance. It’s not enough to have trained teams of employees and published standards. An IS auditor looks for evidence that an organization complies with key requirements and controls risk consistently. Regulators want to see a culture of managing risk effectively through regulatory compliance. Organizations that tend to do well during an audit or a regulatory exam have the following in place to support evidence of compliance:

Organizational functions dedicated to compliance: Management must demonstrate that teams understand regulatory expectations and continually review internal controls for compliance

Examples: Compliance, operational risk, and audit functions

Risk culture: Management must promote a risk culture. More than publishing standards, management must establish a tone at the top—a term that refers to actions taken by leadership to visibility demonstrate active support for the compliance program.

Examples: Leadership placing risk discussion as a priority on agendas and management reaction to noncompliance events

Risk strategy: An organization needs to have a well-articulated risk strategy.

Examples: Policies, standards, and processes to control risk and ensure compliance

Risk registry and risk assessments: An organization needs to have continuous risk assessments and a repository that tracks risks from identification, to remediation, to acceptance. This includes the organization’s ability to evaluate and communicate internal control deficiencies in a timely manner to the parties responsible for taking corrective action, including senior management and the board of directors, as appropriate.

Examples: Audits, risk examination, remediation tracking

Variances and inconsistencies

Risks to the process operations

Complexity

Costs

The job competencies covered in the CISA exam apply equally well to an IS auditor who must assess the quality of the processes deployed and the quality of any management optimization efforts that are under way. The CISA exam treats the topic as a foundational topic required for understanding the importance, needs, and general methods applied to optimizing processes. This section provides two examples of process optimization techniques, referred to as the Taguchi and PDCA optimization methods.

Since its introduction, the Taguchi method has been adopted and adjusted to work across industries beyond manufacturing. All processes are affected by outside influences, which Genichi Taguchi refers to as noise. The Taguchi method offers a systematic way of identifying the noise sources that have the greatest effects on product variability. The idea is that if you can reduce or eliminate this noise, you can produce products (and services) in an optimized and consistent manner.

While the Taguchi method can be used to improve existing processes, most engineers believe that the greatest value lies in applying the method when creating new processes. They believe that the best way to improve process quality is to design it into the process.

We will not go into the math or statistical formulas associated with the Taguchi method. While interesting, that level of detail will not be on the CISA exam and thus is beyond the scope of this text. (If you would like more information, you can find a number of studies related to the method on the Internet by entering “Taguchi method case study” as a key word search.)

The key concept to the Taguchi method is what’s termed an experiment. It’s an iterative process in that the following stages can be repeated over time:

Build → Test → Fix

Basically, each iteration of the test after the build is an experiment to measure the level of noise. As you pass through iterations and use statistical methods to measure the noise and outcome, you can determine the level of optimization being achieved.

The Taguchi principles and methods are a unique quality and process improvement technique. Optimized processes tend to produce consistent quality outcomes and be more insensitive to noise and variations in the environment. The Taguchi method to quality engineering places emphasis on minimizing variation as the main means of improving quality. The Taguchi approach is illustrated in the following iteration steps:

1. Identify the main function and unintended outcomes.

2. Identify the noise factors and testing condition.

3. Identify key quality characteristics.

4. Identify the objective method of measuring optimization.

5. Conduct the experiment.

6. Examine the data; predict optimum control levels and adjust.

7. Conduct the verification experiment.

This is a very useful method because it is statistically accurate. The outcome of the process becomes consistent and predictable, with low levels of variance. The Taguchi method gives you a quantitative way of measuring outcome quality. In addition, you can measure when optimization efforts result in no tangible effort or, worse, a negative effect. These experiments and measurements collectively improve management’s understanding of the process and avoid wasted efforts that do not significantly improve quality.

The PDCA model dates back to 1939, when Walter A. Shewhart, an American physicist, engineer, and statistician, first published the concept that constant evaluation of management practices is key to the evolution of effective processes and a successful enterprise. Since its first introduction, the concept has been widely adopted across different industries as a means of achieving continuous process improvement.

The basic PDCA iterative four-step process is as follows (see Table 3-11):

Plan: The plan step establishes formal control objectives, projects outcomes, and defines the processes needed to achieve the objectives and outcomes. The output from the expectations created in the plan step will become part of the development cycle for the check step. Pilot and prototype testing are encouraged in the PDCA model.

As an IS auditor or control partner assessing the process, you would either obtain these details from existing process documentation or reverse engineer to obtain them.

Do: The do step involves implementing the plan and executing the process. Data is then collected on the outcome, including data on the quality of the product and services produced. Data should be collected on each key requirement specified in the plan step.

Check: During the check step, the outcome from the do step is assessed. This assessment is sometimes referred as the PDCA study. The assessment compares the actual results collected in the do step against the predicted results in the plan step. Variances can be positive or negative. Positive variance means more value is obtained. Negative variance means less value than expected is obtained. Negative variance typically requires some level of corrective action.

Act/adjust: The act/adjust step takes as input the results from the check step and applies corrective action. During the act/adjust step, root causes are determined. Over time, trends are tracked and feedback is considered in the plan step so future processes can benefit. This iterative process establishes continuous improvement. Each pass through a PDCA iteration incrementally improves the process. The goal is to ensure that quality is both initially and continuously achieved.

The Taguchi quantitative approach is far more precise in the identification and statistical certainty of its outcome. Its high cost and complexity make it better suited for expensive and more critical processes. PDCA places a high reliance on qualitative judgment, and it’s far more reliant on the expertise of the assessor. Its comparable lower cost and agility makes it ideal for lower-cost, low-volume processes, such as back-office IT support processes.

Outsourced IT services can improve your organization’s focus. It is neither practical nor possible to be a jack of all trades. Outsourcing lets management focus on core competencies and competitive advantage while suppliers focus on being the best at their business. Suppliers also have the advantage of scale when an organization outsources information technology to a supplier that specializes in a particular area and can spread costs across multiple customers.

An organization must effectively manage the relationship and services it provides—whether on its own or through third parties—and balance the benefits and risk of handing control to an external supplier.

Onsite: Employees and contractors work at the company’s facility.

Offsite: Staff and contractors work at a remote location.

Offshore: Staff and contractors work in a separate geographic region.

Organizations should go through a sourcing strategy to determine what information systems tasks must be done by employees. Commodity services that do not offer a competitive advantage are often targeted for IT outsourcing. That has the benefit of allowing an organization to focus internal IT resources on the services that provide maximum value. Commodity services that are often outsourced include the following:

Data entry

Application/web hosting

Help desk

Payroll processing

Check processing

Credit card processing

One key to the outsourcing decision is determining whether a task is part of the organization’s core competency or proficiency that defines who the organization is. This is a fundamental set of skills or knowledge that gives the company a unique advantage. Outsourcing a core competency could put the company at risk because of the over reliance on the vendor. For example, if the core competencies were moved to a vendor who later went out of business then the company could lose that unique market advantage. Additionally, the company should analyze whether the tasks being considered for outsourcing can be duplicated at another location and whether they can be performed for the same or less cost.

Information security should also play a role in the outsourcing decision because some tasks take on a much greater risk if performed by others outside the organization. Any decisions should pass a thorough business process review. For example, does data entry report a large number of errors, is the help desk backlogged, or is application development more than three months behind schedule? Some of the most common outsourced tasks are data entry and processing. When a task is outsourced, accuracy can be retained by implementing a key verification process to ensure that the process was done correctly. For example, the company’s data entry department might key in information just as the outsourcing partner does in India. After both data sets are entered, they can be compared to verify that the information was entered correctly. Any keystroke that does not match flags an alert so that a data-entry supervisor can examine and verify it.

From a supplier’s viewpoint, having large numbers of customers auditing processes and facilities can be disruptive and can impact costs. Many suppliers recognize the need to provide their customers’ management with evidence that their processes are following industry best practices. Suppliers often hire external audit firms to perform what is called SSAE 16 assessments.

The SSAE 16 is an industry-accepted assessment of a supplier’s general control environment. It allows a supplier to be audited once, and the reports can be provided to multiple customers. Customers’ management can accept an audit in its entirety or call on its right-to-audit statement to focus on specific areas not covered by the SSAE 16 assessment.

With a time-sensitive process, implementing an SLA is one way to obtain a guarantee of the level of service from the supplier. The SLA should specify the uptime, response time, and maximum outage time to which the parties are agreeing.

Think of contracts as the early stages of establishing a relationship with an IT supplier. Both parties in negotiation convey their expectations and commitments. There is a difference between having committed outcomes and trying your best to achieve an outcome. If an organization’s transaction must be completed within a specific time, the supplier should add that SLA to the contract. Once contract terms have been agreed upon, the parameters of the relationship have been set.

An important benefit of effective contract management is clarity. The terms of a contract often become what is measured and managed. For example, an outsourced call center may require that 99 percent of calls be answered within so many rings of the phone. That term in the contract can be used as a measurement point to monitor the vendor’s performance.

A good contract anticipates disputes between management and the supplier and negotiates terms of mutual benefit. This concept of mutual benefit is important. When contract terms for the supplier are not cost-effective, the supplier may cut corners and may fail to deliver the quality and speed needed. Having healthy suppliers benefits the organization and the industry.

The key in performance monitoring of suppliers is to identify the risks that management wants to control. Not every term of a contract will be monitored. Management needs to focus on key risks to the business.

The organization is ultimately accountable for the performance of a supplier. It needs to view the supplier as an extension of the organization. The supplier will have access to the organization’s data and product. As a result, the quality of the organization’s products and services is often tied to a supplier’s performance. Think of it this way: if management chose not to outsource and produced an IT service internally, would they check on the quality? If the answer is yes, then most likely management needs to also check on the supplier’s quality.

Most risks can be avoided altogether if management creates a team that is dedicated to monitoring supplier performance and performing effective relationship management. Such a specialized team can establish a performance monitoring program based on controlling risks related the following themes:

Speed: The SLA terms are typically used to monitor the speed of delivery by the supplier.

Quality: Management should consider monitoring both the quality of the product or services being delivered by the supplier and the quality of the supplier’s staff. The contract should include terms related to the qualifications of the supplier team working on the IT solutions (for example, background checks, technical expertise).

Cost: Billing from the supplier should be monitored against contract terms. Outsourcing IT services often provides financial benefits that should be managed as well. A change order typically involves asking a supplier to vary the normal process. Costs associated with change orders need to be carefully monitored to ensure that a supplier does not overcharge and erode the cost benefits projected.

When unexpected situations arise, you need two reasonable entities to come together to solve the problem to the mutual satisfaction of both parties. At the core of this process should be a well-established relationship. Relationship management takes time and effort. The benefits are obvious when it’s done well. On the other hand, the outcome can be devastating when the supplier relationship is poor or when the supplier does the minimum to stay within prescriptive terms of the contract. For example, say that you have a supplier providing partial hosting services. Let’s assume that your own data center has a significant power disruption that is estimated to last 24 hours. Management would ideally like to shift additional processing to the supplier hosting facility. However, the supplier is at nearly full capacity, and the additional hosting is beyond the terms of the agreement. Sounds like an unsolvable problem, and management simply needs to take the hit on being out of business for 24 hours. When a supplier perceives the relationship with the client as long-term and profitable, however, it will go to great lengths to preserve the relationship. This may include contacting other customers and determining the feasibility of freeing capacity for the next 24 hours so the supplier can support additional hosting services. Now let’s reverse the example and assume that the supplier is moving between data center facilities, and the supplier will not be able to meet the contract’s SLA during that period of time. In this case, management can plan for the SLA disruption and reduce any associated risks.

The point is that effective relationship management with suppliers can bridge the interests of both entities and balance rewards and risks. It can also protect both parties from unexpected situations and ensure that risks are effectively managed. Here’s are some key takeaways:

Treat suppliers as an extension of your organization’s accountability: Maintain a close relationship with each supplier.

Expect the unexpected: Not all situations can be anticipated or covered in a contract.

Anticipate problems: Manage the supplier relationship for the long term and to mutual benefit.

Review core services at least annually: Even if a contract has not expired, the terms should be reviewed periodically.

Monitor performance: Monitor performance against key terms in the contract.

Other requirements are policies, procedures, and standards. These documents not only provide a high-level view of the mission and direction of the company but also guide employees in their day-to-day activities. Auditors play an important role in independently verifying that governance is working as expected. Auditors are tasked with reviewing an organization’s documents, standards, and policies to determine how closely they map to employee activities. This chapter discusses a variety of tools an organization may use, such as maturity models, optimization techniques, and third-party performance management.

Regardless of your role in an organization, it’s important to understand how governance and related layers of controls work.

balanced scorecard (BSC)

baseline

capability maturity model (CMM)

Control Objectives for Information and Related Technologies (COBIT)

data classification

enterprise architecture (EA)

enterprise risk management (ERM)

guidelines

Information Technology Infrastructure Library (ITIL)

IT steering committee

key performance indicator (KPI)

lagging indicator

leading indicator

metadata

outsourcing

Plan-Do-Check-Act (PDCA)

policy

principle of least privilege

procedures

qualitative risk assessment

quality assurance (QA)

right-to-audit clause

risk acceptance

quantitative risk assessment

risk avoidance

risk reduction

risk transference

rotation of assignment

segregation of duties (SoD)

standards

stochastic

Taguchi model

threat

three lines of defense

vulnerability

You have read in this chapter about the importance of risk assessment. Inventorying assets, determining the risks to those assets, and evaluating countermeasure options are all part of good IT governance.

In this exercise, you examine the proper order for quantitative risk assessment.

1. Place the following quantitative risk analysis steps and calculations in the proper sequential order, from 1 (first step) to 6:

_____ Determine the annual rate of occurrence (likelihood of occurrence).

_____ Identify threats to the asset.

_____ Determine the asset value (AV).

_____ Calculate the annualized loss expectancy for each asset.

_____ Calculate the single loss expectancy.

_____ Identify the exposure factor for each asset in relation to the threat.

2. Compare your results to the answers here:

1. Determine the asset value (AV).

2. Identify threats to the asset.

3. Identify the exposure factor for each asset in relation to the threat.

4. Calculate the single loss expectancy.

5. Determine the annual rate of occurrence (likelihood of occurrence).

6. Calculate the annualized loss expectancy for each asset.

a. ISO 17799

b. CMM

c. COSO

d. COBIT

2. Which of the following roles is a role whose duties should not be fulfilled by a network administrator?

a. Quality assurance

b. Systems administrator

c. Application programmer

d. Systems analyst

3. You are auditing a credit card payment system. The best assurance that information is entered correctly is by using which of the following?

a. Audit trails

b. Separation of data entry and computer operator duties

c. Key verification

d. Supervisory review

4. You are reviewing unfamiliar malware event records. Which of the following would be the best source of information to start your review about the file?

a. Trending charts based on the event records

b. Metadata information

c. Security access information

d. Executive summary on malware event

5. Look at the following common policy characteristics. The attribute most closely associated with a bottom-up policy development is that it __________.

a. aligns policy with strategy

b. is a very slow process

c. does not address concerns of employees

d. involves risk assessment

6. Which of the following best describes a balanced scorecard?

a. Used for benchmarking a preferred level of service

b. Used to measure the effectiveness of IT services by customers and clients

c. Used to verify that the organization’s strategy and IT services match

d. Used to measure the evaluation of help desk employees

7. Your organization is considering using a new ISP for time-sensitive transactions. From an audit perspective, what would be the most important item to review?

a. The service level agreement

b. The physical security of the ISP site

c. References from other clients of the ISP

d. Background checks of the ISP’s employees

8. Separation of duties is one way to limit fraud and misuse. Consider the following explanation: “This control allows employees access to cash or valuables.” Of the four separation of duties controls, which one most closely matches this?

a. Authorization

b. Custody

c. Record keeping

d. Reconciliation

9. Which of the following combinations of two job roles can be combined to create the least amount of risk or opportunity for malicious acts?

a. Systems analyst and quality assurance

b. Computer operator and systems programmer

c. Security administrator and application programmer

d. Database administrator and systems analyst

10. You have been asked to perform a new audit assignment. Your first task is to review the organization’s strategic plan. What is the first item that should be reviewed in the plan?

a. Documentation that details the existing infrastructure

b. Previous and planned budgets

c. Organizational charts

d. The business plan

COBIT framework: www.isaca.org/cobit/

IT governance: http://en.wikipedia.org/wiki/Information_technology_governance

Risk-based audit best practices: www.journalofaccountancy.com/issues/2009/dec/20091789.html