Audit Planning Shortcomings Often Arise in EQA Reviews

With the common practice of ranking the areas within the audit universe using a range of detailed “risk based” criteria it might seem straightforward to meet the IIA standards. However, Chris Baker, (Technical Manager, IIA UK) makes an important observation about audit plans that adopt this approach:

“Although the institute is looking for a risk based approach to audit planning, I still see too many audit plans that have a tenuous link to the organizations’ risks and I think this is one of the areas where internal auditors still aren’t very good in demonstrating that they are looking at things that really matter.”

The key point here is that a risk ranking of a series of processes, locations and systems within an audit universe is not necessarily the same as being truly aligned to an organization’s key value drivers and risks.

Sarah Blackburn (Member of IIA Global Board and former AC Chair) provides additional insight from her experience of EQAs:

“I’ve just been doing an EQA in an organization where the audit plan is split up into lots of pieces of work. I can see that the Audit Committee are very unhappy, and I am too, looking at it. Because they’ve got too little depth on things that are important, and even where they are looking at processes it appears that they are focusing on the stuff that’s tickable.

Every member of the audit committee that I’ve spoken to so far has basically said, well the audit reports don’t really tell us anything. They’re generally saying everything’s all right because they are looking at the stuff that’s easy to audit and they are not asking more fundamental questions in relation to the risk areas that are much more significant.”

What is the Problem with an Audit Universe?

Over the course of my career, my views on the importance of an audit universe have varied. The lean test is: “Does the audit universe deliver value to the external customer, or key internal stakeholders?” I think an audit universe can deliver real value, provided the time and effort spent on the audit universe is justified, in terms of the way it helps to focus the work of internal audit on the right areas.

When I explore the question of the balance between effort and payback, some auditors tell me that their audit universe and associated risk scoring is so complex only one or two members of the audit function know how to use it. In addition, several CAEs have confided to me that if they end up with a proposed plan using the audit universe and a risk scoring approach that does not accord with their expectations, they will adjust the risk weighting factors until they get the plan they were looking for!

My current assessment is that an audit universe can be a useful way of tracking what work has been done by audit and other functions, and can be a good way of considering potential future areas to look at (as long as it provides a complete and up to date picture of the risk assurance universe). However, my experience is that it is very easy to find risk weighting factors being applied to the audit universe that can be overly complex and time-consuming for the benefit obtained. Additionally, when I am told about the risk weighting factors that should be used there is no clear consensus what factors should be used and what weighting to apply to each factor.

Should the Plan be Based on Gross or Net Risk?

Another key question is whether gross or net risks should be used as a basis for the audit plan. The use of net risks will normally take into account the things that are being done to manage risks, allowing the audit plan to focus on the things that management judge are not being managed so well.

However, an alternative view is that the use of net risk information can mislead audit, and that gross risks should be considered. The argument is that the use of net risk information may result in audit being steered away from certain areas “because management is confident that area is under control” when – in fact – this is not the case.

Another angle on the gross vs. net debate derives from the still quite common audit practice of asking management: “What do you think we should audit?”

At face value asking management for their views about which risks are of concern appears to be customer oriented (at least in terms of internal stakeholders), but the danger is that internal audit simply addresses known or suspected issues that may or may not deliver additional value or be key from an external customer perspective. In addition, auditing known areas of concern can:

• Result in audit largely confirming what management already know;
• Reinforce the notion that audit is a second line of defence function that should do checking for management, which may also dilute management’s accountability for managing the risk;
• Limit the ability of internal audit to cover other areas.

This topic will be explored further in the next chapter on risk assurance, but it is worth noting that when I talk to CAEs about the gross vs. net dilemma there appears to be no clear consensus on the best approach to follow.

A Focus on Cost Rather than Value

I have worked with audit functions in the UK public sector for several years and have a huge admiration for many colleagues working there. However, following the financial crisis, UK public spending is being severely reduced, leading to a significant amount of downsizing and consolidation across a range of functions, including internal audit. Such an environment can be a catalyst for adopting lean ways of working, and many of my early clients in lean auditing were from the UK public sector.

However, I have realized, from discussions with CAEs I have worked with, that an expectation of a number of stakeholders has simply been to reduce costs. The result has been, as one CAE described it, “a race to the bottom” to look for the bare minimum assurance, with the lowest cost staff.

Looked at from one perspective, members of the UK public might not want the luxury of expensive internal auditing that is doing nice-to-have work. However, the danger is that short-term savings are being achieved at the expense of more damaging and expensive issues arising later, particularly when staff cuts take hold, with a risk of a less effective safety net to catch things before they go wrong.

The risk of stripping audit back to the bare bones applies across all sectors when there are profit and cost pressures. However, it highlights an important choice: when an organization is downsizing, should audit be the first function to be downsized because it is an overhead function, or the last, because it can act as the safety net to catch issues before they cause material damage? I know my preference, so long as the audit function is looking at the right things!

Staff Capability may be a Factor as Well

An additional area of challenge for some audit functions is the extent to which the audit plan may be affected by the skills of internal audit staff. Chris Baker (Technical Manager, IIA UK) offers the following perspective:

“Sadly you’ll find audit functions who are reluctant to tackle the most important areas, because they can be challenging to execute, and they feel as though they don’t have the confidence or the ability to look at them.”

There can therefore be a tendency to preserve the status quo. This can also be reinforced by keeping the audit universe relatively “tame”, mostly focusing on processes, locations and systems, which are easier to audit and therefore do not highlight staff capability shortcomings. However, auditing less important areas reinforces the notion that audit is essentially a lower grade checking function, which makes it less easy to attract, and to justify paying for, higher quality staff.

All the while big value issues and key emerging risks may be unfolding above (or below) the assurance radar screen, which may result in audit failing to deliver the value adding contribution it could.

A concern I hear from audit staff is that working on key risks will be a stretch for their skills, so those assignments may not be very effective or efficient. I personally think these difficulties can be managed, but recognize that this shift towards true risk based assurance can be challenging.

Actions for internal audit to consider:

• Is the time and effort spent on any audit universe scoring clearly worth it?
• Is it possible to clearly see value drivers, key risks, initiatives and customer imperatives etc. within the audit universe?

Prioritize the Audit Plan, and Focus on Value

Chris Baker (Technical Manager, IIA UK) provides an important insight:

“When I do an EQA I always look at audit planning, to begin with, to try to get a feel for where the internal audit function sits. If I can see the audit team getting involved with senior stakeholders and what’s on their agenda, and looking at how well risk management has been designed, and whether risk management processes are being applied and are working, and building an audit plan from this that has a clear linkage to key risks, then it is highly likely the rest of the EQA is going to be positive.

If there is a disconnect from senior management, or the risk management process, or key organizational priorities and risks, there is likelihood that other shortcomings will follow.”

Here is advice from Norman Marks (GRC thought leader):

“Consider what is on the agenda of the Board. And ask: are you addressing all the issues that arise from, or contribute to, the success of the Board in managing these agenda items?

What are you doing that’s not on their agenda and if it’s not on the agenda why the devil are you doing it?

Of course it’s also sensible to look at any issues that should be on their agenda but are not, but you need to be clear that they are of importance.”

Jonathan Kidd (Chief Audit Executive for the UK Met Office) explains what his function is doing:

“Increasingly we are moving towards the more strategic view where the senior managers and the audit committee appreciate the limited resource we have and the opportunity cost of not positioning the audit function to support the delivery of strategic priorities.

Everything that we do, the whole way we present what we do is now aligned with what the organization is trying to achieve.”

Actions for Internal Audit to consider:

• Create a planning process that clearly is intended to deliver the maximum value add;
• Establish a clear, transparent link between key value and risk areas and the audit plan.

Create a Streamlined Planning Process

I well recall that when we looked at the audit planning process when I was CAE, we had a number of auditors engaged in the audit planning process. This gave us a lot of insight into what we could look at, but risked taking up a lot of time and resource (within audit as well as management). We clarified and streamlined this process, which also helped us ensure we did not disappoint middle managers who might make suggestions for audit work, which we could not deliver due to having only finite resources.

Greg Coleman (Corporate Assurance Director, ITG) explains his approach to engaging internal clients and stakeholders in a focused and efficient way:

“We have a reasonably structured audit planning approach where we start with the strategic plan from the organization and look at what that means in terms of new initiatives and related risks over the next year or two.

We then hold one on one meetings with senior management to talk about key risk areas and consider what other functions provide assurance. But then to save time, and maximize stakeholder insights we hold a couple of workshops to prioritize key risks and validate assurances. This is done with members of the audit team combined with other colleagues from key functions, such as legal, finance, health & safety, IT and corporate affairs. After doing this we obtain sign off of the plan from senior management and the audit committee.”

Actions for Internal Audit to consider:

• Ensure there is a simple process map of the audit planning process and ensure the audit function and key stakeholders understand how it works;
• Look out for a planning process that consumes a lot of audit resource and sets unhelpful expectations within management;
• In addition to one-to-one meetings, consider workshops with key contacts as a way of validating risks and assurances.

Take a Gross Risk Perspective (at Least at the Start)

Here is advice from Phil Gerrard (CAE, Rolls-Royce):

“You need to understand what are the big gross risks because if they are not featuring on the key, or top risks, the inference is that they are being managed down to a relatively low likelihood or a more acceptable impact. That may be valid, but the impact could be still significant if mismanaged. So whilst they no longer look like top priorities, there are lots of assumptions underpinning that.

The role of audit has to be to challenge the quality of the mitigation that’s gone into that risk assessment.”

Sarah Blackburn (Member, IIA Global Board) endorses this view:

“I believe risks should be addressed by internal audit at the inherent level. My logic is as follows: Whilst I’m glad to hear the organization thinks it has got good controls over some areas, I still want someone to take a look that this is indeed true from time to time.”

Wee Hock Kee (former President of the IIA Malaysia and a former colleague of mine at AstraZeneca) comments:

“I think we have a duty to move up the value chain, not only looking at issues from a control perspective, but increasingly trying to tie things back to the risk management and the governance framework.”

Actions for Internal Audit to consider:

• Pay close attention to the way the planning process weights gross vs. net risks; ensure gross risks are not inadvertently downplayed;
• Pay close attention to “black swan” risks that have a high impact and a low likelihood; these are often discounted in audit planning processes.

Upgrade the Audit Universe, but Don’t Necessarily Let it Drive the Plan

As discussed earlier, all too often an audit universe does not really align with key organizational risks and value drivers. My advice is to look at how it can be expanded to better capture all of the key priorities, major projects and key risk areas that matter. Greg Coleman (CAE, ITG) explains the outcome of work on the audit planning process that incorporates lean principles:

“We now have a more advanced audit universe, that includes not just the locations but the key processes within the locations, as well as key third parties we trade with, a range of IT risk areas as well as key projects and other risk areas. It is significantly bigger than it was in the past and better captures the total risk assurance dimensions of the group.

However, we actually drive the audit plan by focusing on organizational priorities and key risks and use the audit universe to complement this rather than to drive it.”

The power of having a good audit universe is that it can help inform a discussion with senior management and the board about the amount of coverage that internal audit is able to provide against key areas. In addition, if the audit function is required to deliver an opinion on the overall effectiveness of GRC activities in an organization, a robust risk assurance universe or audit universe can be a very helpful tool to ensure no material gaps in coverage – by either audit or other assurance providers.

Norman Marks (GRC thought leader) explains:

“It’s about stepping back and asking what are the key risks that we should be considering, if not actually addressing. I call these the risks that matter.

I believe that top executives and board should have confidence that through internal audit assurance over the management of key risks, they can assume that everything is working right in terms of managing their more significant risks, unless they’re told otherwise.”

Actions for Internal Audit to consider:

• Does the current audit universe solely concentrate on processes, systems and locations? If so, look at ways to upgrade this to better reflect what the key risk assurance areas should be (e.g. latest objectives and projects);
• Use an expanded audit universe to better understand the completeness of the overall risk assurance picture and coverage by audit.

Gather a Picture of Current Performance as well as Issues, Incidents, “Near Misses” and External Intelligence

A vitally important part of having a value add focus is to look at the performance management information and progress against key value drivers, as well as:

• Other performance metrics (e.g. customer satisfaction levels);
• Incidents (e.g. product defects or recalls);
• Near misses (e.g. a systems interruption);
• Other external data of interest (e.g. developments in a significant market or recent regulatory fines or other issues of note externally).

Gathering a range of information about key value related issues in a disciplined manner can provide huge insights into risk assessments and can also be invaluable when considering potential areas for audit attention.

Nancy Haig (Chief Audit Executive for a global consulting firm) explains the approach of her audit team:

“We also are continuously monitoring the external environment. We’re doing a lot of research – paying attention to what’s going on outside, what might have gone wrong elsewhere, so we can bring that to the table ourselves, and we can present those ideas back, both for management attention, and for consideration in our plan.”

Some organizations, especially in financial services, make considerable use of data analytics as a means of guiding the work of audit. For others, the first step is simply to pull together the basics around all key issues and incidents (over and above past internal audit findings and open internal audit issues), since these can provide warning signs of potential value destruction.

Actions for Internal Audit to consider:

• Consider how performance information, incidents and near misses (as well as other audit and regulatory reviews) can give insights to areas of actual or potential value creation and value destruction;
• Consider how this information should be factored into the audit planning process (and by extension the risk management process);
• Determine whether there is a disciplined approach to gathering relevant external intelligence (e.g. regulatory developments, fines or other news stories) that may shed light on organizational risks.

Examine Carefully Which Issues will Really Impact Value and the External Customer

There has been some excellent research over a number of years in relation to how value is created and also the key reasons that value is destroyed.

Obviously the reasons for value destruction are numerous and include failings in the management of “traditional” risk areas, such as financial, regulatory and operational. However, according to a range of studies (including research by Booz & Company in 2013), the largest source of value destruction is often cited to be the mismanagement of strategic risks, such as the failure to adjust to changing customer demands, the failure to effectively integrate a new acquisition, or the mismanagement of reputational risk.

Taking the perspective of adding value, or protecting value from being destroyed, if the mismanagement of strategic risks results in, say, 60-80% of the major instances of value destruction, why wouldn’t audit resource be allocated to ensure these risks are assured in a similar proportion?

Actions for Internal Audit to consider:

• Examine/research the main reasons for value destruction or reputational damage of relevance to your organization (see Figure 9.1);
• Consider the allocation of audit resource across key risk areas and the reasons for any disconnect between these proportions and actual audit coverage in the audit plan.

Gain Inspiration from the Committee on Internal Audit Guidance for Financial Services in the UK

Another excellent source of inspiration for a more value added approach to audit planning has come from the “Committee on Internal Audit Guidance in Financial services.” It was set up to consider the lessons for UK internal audit functions in the aftermath of the financial crisis of 2007–2008. Recommendations were issued in final form in July 2013, and have been supported by the UK Financial Conduct Authority, representing a new benchmark for internal audit functions in the financial services sector in the UK.

Although the recommendations focus on financial services, they are actually very interesting for internal audit in other sectors. In particular, the recommendations get to the heart of a number of big issues around value. Of particular relevance are the recommendations which say that internal audit functions should consider within their scope:

• The design and effectiveness of governance structures and processes;
• The strategic and management information presented to the board;
• Risk appetite and control culture;
• Key corporate events;
• Risks of poor customer outcomes.

These recommendations closely align with the key value considerations for many organizations (note the explicit mention of the external customer!). Also consider how many conventional audit universe models will miss these areas.

Actions for Internal Audit to consider:

• Whether or not you work within UK Financial Services, familiarize yourself with the recommendations of the UK Committee for Internal Audit guidance in Financial Services;
• Consider how the areas listed in that guidance could impact the value issues for your organization;
• Address any key gaps in the audit universe and audit plan (or assurance from other sources).

Use Audit Planning to Drive Additional Value Add Beyond the Audit Plan

One of the key things I learned as a CAE was how much value can be obtained from the audit planning process over and above the creation of the audit plan itself. For example, a good planning process can also deliver:

• A deeper and more up-to-date understanding of senior management and board perspectives in relation to key and emerging risks;
• An understanding about perceptions around risk appetite and the need for assurances, including differences in perceptions on risk appetite between key stakeholders and questions about assurance roles and responsibilities;
• An opportunity to reflect on themes in relation to past audit findings, alongside other issues, to consider whether cultural issues may be causing problems.

Consequently, it can be beneficial to use the audit planning process to:

• Properly engage all key senior managers and relevant board members in the audit plan, and demonstrate “spin off” benefits in the process itself;
• Gain perspectives in relation to the risk register and risk management process (e.g. “A number of key stakeholders are all worried about a particular risk, or an aspect of a key risk, but the risk they highlight doesn’t really feature in the latest risk assessment”);
• Facilitate deeper discussions about risk appetite and/or current assurances;
• Build closer relations with key stakeholders.

Actions for Internal Audit to consider:

• Consider the added value that the planning process itself is delivering over and above the audit plan;
• Make appropriate adjustments to the audit planning process paying particular attention to the engagement of key stakeholders;
• Be sure to feedback to stakeholders to ensure that the audit planning process is seen to add value in its own right.