Getting Data and Documents in Advance is not Always Straightforward

As I spend time with different audit functions I see a wide range of different cultures they must operate within. Each organizational culture will be influenced by a number of factors, including the country, the type of organization and sector, its history, senior management style, and also the current challenges the organization is facing.

Against this backdrop some audit functions find it takes a lot of time and effort to get the documentation and information they require for the assignment, whilst others encounter little or no difficulty.

The reasons for delays in getting information are manyfold and can include the fact that some audit staff do not communicate early enough with management concerning what information they need. This may be due to a lack of planning on the auditor’s part or because the auditor was only recently given the audit assignment by audit management. Either way, unless “auditees” are given sufficient notice of the information that needs to be provided, it is likely that delays will arise, partly because of the other priorities and resource constraints that “auditees” work within, but also because of practical factors (such as the need to run tailored system enquiries or obtain documents from a filing room).

Additional problems auditors face can include finding that the information supplied is not complete and therefore having to request more information, often just before or at the start of the planned fieldwork. Inevitably, delays in getting data and documentation can result in a lot of effort “chasing after” relevant files and even delay the start of the fieldwork: all constituting Muda.

Process Mapping

Another challenge I hear in relation to the early stages of an audit assignment is the time it can take “making sense” of processes and controls before they are audited in detail. A practice for some audit functions is to engage management about key activities and processes, through discussions or a process walkthrough, and then produce some process mapping documentation that sets out the “as is” situation and the important control activities that then need to be tested in more detail. This process mapping by audit can consume several days depending on the extent of documentation already in place, the complexity of the process, and the auditor who is assigned the task.

Managers and Staff are Often Busy and have Limited Time to Engage in the Audit Process

As assignments progress auditors tell me about the feeling that management and staff are “squeezing in” their support for the audit alongside their day-to-day work. Thus, auditors can often experience being told: “You will need to wait until later today,” or “Can I look into that issue and then perhaps we can discuss it tomorrow?”

The best auditors plan meetings with key staff in advance, but managers and staff are not always reliable in making these appointments, or auditors can find that the time for these meetings is squeezed. Looked at from the perspective of line managers, some comment: “The audit staff were poorly prepared and I spent half my time educating them” or “The auditor kept coming back with additional questions and information requests.”

Even where audit functions are more disciplined with their ways of working, one of the dilemmas auditors experience about “pushing” for information, is the feeling that it will adversely impact relationships with management. There is no doubt that a way to create dissatisfaction is to ignore the reality of the pressures facing managers and staff. However, underlying this can be a question of what constitutes sufficient notice for information requests, and how quickly audit requests should be turned around.

If auditors regularly experience managers and staff struggling to provide information, this can provide important clues about Muda, either within the audit process or management processes. It can also yield clues about potential control environment issues (i.e. over-stretched managers and staff regularly in fire-fighting mode, or poor information management disciplines, or a weak risk and control mindset).

The Impact of Auditor Preferences on Testing

Testing is clearly a fundamental part of what an auditor must do and yet, time and time again, I hear of CAEs who are concerned about this area. Their concerns can range from finding that some members of the team have a tendency to do “auditing by anecdote”, and not do enough detailed fact-finding. At the other extreme can be auditors who get lost in the detail of a particular area out of all proportion to its real importance. Richard Chambers (President & CEO, IIA) makes the following observation:

“I think part of our problem as a profession is that sometimes we have a tendency to over-audit. Sometimes we do things in the audit process to validate things that aren’t really going to be important.”

(See Figure 14.1).

One of the reasons for this can be that some auditors have “pet topic areas” that they like to focus on. These focus areas may be justified, but sometimes the time devoted to a specific area can be influenced by an auditor’s personal interests, or reflect areas they have expertise in, or enjoy, auditing. This is an important topic and comes up regularly in workshops and consulting assignments; the key point being that an auditor’s preference to look at some areas may not match the reality of what is truly important to the organization at that point in time, and therefore constitutes Muda.

“Innocent Until Proven Guilty?”

Another common point that emerges from discussions regarding testing is the question of who has the onus to demonstrate that there are risk assurance issues? Often auditors tell me that they will find a problem, based on a sample of information, but when this is shared with management they will retort with comments such as: “Your sample is too small, I’m sure that’s just a one off problem” or “Yes, but I don’t think this is a problem elsewhere.” And as a result, internal audit can find it is being asked to do more testing, with a bigger sample size about the extent and impact of the issue they believe they have uncovered.

The mindset seems to be that management is “innocent until proven guilty” (as commonly understood in many legal systems), and therefore the burden should fall on audit to prove “beyond reasonable doubt” both: i) there is an issue and ii) that the issue matters. This mindset can result in audit having to devote a considerable amount of time and effort to addressing these questions; raising the question of whether this effort is really adding value.

Audit Tools: A Blessing or A Curse?

I have been fortunate to spend time with a number of audit functions looking at their audit strategies and I also get the opportunity to look at audit functions through my work on External Quality Assessments (EQAs). On occasions I might recommend the need to better leverage audit tools such as audit software or data analytics, since these can save a considerable amount of time and enable audit to “zoom in” on key areas of potential weakness.

With larger audit functions audit tools are normally in place, but may not be delivering all that was hoped for. One auditor summed up the problem by observing:

“Often my assignments are as much about ‘feeding the machine’ as they are about doing the audit.”

I have even seen audit functions, disappointed with one audit software, that then put a lot of effort into migrating to another software only to be confronted with many of the same issues!

Waste Associated with Meetings

At the lean auditing workshops I run, we look at the difficulties that can arise in relation to meetings with management during the course of an assignment. Difficulties can include meetings being cut short or meetings being cancelled, or managers explaining: “I’m really not the right person to speak to about that, you need to speak to Joe.”

Auditors also speak of “side tracking” by managers, who do not answer a question directly and instead talk about other issues. After meetings are completed, some auditors may take several hours to write up the minutes of what happened, or find that a key follow up action that they thought they had agreed with management is not delivered on time.

Another issue that regularly comes up at my lean auditing workshops and during consulting work, is an awareness that sometimes a key point in an assignment does not get properly pinned down, causing disagreements and delays towards the end of an assignment. This can include staff disowning comments made earlier in the audit when in the presence of a senior manager.

What Is a Finding?

Even when auditors have successfully established facts that demonstrate there may be a weakness in control, various further challenges can arise, each of which can undermine audit’s position, or result in additional work:

• A statement that there is a compensating control that “covers” the gap in control that audit has found;
• A statement that whilst there could be a weakness in control, the problem is not that serious and therefore no remediation is required because it is within management’s “risk appetite”; and
• An argument that audit has raised a hypothetical issue that management doesn’t have the time to worry about given other more pressing priorities.

Karen Dignan (CAE, Group Head Office, OMG) offers the following reflections on risk appetite:

“I think that auditors generally could be better at thinking about risk appetite. Because it’s easy to raise points and then hear management saying ‘Why are you raising that point as a finding? We’re not really concerned about that. We are happy to accept that risk and we don’t see it as a key risk’.”

When this is the outcome of an auditor’s work there is a real question whether the audit has added value, especially if management’s views would be shared by board members and/or external customers.

These challenges highlight Muda that can be commonplace during audit assignments: delays, wasted time and effort, with a good portion of the assignment that is actually delivering very little value.

Make the Assignment as Painless as Possible

Lean encourages us to be sensitive to those who are directly, or indirectly, involved in delivering customer value. As a result audit should consider the impact of its assignments. Richard Young (Director, UNIAC) gives his perspective:

“When our best auditors are working on an assignment most staff in the area concerned hardly realize an audit is underway. The auditors understand the context those people work in, being mindful of how they use their time and cutting out the things that aren’t going to add any value. This mindset is crucial when you think of lean auditing.”

Thus, preparation by audit, being mindful of management priorities and being very focused on what really matters are all hallmarks of progressive ways of working. Linked to this is the importance of encouraging management to advise audit about any known or suspected issues or concerns they have and what they are doing about these – it doesn’t add much value for audit to spend time and effort unearthing an issue that management are already aware of!

Actions for Internal Audit to consider:

• Discuss good practices in terms of preparation and assignment approach to ensure that managers are not unnecessarily disrupted during the assignment process;
• Ensure management are explicitly asked to outline known issues or areas of concern and existing or planned remediation actions – paying close attention to what this means for the execution of the assignment.

Aim for a Flow of Data and Documents Through Direct Access

A specific good practice by progressive lean audit functions is the way they get information for assignments, with the minimum of disruption to management and without delays. Different approaches can be adopted, as outlined below:

• Requesting key information and data at the time the assignment scope is issued (e.g. many weeks ahead of the field work);
• Requesting read-only access to key systems and folders, so that audit can access this information for itself;
• Already having direct read-only access to a range of key systems and folders, so access does not need to be requested.

The “direct access” approaches of options b) and c) are likely to be the most efficient. Option b) can sometimes require the audit function to negotiate with a system or data owner to get access to data and information. Option c) can be achieved by agreeing a protocol with senior management and the board that the IT department will, as a matter of routine, grant audit access to systems that are most used or will be covered as part of the agreed audit plan.

Both options b) and c) enjoy a further benefit over option a) since direct access to data and folders gives audit an insight into the Gemba of information storage “in the wild”. This avoids the problem of management doing “window dressing” or “tidying up” before or during an assignment, which is a worry a number of auditors have when they are waiting to receive information.

Actions for Internal Audit to consider:

• Seek direct access to data and documents wherever possible, in order to more closely see the state of information as it is “in the wild”;
• Agree an access protocol with senior stakeholders for systems and key folders if time is being wasted negotiating access on a case-by-case basis.

Consider an Audit Liaison Role and Agree Expectations Around Timelines

As discussed earlier, having a senior sponsor for an assignment can be a powerful way to focus and cut through disagreements and delays associated with scoping and scheduling an assignment. Another role that can greatly smooth the assignment process is an audit liaison role. This will typically be a person nominated by the assignment sponsor or the manager of the area being reviewed. The person nominated to act in a liaison role should support audit before and during assignment fieldwork by organizing meetings, helping to get information and data when needed and supporting audit by escalating issues to the sponsor.

At one of my lean auditing workshops an audit manager said that she had encountered a department that created a liaison role, but observed, half joking, that the “liaison person seems to specialize in keeping us away from anything sensitive, rather than helping us”. Needless to say, if there is a risk of this, the role and responsibilities of the liaison role may need to be explicitly defined.

Alongside a liaison role, it may be necessary to spell out a protocol for the provision of information during an audit assignment. This can relate to the time allowed for a management response to a draft audit report, for example, but can be extended to time expectations for all stages of the assignment – for both managers and auditors. Tellingly, one auditor once remarked to me: “I’m not sure about this timelines protocol idea; of course it would be good to get management to give us information on a more timely basis, but if it applies to both sides I’m not so sure how easily our audit function would meet its side of the bargain!”

Needless to say, being lean in assignment delivery is something that requires improved discipline by both management and audit. Where there is repeated slippage audit functions need to be prepared to escalate this upwards so that timelines are adhered to.

Actions for Internal Audit to consider:

• Within audit, determine the preferred timelines for the key steps of an assignment;
• Seek senior manager or sponsor endorsement of typical timelines;
• Discuss role expectations and timelines with key management contacts at the start of each assignment;
• Agree liaison contacts where this is going to be helpful;
• Note and escalate repeated delays or difficulties.

Prioritize Scope Delivery and Use “If Time Permits”

In the last chapter I discussed the importance of having a clear focus on what, exactly, in an assignment is adding value. However, unless this is very clearly communicated and managed throughout the assignment, it can be easy for an assignment to slip “off track”.

A simple way of keeping a clear focus on what matters the most is to ensure that any assignment scope that is documented is ranked in terms of the importance of the areas under review. Thus, an audit methodology that is lean should encourage auditors to focus, in the first instance, on the most important items within the assignment scope. Auditors should be warned about the pitfalls of starting work on areas they regard to be straightforward (or that they enjoy working on) in order to “get off to a good start”.

Taking a value approach, the early stages of the assignment should be focused on what is most valuable for the organization, and move to other areas of scope when it is clear that the key areas have been properly addressed.

The advantage of this approach, which was adopted by internal audit in AstraZeneca after our work on lean, is that it enables more assignments to deliver to time. This is because, if an assignment runs into difficulties, the areas remaining to be done are likely to be the less important ones, and therefore can be more easily dropped.

Some audit functions explicitly address the fact that the assignment should focus on what is most important by stating that some areas of scope will be covered “if time permits”. Of course, the best auditors will usually deliver all areas of scope within an assignment, but this approach allows for some explicit contingency if there is a risk of an assignment running over budget.

Using this approach can also have some interesting spin off consequences; for example, I have heard of instances when management have asked: “What do you mean, ‘if time permits’? I was hoping you were going to look at this area for me. I want to know what is going on!” This can then allow for discussions about the assignment purpose and value add as well as the role of audit and management’s monitoring role, e.g. “It’s not the role of audit to carry out your monitoring role for you.”

Actions for Internal Audit to consider:

• Rank the assignment scope based on value add;
• Create clear expectations about what areas should be looked at first;
• Consider stating “if time permits” for less important areas in audit scope documents to allow audit to meet its resource and timeline plans.

Deriving Value from Process or Systems Mapping, and Clarifying the Role of Audit in this Regard

It is clearly sensible that auditors should understand the areas that they are working on, and it will often make sense for them to walk through key processes, systems or other activities – assuming these address key areas of potential value loss.

However, if process or systems documentation does not exist, or needs to be significantly updated, there is an important question about whether it should be the role of internal audit to pull together this documentation. Conversely: what accountability do management and staff have to maintain up to date documentation of processes and procedures?

Several choices arise in relation to systems and process documentation:

• To agree that it is management’s role to keep this documentation up to date, and if this is not happening audit can raise a finding immediately that this should be done and documentation updated on an ongoing basis;
• To agree that in the long run it is management’s role to keep this documentation up to date, but that audit will develop a “starter for 10” process or systems maps, which management must then own and keep up to date, (and if they do not do this in the future, audit can raise a finding);
• To agree that in the long run it is management’s role to keep this documentation up to date, but that audit will i) help advise what good process or systems documentation will look like and, perhaps, ii) work jointly with management to prepare this.

My advice is that internal audit should be wary of writing process and systems documentation on a regular basis without considering the question of roles.

However, if auditors do go to the trouble of writing or updating systems and process documentation, this should not just be done “for the audit file” but shared with staff or managers in the area for their ongoing use. “The audit file” is not a customer as far as lean is concerned!

Actions for Internal Audit to consider:

• Agree as an audit function whether the accountability for keeping process and systems documentation is clear;
• Establish a way of working in relation to system and process mapping, ideally moving towards management ownership of this task;
• If audit does any work to update or improve systems or process maps, this work should be shared with management, with an expectation they will keep it up to date in future, as far as possible.

Managing Auditor Pet Topics and Risk Control Matrices

In the last chapter I discussed the role of risk control matrices. These can be used as a tool to help assignments focus on the most important risks and controls (i.e. the key risks and the key controls). They can also be very helpful during assignments. Here are some reflections from a senior audit manager:

“You’ve got to recognize that auditors will feel more comfortable auditing some things rather than others. What you mustn’t have is an audit that’s driven by what an auditor feels most comfortable looking at.

You have to sit down as part of planning and go through with the auditor what’s the risk in this process, what’s the risk in this business unit, whatever it is you have to be looking at. What are the key controls that address those risks? And how would we test for them?

That way you don’t end up testing things that don’t matter.”

Risk control matrices are one tool that can be used so that assignments stay focused on what matters the most. (See Figure 14.2).

However, some words of caution:

• Pre-prepared risk control matrices may not cover key risk areas that have been identified during assignment planning;
• Pre-prepared risk control matrices may include risks and controls that are not really key in the context of the assignment purpose or management risk appetite.

As a result, it is vital that during the assignment planning stage, risk control matrices are modified to:

• Incorporate key risks and controls not included in a standard risk control matrix;
• Exclude risk areas or controls that are not relevant to the scope, or the agreed risk appetite.

The sign of a value adding assignment, without waste, is often the intelligent editing of risk control matrices, rather than slavishly sticking to pre-prepared formats.

Actions for Internal Audit to consider:

• Discuss as a team the use of risk and control matrices;
• Agree disciplines to ensure risk and control matrices are properly focused for each assignment.

Step Up Meeting Disciplines

As outlined earlier, time can be wasted in relation to meetings, either beforehand (e.g. the meeting is cancelled), during (e.g. the manager going off topic or the auditor missing a key point) or after (e.g. as a result of poor follow up). To drive out Muda, auditors need to adopt a proactive approach to address the areas of most concern.

Auditors often find that some of the improvement areas discussed earlier, such as the creation of a sponsor for each assignment, will help reduce significantly instances of meetings being delayed or cancelled. In addition, auditors who want to enhance the value add from meetings tend to prepare better for what they are going to ask. Andy Weintraub (experienced internal audit leader) offers the following advice:

“Do your homework. Go into meetings with an agenda and consider providing it to the audit customer in advance, including some of the detailed questions. That way everyone’s had a chance to think about what we’re going to talk about.”

Phil Gerrard (CAE, Rolls-Royce) highlights additional benefits from better preparation:

“It enhances the credibility of audit when management say ‘I can see from your analysis you’ve done robust planning’. It can also lead to management agreeing to take action, in which case there’s no point doing loads of testing to prove something when management has already accepted it.”

If an auditor finds that meetings are often being cancelled, be prepared to discuss any patterns of cancellations. For example: “I realize that John’s team are busy, but over the past quarter there have been four occasions when they have had to cancel meetings. This is significantly out of line with other departments who are also very busy. What should we do about this?”

The key to this approach is: i) to keep the factual details of what has happened (even a log of meeting bookings and cancellations); ii) to spot the pattern, rather than the individual cancellations, and iii) to use the fact that regular cancellations or rescheduling is out of line compared to other areas, to bring pressure on them, and iv) if necessary, be prepared to escalate further, focusing on the impact on wasted audit resource.

Some CAEs I have worked with on these issues have noted that managers rarely cancel meetings with the external auditor because they will be charged for the lost time. As a consequence, some internal functions have started to threaten charging for cancellations in order to highlight that it is wasteful.

During meetings, the number of auditors in attendance needs to be weighed up. Some audit functions just allocate one auditor to a meeting, and use voice recorders to record what is said. Clearly recording conversations can affect the dynamic of a meeting, and there may also be legal and privacy issues, but it is clearly an efficient way to use time. In other audit functions a second auditor will be a note taker, with the best making notes directly into a computer, so little time is lost writing up notes afterwards.

Thinking about Muda should make auditors mindful of writing up long minutes of meetings. One audit function I have worked with has a rule that the minutes of a meeting should take no longer to complete than the length of the meeting itself. Another audit function has a rule that the key points from any meeting should be summarized in bullet point form and sent to the interviewee within the hour! Some auditors look stunned when I explain this, but I think many managers (and external customers) would be equally stunned to find some auditors can spend half a day writing up minutes of a 90-minute meeting! If we are concerned with maximizing value and eliminating waste there can be no blank cheques, or free resources. Remember Parkinson’s law: “Work expands so as to fill the time available for its completion.”

Finally, one practice we implemented after the lean review of AstraZeneca’s internal audit function, was to spend more time thinking about the quality of our interview questioning and follow-through. In one assignment, I remember sitting in on an interview between an audit manager and a business manager. I explained that I was there to take some notes, but my main purpose was to coach the audit manager after the interview. I also said I would ask a few questions at the end of the meeting if I felt something needed to be clarified.

The interview progressed and I observed and took notes. Towards the end of the interview I asked a couple of questions for further clarification, one of which opened up an important line of enquiry. After the interview, I sat down with the audit manager over lunch and we discussed what had happened. We discussed the new line of enquiry that had opened up at the end, and I explained: “You did the groundwork for that. The manager hinted at the issue in response to your earlier questions, but you seemed to be keen to move on to other areas, so you missed the follow-up question. All I did was to make sure it didn’t get lost.”

Afterwards, we discussed the challenge of knowing when to distinguish between unimportant “noise” and a key “signal” that needs to be pursued. The audit manager recognized that “getting on with the questionnaire” sometimes needs to be put on hold when a key point needs to be resolved.

Similarly, I have observed interviews in which auditors tense up, lean forward or rapidly scribble down notes as a reaction to something that a manager has said. When this happens I can see the manager spotting the auditor’s interest and starting to become more careful about what they say. This sort of subtle behavioural signal would never appear in the minutes of a meeting, but can be one reason an auditor was unable to get to the heart of certain issues. It can also explain why some managers may dislike being audited, since they may feel they are being judged.

My experience is that better meeting management disciplines can significantly impact both the efficiency and effectiveness of an audit function and also improve relationships between auditor and managers. Training courses in this area can be useful, but I strongly recommend coaching “on the job” as well. This has the power of focusing in on the specific strengths and improvement areas of individual auditors and typically has a very notable, immediate impact, alongside improvements to the audit assignment itself.

Actions for Internal Audit to consider:

• Discuss best practices and areas of difficulty in relation to managing meetings;
• Plan questions and identify which are most important before the meeting starts;
• Agree expectations around note taking – encourage quick feedback to those involved, a clarity of key points discussed and actions agreed, and a proactive approach to following up on open points;
• Observe and coach staff on the effectiveness of their interview techniques.

Utilizing Data Analytics

When I discuss lean ways of working with internal auditors, it is often not long before auditors want to talk about the use of data analytics tools (which can, among other things, analyze large quantities of data with a view to finding exceptions and other useful information). There can be no doubt these tools can be useful in the right context and I am sometimes surprised by audit functions that make little use of these tools, when reasonably priced options are increasingly available. Here are the reflections of one CAE:

“I think it is absolutely mandatory that audit should have some capability in audit tools, especially if you are doing financial compliance work. We have saved a lot of time in the team by much more automated testing. It’s stuff that we are asked to do each year so we just run it and report.”

However, the most progressive approaches I have seen exploit this technology, not just simply in routine areas, but as a way of identifying important value opportunities. Leigh Flanigan (CAE, CSIRO, Australia) explains:

“When people talk data analytics their mind immediately goes to financial data. However, there is a lot more data in an organization than that which is financial. Analyzing operational data you can do all sorts of interesting things with data analytics that tell you things about your key performance indicators, operational and strategic.”

Thus, the more progressive audit functions are able to use data analytics outside of the financial area and also to use them as a tool to aid audit planning, not just speeding up testing.

However, the extensive use of data analytics does not automatically equate to being lean. Norman Marks (GRC thought leader) explains:

“Data mining and analysis is just a set of tools. But by the same token talking to people is a set of tools, probably more important than even analytical routines.

You need to have the right mindset. Whether it’s the analysis of big data or going around talking to people, that comes after you’ve identified what you are trying to achieve.”

Thus, an essential message in relation to data analytics is the ability to use these tools in order to deliver value to clients and stakeholders (and ultimately external customers) and not to regard them as “playthings” within the audit function. Linked to this is the need to think carefully about whose role it should be to deploy data analytics. Here are the reflections of Jonathan Kidd (CAE, UK Met Office):

“My ultimate aim is to embed a data analytics capability into finance and purchasing. I believe that management should be doing continuous monitoring and we should just be able to go to them and say okay, it’s that time of year again. Show us your exception reports and what you have done with them. We would get a more continuous controls monitoring culture and proactive approach to remediating issues.”

Encouraging greater accountability in the first and second lines of defence is an important ingredient in improving the risk and control culture of an organization; after all, stopping a fraud is something that is mostly achieved “on the ground” in real time by finance or purchasing procedures, or management supervision. All too often the work of internal audit will take place after a fraud has taken place, which is clearly less valuable than stopping the fraud in the first place.

Actions for Internal Audit to consider:

• Discuss the use of data analytics tools – are these being underused?
• Are any auditors using data analytics as a toy rather than a tool?
• Consider whether the ownership of data analytics should be extended, so that continuous monitoring and auditing can be deployed by other functions (such as purchasing, finance and payroll, etc).

Taking a Step-by-Step Approach to Testing

The use of a prioritized assignment scope and objectives, an understanding of key risks and key controls, and leveraging of data analytics tools can considerably help assignments to be both more efficient and impactful. However, “in the trenches” of an audit assignment, good plans can easily get bogged down in detail and “rabbit holes”. One former senior audit manager makes the following observations:

“There isn’t really any business – large or small – where you can look at the detail of everything, so deciding what to look at is really critical. And the scale of activity means that often times random sampling isn’t really going to be very helpful. You’ve got to be much more focused to say where it would be best to look – and stay focused, as much as possible, in delivering that.”

Leigh Flanigan (CAE, CSIRO, Australia) offers his reflections about how auditors should manage the detail:

“Something I tell my staff is that more work can always be done, but you can’t undo work you’ve already done. If you’ve done some work and you think you’ve found an issue, communicate it up; don’t just do more work and leave the communication to the end.

If you think there’s an issue, you can look into it better by engaging with the right people in the organization. Then, if more work needs to be done, you can do it; but if no more work needs to be done, it’s down tools.”

Andy Weintraub (experienced internal audit leader) continues:

“From my perspective, every stone doesn’t have to be uncovered. Every single detail doesn’t have to be figured out. If you’re testing and you run into a question or something’s not adding up then you’ve got to use your judgment and know when to dig a bit more and when to let go.

To help strike the right balance it can really help to have checkpoints in the audit team. Here’s what we found so far and this is what we’re planning to focus on now. Great. Go design your tests. Great. How many are you going to look at? What are the attributes you’re going to look at? Great.

It’s not micromanaging an assignment, but rather it’s having key checkpoints to make sure that auditors are on track, that they’re going down the right direction and not getting bogged down in details.”

Chris Baker (Technical Manager, IIA UK) provides the following perspective:

“IIA Standards say that you need to gather sufficient evidence and have sufficient relevant information to be confident about what you are concluding and in order to be able to express an opinion.

The basic principle is clear: you’ve got to do enough work and gather enough information and interpret and analyse that information to form a view.

That’s often translated into a whole load of advice about how many records you need to look at and how many tests you need to do to substantiate everything, when, in point of fact, when we are focusing on risk and adding value it should be different from that.

It’s wrong to stick to sample requirements in a rigid way.”

Actions for Internal Audit to consider:

• Review the audit methodology and practice in the audit function around testing;
• Encourage early engagement with management;
• Instill discipline in the testing process to appropriately balance rigour and efficiency.

Keeping Your Eyes Open for “Killer Facts”

Whilst one of the key ways to drive efficiency in audit assignments is to adopt a more focused and step-by-step approach to testing, it is important to recognize this does not mean audit should abandon working in the detail. An experienced Health & Safety auditor provides the following reflections about the challenge of auditing to the right level of detail:

“If you regard lean auditing as simply a ‘doing less’ approach, you might fail to test something in sufficient detail and therefore give false assurance. I can think of at least two or three examples in my career where simply doing a high level review would have suggested a management system in place, but below this were quite serious issues. As auditors we know that sometimes it’s only by doing the deep dive down to the transactional level of analysis that you are going to uncover some gaps in control, or improvement opportunities.

In a nutshell, you can have the wool pulled over your eyes if you don’t go deep enough, but this fear does not mean you should check absolutely everything.”

Richard Chambers (President & CEO, IIA) offers his perspective on striking this balance:

“I would never advocate that internal audit ease its vigilance for fraud or other irregularities, even in the course of narrowing objectives or scope.

However, all too often we end up trying to tell someone how to build a watch – when all they really need to know is the time of day.”

Another perspective on the dilemma of keeping your eyes open, without getting distracted, is to look out for a “killer fact” that will grab the attention of stakeholders, over-turn counter-arguments, and galvanize appropriate action. To illustrate, one CAE explained that for some audit reports he would ask his auditors to include copies of key documents (e.g. key documents with missing signatures), or photographs (e.g. confidential documents visible on a desk), to demonstrate that audit had found something indisputable and important, that needed to be discussed and/or remediated.

A “killer fact” need not simply be a piece of conventional audit evidence. It could also be other internal data and information, or even come from external sources. For example, if audit finds a problem with disaster recovery arrangements, a typical management response can be: “Well you’ve just highlighted a hypothetical issue, why should we spend time and effort on something that might happen?” The “killer fact” response could be: “This is not a hypothetical issue. In our organization, in the last year, there were six occasions when systems experienced service interruptions” or “This issue is not hypothetical. Major continuity issues have happened in three organizations in our sector in the last year, with the following impact…”

Thus, having a value oriented approach to audit testing and findings recognizes that value does not simply come from having bigger sample sizes: finding 40 issues out of a sample of 100 is not necessarily twice as valuable as finding 20 issues out of a sample of 50. In fact, sometimes it’s about finding the right 10 issues out of 25 that will enable audit to: i) find the root cause of an issue and ii) persuade management to take action.

Actions for Internal Audit to consider:

• Never underestimate the power of a good example (i.e. “killer fact”) to highlight an area for improvement and generate management action. These should be captured in even the shortest audit reports;
• Review some recent audits and identify how clearly these “killer facts” are reflected;
• Discuss ways of working when doing testing work to maximize the use of “killer facts”.

Using Root Cause Analysis to Add Value and Streamline Reports

With one of my CAE clients we noticed reports that were becoming too lengthy. I discussed this with the auditors concerned and we identified – amongst other things – the need to improve the way in which findings were being examined for their root causes. In one case we reduced 25 “findings” down to seven key root cause issues, and as a result slimmed the report in half.

Root cause analysis has become an increasingly important area in my consulting and training. Shagen Ganason (former Chief Assurance officer at Department of Conservation, New Zealand) has seen the same benefit from using root cause analysis:

“Root cause analysis reduces the number of findings. Because instead of having ten findings you probably have three findings with common root causes. If you are able to identify the proper root causes you can then actually combine a lot of the findings, which leads to shorter more impactful reporting.”

A related point is that audit actions allocated to relatively junior staff are often unlikely to be addressing root causes. At AstraZeneca we had a minimum level of seniority within the management structure that we aimed to agree actions with to try to ensure: i) the right level of management engagement and ii) that the underlying causes for issues were considered and dealt with.

Norman Marks (GRC thought leader) considers other more strategic benefits from root cause analysis:

“I think that reporting the findings in terms of symptoms and then stopping is ridiculous. If you just report the reconciliations are not being done, without asking five or six more questions that may be needed to identify the root cause, the issues don’t go away. You’re actually not curing the patient. You’re just pointing out the problem.”

Thus, root cause analysis, if done well, can not only result in streamlined reporting, but also start to get to the heart of important issues in relation to GRC and assurance matters. This is also why an analysis of common themes from audits can be powerful, since it might point to common root causes in the risk and assurance culture. Addressing these can drive improvements in organizational efficiency and effectiveness, for the benefit of all key stakeholders.

Of course, root cause analysis can sometimes add time onto an assignment so there may be judgments to be made about how best to do this. However, if auditors approach their assignments with a root cause mindset this does not need to be a significant additional burden. Here is some advice from Phil Gerrard (CAE, Rolls-Royce):

“When planning an assignment think about what the root causes might be and implant those in the minds of your auditors. So at least they are clear about what you mean when you discuss this with them.”

Actions for Internal Audit to consider:

• Look at some recent audit reports and consider whether the real root causes have been clearly identified;
• Consider how clear the audit methodology is on root cause analysis and how clearly it is understood and applied by auditors in practice;
• If relatively junior persons are normally tasked with actions, consider whether this is a sign that root causes are not being identified;
• Pay close attention to issues, incidents and audit themes since they may point to underlying cultural issues that have not yet been addressed in the organization.