Chapter 6. Auditing and Understanding System Controls

The following exam domains are partially covered in this chapter:

Domain 1—The Process of Auditing Information Systems

Domain 2—Governance and Management of IT

Domain 3—Information Systems Acquisition, Development and Implementation

This chapter covers the following topics:

Images Audit Universe and Application Auditing: This section describes how an audit universe can be used to define the range of audit activities and auditable entities to be covered in an application audit.

Images Programmed and Manual Application Controls: This section defines, compares, and contrasts the differences between programmed and manual controls.

Images Auditing Application Controls: This section describes different types of application controls and techniques used when auditing applications.

Images Auditing Systems Development, Acquisition, and Maintenance: This section describes the systems development life cycle and its phases.

Images Business Application Systems: This section describes different types of business systems and why they are important.

System controls and related application topics in this chapter apply to multiple domains of the CISA exam. This chapter examines elements in Domain 1, “The Process of Auditing Information Systems,” that touch on creating a risk-based approach to application audits. It also explores the continuous auditing concepts covered in Domain 2, “Governance and Management of IT.” Finally, this chapter explores elements of Domain 3, “Information Systems Acquisition, Development and Implementation,” related to the definition and development of application controls.

To understand this convergence of domain elements, this chapter discusses application controls and how these controls can be audited to verify their functionality. This chapter also examines business application systems and looks at the risks such as those related to electronic funds transfer and electronic banking.

“Do I Know This Already?” Quiz

The “Do I Know This Already?” quiz allows you to assess whether you should read this entire chapter thoroughly or jump to the “Exam Preparation Tasks” section. If you are in doubt about your answers to these questions or your own assessment of your knowledge of the topics, read the entire chapter. Table 6-1 lists the major headings in this chapter and their corresponding “Do I Know This Already?” quiz questions. You can find the answers at the bottom of the page following the quiz and in Appendix A, “Answers to the ‘Do I Know This Already?’ Quizzes and Review Questions.”

Table 6-1 “Do I Know This Already?” Section-to-Question Mapping

Foundation Topics Section

Questions

Audit Universe and Application Auditing

1, 6

Programmed and Manual Application Controls

2, 7

Auditing Application Controls

3, 8

Auditing Systems Development, Acquisition, and Maintenance

4, 9

Business Application Systems

5, 10


Caution

The goal of self-assessment is to gauge your mastery of the topics in this chapter. If you do not know the answer to a question or are only partially sure of the answer, you should mark that question as incorrect for purposes of the self-assessment. Giving yourself credit for an answer you correctly guess skews your self-assessment results and might provide you with a false sense of security.


1. Which of the following is not a criterion typically considered when assessing risk during an IT audit?

a. Core business

b. Regulatory compliance

c. Market risk

d. Brand

2. The act of balancing debits, credits, and totals between two systems is what processing control technique?

a. Manual recalculations

b. Programming controls

c. Reasonableness verification

d. Reconciliation of file totals

3. Which part of the ACID test means to ensure that transactions are processed only if they meet system-defined integrity constraints?

a. Atomicity

b. Consistency

c. Isolation

d. Durability

4. In which stage of the audit controls and quality assurance checks does an auditor review how the system handles erroneous input and data?

a. Software acquisition process

b. Feasibility

c. Design and development

d. Requirements definition

5. Which of the following is a technology designed to facilitate the exchange of data between computer systems?

a. ATM

b. EDI

c. EFT

d. Application system

6. True or false: An audit universe is not required in the audit process planning stage.

a. True

b. False

7. Which of the following is not an example of data file control categories?

a. System control parameters

b. Static data

c. Reconciliation totals

d. Transaction files

8. What typical control should be observed to verify that run-to-run totals are reconciled on a timely basis?

a. Separation of duties

b. Report distribution

c. Input authorization

d. Balancing

9. In which stage of the audit controls and quality assurance checks does an auditor evaluate how the chosen solution meets the user’s needs?

a. Requirements definition

b. Feasibility

c. Design and development

d. Software acquisition process

10. The data architecture needed to support a business intelligence solution includes which of the following components?

a. Data presentation

b. Data access

c. Data mining

d. All the above

Foundation Topics

Audit Universe and Application Auditing

An audit universe represents the range of audit activities and auditable entities to be covered. An audit universe is typically refreshed annually as part of the audit planning phase. An auditable entity is the part of a business that has sufficient risk to audit. For example, account receivables within the accounting department may be an auditable entity.

An audit universe is not required in the audit process planning stage. However, the audit universe concept is a useful tool for identifying focus areas for an application audit. It allows an individual to create a risk-based application audit approach by systematically breaking down the business environment into key functions and processes and then rating the risk of each.

Let’s consider a few steps in a simple e-commerce transaction:

1. The customer must browse the merchant’s online store.

2. The customer must complete an order form and pass on his or her personal information and credit card number.

3. The customer must review and approve the form.

4. The credit card information must be authorized through the bank’s payment system.

5. The merchant must send a notice to the customer that the order was received and is being processed.

6. The bank must send payment to the merchant.

7. The order is shipped.

Even this high-level overview illustrates the level of complexity that an auditor must understand and the many controls related to the business. Each process is further broken down and can be highly complex. Issues such as when an order was processed, where the customer’s data is stored, where the product is shipped from, and jurisdictional tax requirements are all relevant concerns. Without this knowledge, an auditor cannot verify that proper security measures have been implemented. Think of an audit universe as a means of organizing this business understanding.

Once key functions and processes are mapped, they must be ranked based on risk. There is no one method of risk ranking, but the common objective of risk ranking is to understand the impact that a system failure or security breach would have on the organization. Three criteria are typically considered when assessing the risk to functions and processes:

Image

Images Core business: Will the core business cease or be significantly disrupted (typically measured in loss of revenue)?

Images Regulatory compliance: Will the business or officers be subject to significant legal liabilities, including fines and potential lawsuits?

Images Brand: Will significant disruptions impact brand and market share?

Once the risk is ranked, the higher-risk entities are assessed. Such assessment generally includes a review of controls related to relevant applications, processes, and resources that collectively contribute to the achievement of the strategic objectives.

An audit universe can be used broadly to identify higher-risk functions and which parts of an organization carry significant risk. This chapter focuses on applications and related controls. From the high-risk functions and processes identified, you would then identify the underlying applications and controls to audit.

A risk-based audit plan would be developed through consultation with senior management and business planners. This audit universe approach also allows an audit to leverage the understanding of risk to identify key focus areas of the audit, such as controls related to the following:

Images Supporting technologies

Images Alignment with business and technology strategies

Images Compliance with laws, rules, and regulations

Images Effectiveness of organization support

Programmed and Manual Application Controls

This section discusses programmed (also referred to as automated) and manual application controls as they relate to input, output, and processed information. These controls can be either manually or automatically programmed:

Images Programmed, or automated, controls include validation and edit checks, programmed logic functions, and controls.

Images Manual controls are those that the staff manually verify, such as the review of reconciliation reports and exception reports.

Answers to the “Do I Know This Already?” Quiz:

1. C;

2. D;

3. B;

4. C;

5. B;

6. A;

7. C;

8. D;

9. B;

10. D

The purpose of automated and manual controls is to verify the following:

Images The validity of data processed is ensured.

Images The accuracy of data processed is ensured.

Images The data is stored so that controls maintain the security of the data so that accuracy, validity, confidentiality, and integrity of the data are maintained.

Images Processed data is valid and meets expectations.

Business Process Controls

Before controls can be examined, an auditor must understand the business strategy and the business process. The information gathered as part of the audit universe risk ranking should be leveraged as the starting point. This should include a review of the company’s business plan. Next, the auditor should review in more detail the long-term and short-term business goals.

Long-term business goals are considered strategic and focus on activities planned for the next three to five years. Short-term business goals are tactical and address immediate concerns that are no more than 18 months into the future.

After reviewing this background information, examine process flowcharts and determine key applications and controls. Next, review application controls, data integrity controls, and controls for business systems. While not an exhaustive list, Table 6-2 provides examples of controls.

Image

Table 6-2 Business Process Control Examples

Controls

Processing Controls

Input Controls

Input authorization and Batch Controls

Processing Controls

Processing, Validation, and Editing

Output Controls

Logging and Security signature

The following sections look at each of these categories of controls in more detail.

Input Controls

When reviewing input controls, an auditor must ensure that all transactions have been entered correctly. Whatever controls are used, they should be capable of checking that input is valid. This is important because in many automated systems, the output of one system is the input of another. Data should therefore be checked to verify the information from both the sending and receiving applications.

Controls can have either automated authorization or manual authorization. For example, consider the last time you were at your local discount store and, at checkout, an item did not ring up at the advertised sale price. Most likely, you had to wait for the clerk to signal a supervisor to advise of the error in the sale price. The supervisor then had to enter a second-level password to authorize the price change. This is a manual authorization control. Other types of authorization controls include these:

Images Signatures on forms or documents approving a change.

Images Password controls that are required to process a change.

Images Client identification controls that allow only certain clients to authorize the change. (For example, the clerk at the store cannot authorize a price override, but the manager can do so by using a special access login.)

A batch control is another type of input control. Batch controls combine transactions into a group. The group then has a value assigned. The total of this transaction can be based on dollar amounts, total counts, total document numbers, or hash totals.


Note

The CISA exam will expect candidates to understand the difference between a batch total and a hashing control.


Total dollar amounts verify that each item totals up to the correct batched total amount. Total item counts verify the total counts match. For example, if 312 items were ordered, 312 items should have been processed and shipped. Total document numbers verify that the total number of documents in the batch equals the total number of documents processed. Documents could be invoices generated, orders, or any document count that is used to track accuracy. Hash totals are generated by choosing a selected number of fields in a series of transactions. These values are computed again later to see if the numbers match. An incorrect value indicates that something has been lost or entered incorrectly or that the transmission has been corrupted somehow. As an example of hashing, consider that the totaling of part numbers on an order normally provides no usable value, but the total can be compared to the shipping order to verify accuracy.


Note

Hash totals can be thought of as working similarly to the way that cryptographic hashing algorithms (such as MD5) work to verify integrity. In this case, a hash is taken on the entire document. Any alteration or change in the document will result in the hash being different between the sending and receiving points.


Batch controls must be combined with the proper follow-up procedures. For example, if rejected items are resubmitted, controls must be in place to detect this anomaly. In addition, procedures must be in place to follow up on any discrepancies found when batch controls are performed. Batch balancing is used to verify that a batch was processed correctly. It can be accomplished by comparing computer-generated batch quantities to manual batch counts. Control accounts can also be used. Control accounts write an initial batch value to a data file. After processing, the processed value is compared to the initially stored value. An example of this can be seen in batch registers, which allow for the manual recording of batch totals. These are saved and then compared to totals that are generated by the system. If they do not agree, a batch can be rejected.

Processing Controls

Processing controls are used to ensure the accuracy, completeness, and timeliness of data during online or batch processing. Controls should be in place to verify that data is processed only through authorized routines. Processing controls should be designed to detect problems and initiate corrective action. If procedures are in place to override these controls, their use should be logged. Individuals who have the ability to override these controls should not be the same ones responsible for reviewing the log. Edit controls can be used after the data has been entered into the system but before it has been processed. Edit controls can be considered a type of preventive control. Table 6-3 describes edit control examples.

Image

Table 6-3 Processing Edit Control Examples

Validation Edit

Description

Sequence check

Sequence numbers ensure that all data falls within a given range. For example, checks are numbered sequentially. The day’s first check that was issued had the number 120, and the day’s last check was number 144. All checks issued that day should fall between those numbers, and none should be missing.

Limit check

Data to be processed should not exceed a predetermined limit. For example, the weekly sale item is limited to five per customer. Sales over that quantity should trigger an alert.

Range check

A range check ensures that the data is within a predetermined range. For example, a range check might verify that the data is after 01/01/2017 and before 01/01/2025.

Validity check

This type of check looks at the validity of data.

Reasonableness check

This check verifies the reasonableness of the data. For example, if an order is usually for no more than 20 items and the order is for 2,000 items, an alert should be generated.

Table look-ups

This check verifies that the data matches the data in a look-up table.

Existence check

An existence check verifies that all required data is entered.

Key verification

Key verification requires a second employee to re-enter the data. A match must occur before the data can be processed.

Check digit

A check digit verifies accuracy. A check digit is a sum of a value appended to the data.

Completeness check

This check ensures that all required data has been added and that no fields contain null values.

Duplicate check

This check ensures that a data item is not a duplicate. For example, before payment is made, accounts payable verifies that invoice number 833 for $1,612 has not already been paid.

Logical relationship check

This type of edit check verifies logic. If one condition is true, additional items must also be true. For example, in 2017, if the data shows that an applicant is old enough to vote, logic dictates that person must have been born before 1998.


Note

The CISA exam typically includes questions on preventive versus detective controls. Validation edit controls are applied before processing the transaction, and they are therefore considered preventive controls.


Now that you understand edit controls, let’s turn our attention to processing controls used to ensure that the data remains unchanged until processed by an authorized process. Table 6-4 outlines the control techniques used to protect the integrity of the data.

Image

Table 6-4 Processing Control Techniques

Processing Control

Description

Manual recalculations

Some transactions might be recalculated to ensure that processing is operating correctly.

Editing

A program instruction controls input or processing of data to verify its validity.

Run-to-run totals

Various stages of processing ensure the validity of data.

Programming controls

Software-based controls flag problems and initiate corrective action.

Reasonableness verification

This control ensures the reasonableness of data. For example, if someone tries to process a negative amount through a payment system, a reasonableness control should flag the result as invalid.

Limit checks

This control sets bounds on what are reasonable amounts. For example, someone might attempt to order 55 flat-screen TVs.

Reconciliation of file totals

This refers to the act of balancing debits, credits, and totals between two systems. This control should be performed periodically to verify the accuracy and completeness of data.

Exception reports

This type of report should be generated when transactions appear to be incorrect.


Note

The CISA exam typically includes both definition questions on the different controls and situation questions about which is the best control. For example, a customer has entered an order for 55 TVs in error instead of the intended 5 TVs to be ordered. Would a limit check or exception report be the more effective control? A limit check could catch this type of error because it sets a practical limit on what a typical customer would buy and can force an out-of-band order to be manually verified through customer service. Exception reports would only detect an error in the transaction itself, such as an invalid shipping address.


Data File Controls

Data files or database tables can be put into categories to better understand their content and purpose. There are many data category types. The following are four typical examples:

Image

Images System control parameters control values such as how much money can be transferred in a single transaction with approval.

Images Static data refers to information that changes infrequently. An example of static data is a customer’s name, address, and phone number. Because these values do not frequently change, an alteration should be controlled and should require authorization.

Images Balance data refers to various values and totals that might be held temporarily during processing. These values should be strictly controlled; any manual alteration of these values should require authorization and should be logged.

Images Transaction files deal with the transmission of information between two systems or applications. Transaction files should be managed with exception reports or validation checks.

Each data category can then have various controls applied, such as the following:

Images Before-and-after image reports: By taking a snapshot of computer data before and after a transaction, you can determine what changes have been made to the data. There are applications that can apply snapshots automatically and send an alert when a file changes. These automated tools are useful for detecting changes to files outside normal change windows, such as changes to a security configuration file, which may be evidence of a breach.

Images Maintenance error reporting: Procedures should be established to verify that any errors or exceptions are rectified and reconciled. The employee who generated a transaction should not have the authorization to clear an error with it.

Images Internal and external labeling: Internal labeling ensures that proper data files are used and that all information is present. External labeling applies to removable media.

Images Data file security: This control ensures that individuals who process data cannot bypass validity checks, clear logs, or alter stored data.

Images One-to-one checking: Documents entered for processing should match processed totals.

Images Transaction logs: Transactional information such as date, time, terminal, user ID, and so on should be recorded to create a usable audit trail.

Images Parity checking: Data integrity should be verified in the event of a transmission error.

Output Controls

Output controls are designed to provide assurance with data that has completed processing. Output data must be delivered accurately and in a timely manner so that it is useful in the decision-making process. Output controls should be designed to ensure that data is distributed and stored in a secure manner. Sensitive data should have sufficient controls to monitor usage and control who has access to it. These controls will vary depending on whether the information is centrally stored or distributed to end-user workstations.


Note

Auditors should be aware that one way to control distribution is to place controls on data that limit what information is to be printed and to whose printer it should be directed. For example, some reports might be configured so that they are printed only to the supervisor’s printer. Software printing controls are another example of this. Products such as the Adobe Reader can be used to limit printing or embed password controls for viewing or printing.


Auditing Application Controls

Application software is the engine behind business transactions. Business transactions today are fast paced and are often measured in sub-second response time. To remain competitive, businesses must automate many customer transactions. This keeps costs controlled and enhances reliability—and it means increased use and integration of application software.

Two control terms are often used:

Images Application controls: These controls are related to a specific individual process or application.

Images General controls: These controls apply across all system components, processes, and data.

These systems might process payroll, manage inventory, or even invoice and bill customers. Most users see only the application interface, but what does the application really do? An auditor should be most concerned about the limits, controls, and rules that define how an application interacts with the organization’s data.

Understanding the Application

One of the first questions an auditor should ask is “What does the application do?” How this question is answered will vary from organization to organization and from business to business. The audit universe and related risk ranking are important tools that help focus an auditor on higher-risk applications and transactions.

An auditor can start by asking for documentation. If an application was developed in-house, system development methodology documents might be available. These documents will provide the auditor with insight into what the user requirements were and what cost–benefit analysis was done to justify development of the application. Functional design specifications should also be reviewed. These specifications are a great resource because they detail how the application is designed and what it was developed to achieve. If the application has been in use for a while, program change documents that list any changes or updates also might exist. Checking outstanding bugs and issues in release notes documentation and software build information also can provide good information. After reviewing these documents, the auditor can develop an application flowchart to accomplish the following:

Images Validate every input to the system against the applicable criteria

Images Review logical access controls and authorization controls

Images Evaluate exception handling and logging

Images Examine data flow to find control weaknesses

With an understanding of the application and knowledge of how transactions flow through the application and related risks, the auditor can move to the next step: observation and testing.

Observation and Testing

A big part of an auditor’s job is observation. Auditors are tasked with observing how users interact with an application. Auditors should also test the limits of the application, such as buffer overflows. Buffer overflows can occur when attackers try to stuff more than the total number of characters allowed in a field. The design specification might state that the application does not accept negative numbers, but is that actually the case? If you enter a negative number or a negative quantity in a field, will the application actually accept it? It shouldn’t.

Even if the application were to accept an invalid input activity, reports should track the IP address and device name used to complete the entry. Logging should also track the date and time of activity. In addition, invalid entries should be logged so that violation reports can be created.

When working with applications, auditors should observe and test the items listed in Table 6-5.

Image

Table 6-5 Observation and Testing

Observation or Test

Details

Separation of duties

Auditors should verify separation of duties, which provides control by limiting the ability of each employee. For example, one department might have the capability to issue checks but might be required to send the checks to a second department for signatures.

Input authorization

Auditors should review records to verify who is authorized to access applications. Supervisor override being used frequently might signal problems.

Balancing

Auditors should verify that run-to-run totals are reconciled in a timely manner.

Report distribution

Auditors should review report distribution logs to see who has access to view and print reports. Controls used to limit the distribution of reports should also be reviewed.

Error correction and control

Auditors should review past error corrections and verify that they are viewed and addressed in a timely manner.

Access control and authorization

Auditors should verify that access is limited to individuals who have a clearly demonstrated need. Testing can be performed to ensure that access controls are in place as specified.

Data Integrity Controls

Data integrity testing is performed to ensure the accuracy, completeness, consistency, and authorization of data. Integrity testing is considered a substitutive test. Data can be stored in files or in databases.


Note

Data integrity testing is the best method of examining the accuracy, completeness, consistency, and authorization of data. Data integrity testing can be used to find failures in input and processing controls.


Data stored in databases has unique requirements because it differs from data stored or processed by an application. Database integrity testing can be performed using several methods. Referential integrity guarantees that all foreign keys reference existing primary keys.


Note

If you are not familiar with the database terms primary and secondary keys, you should supplement your study with database concepts by searching for related whitepapers on the Internet.


Controls in most databases should prevent the primary key from being deleted when it is linked to existing foreign keys. Relational integrity ensures that validation routines test data before it is entered into a database and that any modification can be detected. A third integrity control is entity integrity. For example, if the primary keys are names of banks, entity integrity ensures that each database transaction record contains a primary key. Without the capability to associate each primary key with a bank, entity integrity cannot be maintained, and the database is not intact.

Online data integrity has somewhat different concerns because online databases most likely are distributed or clustered for performance and fault tolerance. Online databases work in real time. This might mean that several databases must be updated simultaneously. These complexities mean that the ACID test should be applied:

Image

Images Atomicity: Ensures that the results are either all or nothing.

Images Consistency: Ensures that transactions are processed only if they meet system-defined integrity constraints.

Images Isolation: Ensures that each transaction is isolated from all others until complete.

Images Durability: Ensures that when a transaction is processed, the transaction cannot be rolled back and is accurate.

Application System Testing

Application testing is a critical part of the audit process. Testing enables an auditor to evaluate a program, review its controls, and monitor the transaction process. The primary methods used for application testing are as follows:

Images Snapshots: These are used to monitor and record the flow of data through an application. Although snapshots require in-depth knowledge about the application, they are useful in verifying logic.

Images Mapping: Unlike snapshots, mapping verifies program logic that might not have been performed or tested. This is useful because it might detect undiscovered problems.

Images Tracing and tagging: Tagging is used to mark selected transactions, which enables these tagged transactions to be monitored.

Images Using test data: An auditor can use test data to verify program operation. This technique requires little knowledge of the environment and does not require the review of the source code.

Images Base case system evaluation: This compressive test uses test data that was developed to thoroughly test the environment. This method is useful because it requires great effort and close cooperation among various internal groups.

Images Parallel operation: Both old and new systems process data so that the results can be compared.

Images Integrated test facility: This test method creates a fictitious entity in a database to process sample test transactions at the same time live input is being processed.

Images Parallel simulation: Another useful audit tool uses computer programs to simulate program logic.

Images Transaction selection: This method of testing uses audit software to determine what transactions should be processed.


Note

A number of these test techniques are invasive, which means the test can alter or disrupt normal transactions and reporting. To minimize production disruption, a number of these tests can be applied to the development environment rather than the production environment. In these cases, it is critical that the development environment have the same code set and properly mimic the production environment.


Continuous Online Auditing

Testing a system before rollout might provide a baseline of information, but it offers no ongoing feedback on the operation of the application. Continuous online auditing gives auditors the tools needed to perform ongoing monitoring. Continuous online auditing produces audit results either at real-time intervals or after a short period of time. This method actually can reduce costs because the need for conventional audits might be reduced or eliminated. In a conventional audit, the auditor has a limited amount of time to design tests and examine data. Continuous online auditing greatly increases the quantity and scope of data available to the auditor. With continuous auditing, the auditor can evaluate data on an ongoing schedule and alert management to problems as needed. Paperwork is reduced, and the auditor can electronically examine application data and report problems directly as needed.

Continuous online auditing also increases security. Consider a bank that allows online access to customer accounts and funds. Although such systems are convenient for users, they present an additional risk for the bank. Continuous online auditing allows the bank to monitor transactions as they occur. If some type of misuse occurs, the time between the misuse and discovery is greatly reduced. If additional controls are needed, they can be deployed in a shortened time frame. Five continuous audit techniques commonly exist, as described in Table 6-6.

Image

Table 6-6 Continuous Audit Techniques

Technique

Description

Issues and Concerns

Systems control audit review file and embedded audit modules (SCARF/EAM)

The application must contain embedded audit software to act as a monitoring agent.

Cannot be used to interrupt regular processing

Integrated test facilities

Live and dummy data is fed into the system. The results of the dummy data are compared with precalculated results.

Should not be used with test data

Continuous and intermittent simulation (CIS)

CIS simulates the transaction run. If data meets certain criteria, the simulator logs the transaction; otherwise, processing continues.

Requires examination of transactions that meet specific criteria

Snapshots

This technique tags transactions and then takes snapshots as the data is moved from input to output.

Requires an audit trail

Audit hooks

This technique uses embedded hooks that act as red flags if certain conditions are met.

Detects items that meet specific criteria


Note

Businesses are often nervous about adding interfaces to live production data because they fear system disruptions. Many audit departments work using extracts of production data to reduce such risks.


Auditing application controls is a big part of auditing system infrastructure. Table 6-7 describes the five major phases and activities performed during each phase of application auditing.

Image

Table 6-7 Auditing Applications

Phase

Activity

Understanding the application

Validation of every input to the system against the applicable criteria

Review of logical access control and authorization controls

Evaluation of exception handling and logging

Examination of data flow to find control weaknesses

Observation and testing

Review Separation of duties

Test Input authorization

Review Balancing controls

Evaluate Report distribution

Test Error correction and control

Assess Access control and authorization

Data integrity controls

Test Referential integrity

Evaluate Relational integrity

Evaluate Entity integrity

Application system testing

Obtain Snapshots

Review Mapping

Evaluate Tracing and tagging

Evaluate Use of test data

Assess Base case system evaluation

Examine Parallel operation

Assess Integrated test facility

Evaluate Parallel simulation

Review Transaction selection

Continuous online auditing

Implement Systems control audit review file and embedded audit modules (SCARF/EAM)

Monitor Integrated test facilities

Assess Continuous and intermittent simulation (CIS)

Obtain Snapshots

Review Audit hooks

Auditing Systems Development, Acquisition, and Maintenance

Today’s systems are complex, and systems might be used by different branches located in different areas of the world or accessed by users through the Internet. Many legal regulations and requirements, such as various privacy laws, must be satisfied. This means that coding must be performed by teams of programmers with the help of architects, analysts, testers, auditors, and end users, who must all work together. To manage such a large endeavor, the system development life cycle (SDLC) was created.

The auditor’s role in the SDLC process is to work with the development team to ensure that the development, acquisition, and maintenance processes yield a final product that meets user requirements while possessing adequate controls. Throughout the system development life cycle, an auditor should work with the development team to minimize risks and exposures. The following are the general steps an auditor should follow during the development process:

Image

1. Determine the objectives and user requirements of the project.

2. Perform a risk assessment that identifies threats, risks, and exposures.

3. Assess existing controls to determine whether they will adequately reduce risk to acceptable levels. Discuss any needed changes with the development team.

4. Monitor the development process and evaluate controls as they are designed and created.

5. Evaluate the system during rollout and review audit mechanisms to ensure that they function as designed.

6. Take part in any post-implementation reviews.

7. Verify system maintenance procedures.

8. Review production library control to ensure the needed level of security.

Project Management

Implementing good application controls is just part of the task. Organizations must also use good project management techniques. An auditor should ensure that the overall process is sound and meets industry standards. The involvement of an auditor in a project will vary depending on risk. An auditor will most likely not be involved in every project. Auditors’ involvement at a project level is typically reserved for large projects with significant impact to the organization. Auditors place reliance on their assessment of the overall process to control risks of smaller projects in which they cannot be personally engaged. An auditor must be comfortable that the end result will be to minimize risk and ensure that adequate controls are in place.

An auditor must evaluate the level of oversight that a project committee has over the process. Other issues, such as reporting, change control, and stakeholder involvement, are also important. Table 6-8 lists additional checks that the auditor should perform. Although this is not a complete list, it should give you an overall idea of how important it is for the auditor to play a proactive role in the process.

Image

Table 6-8 Audit Controls and Quality Assurance Checks

Phase

Items to Review

Feasibility

Examine proposal and documentation.

Assess the criticality of the user’s needs.

Evaluate how effectively the chosen solution meets the users’ needs.

Investigate the possibility of an alternate or existing solution.

Requirements definition

Assess the total cost of the project and verify that the project sponsors have approved.

Examine the conceptual design and verify that it meets user demands.

Evaluate the possibility of embedded audit routines.

Examine the proposed user acceptance plans.

Software acquisition process

Examine the RFP to ensure that it is complete.

Examine vendor contracts.

Verify that the legal department has approved the vendor contract.

Design and development

Study system flowcharts.

Evaluate input, process, and output controls.

Examine proposed audit trails and determine the usefulness.

Review how the system will handle erroneous input and data.

Testing

Examine proposed test plans.

Verify audit trails, error processing, and error reports.

Evaluate user documentation and manuals.

Review test results.

Examine system security.

Implementation

Examine system documentation.

Examine system parameters.

Examine any data conversion activities to verify correctness.

Post-implementation

Review requirements and user needs to verify that the systems meet user needs.

Examine user satisfaction and cost–benefit analysis.

Examine the change-request process.

Examine error logs.

System change procedures

Determine whether emergency change procedures exist.

Evaluate the separation of production code from test code and access security controls.

Interview end users to determine their satisfaction with the turnaround of the change process.


Note

A CISA candidate needs to understand what activities occur at each stage of the project management process. For example, user acceptance test plans are reviewed at the requirements definition stage.


Business Application Systems

A business application is any program that is used to run a business. As discussed previously, not all applications have the same importance. A risked-based audit approach looks at the business applications that represent the greatest risk to the business. This means that auditors must understand the business and technical context of an application that is being reviewed.

One good place to start is reviewing application system flowcharts. Business applications can be categorized according to where they are used or by their functionality. Business application programs are used for accounting, payroll, inventory, sales, and so on. These systems can be used in e-commerce systems, for web-based applications, in electronic banking, and even for electronic payment systems. This section discusses a number of different business application systems an auditor will encounter.


Note

Flowcharts are one of the first things an auditor should examine to get an understanding of an application or business function. It’s not uncommon for flowcharts and application specifications to become outdated. The maintenance of this material may be an audit concern that is raised with management.


E-commerce

Electronic commerce (e-commerce) is about the buying, selling, and servicing of goods via the Internet. The process usually begins with a company advertising its goods on a website. When a buyer finds the goods he or she is looking for, the buyer adds them to a shopping cart. Upon checkout, the buyer is redirected to a secure web page so that credit card and shipping information can be entered.

E-commerce saw tremendous growth in the late 1990s with the wide adoption of the Internet. This created an opportunity for businesses to offer goods and services without the traditional overhead and at better prices than in brick-and-mortar stores. Although a pure Internet model has somewhat held, many companies have a bricks-and-clicks model that supports both online and offline presences.

The following are the different types of e-commerce transactions:

Images Business to business (B-to-B): Transactions between two or more businesses, such as a business and its suppliers.

Images Business to consumer (B-to-C): Transactions between businesses and consumers. This area is one of the greatest growth areas for e-commerce. For companies that don’t sell products directly to their customer, there are brokers that can sell products for them.

Images Business to government (B-to-G): Transactions between businesses and governments, such as the online filing of legal documents and reports.

Images Business to employee (B-to-E): Transactions between businesses and employees. This model can be seen when organizations set up internal websites and portals for employee services such as health care, job benefits, and payroll.

E-commerce adds an additional level of challenge to an organization because data and applications must protect availability, integrity, and confidentiality 24 hours a day, 7 days a week. Companies must also be careful in handling customers’ personal information and payment information, such as credit cards. Authentication and nonrepudiation are important aspects of this because customers need to know that they are really dealing with the company, not an impostor.

Cloud computing has both simplified and created a number of significant challenges for auditors. Cloud computing involves using a network of remote servers hosted on the Internet to store, manage, and process data. In other words, a business no longer has to host its own servers and computing power. Cloud computing can provide major cost savings as businesses do not have to build and maintain their own data centers. Amazon, Google, Microsoft, IBM, and many other technology companies offer cloud computing services. But moving to a cloud service provider also means giving up control. The major cloud service providers have standardized how servers are configured and managed, and this standardization limits choices and can limit auditor access.

Electronic Data Interchange

Electronic data interchange (EDI) is a technology designed to facilitate the exchange of data between computer systems. It was designed to bridge the gap between dissimilar systems. EDI is used to exchange invoices, shipping notices, inventory updates, and so on in a format that both the sending and receiving systems can understand. ANSI X12 is the most common of the formats used. EDI offers benefits for organizations: It reduces paperwork and results in fewer errors because all information is transmitted electronically. Traditional EDI consists of the following components:

Images Communications handler: The handler transmits and receives electronic documents. Much of this activity occurs via the Internet.

Images EDI interface: The EDI interface handles data as it is being passed between the two organizations’ applications. Security controls are usually placed here. The EDI interface is composed of the EDI translator and the application interface.

Images Application system: The application system is the program responsible for processing documents that have been sent or received. Additional controls are usually not placed here.

Electronic funds transfer (EFT) is an example of EDI used among financial institutions, in which money is transferred from one account to another. Examples of EFT transactions include electronic wire transfers, automatic teller machine (ATM) transactions, and direct deposit of payroll via the Internet.

EDI adds a new level of concern for organizations because documents are processed electronically. One big concern with EDI is authorization. EDI processes should therefore have an additional layer of application control to address the issue of authorization, as well as lost or duplicate transactions and issues of confidentiality and invalid distribution. Some common controls include the following:

Images Transmission controls to validate sender and receiver

Images Manipulation controls to prevent unauthorized changes to data

Images Authorization controls to authenticate communication partners

Images Encryption controls to protect the confidentiality of information

Auditors should seek to verify that these common controls have been implemented. Other controls include the deployment of audit monitors, which are devices used to capture EDI activity as it is sent or received. An auditor should also review systems that process inbound transactions to make sure each transaction is properly logged, as well as use transaction totals to verify that totals agree with those collected by trading partners.

Email

Virtually every business uses email to communicate with its employees, business partners, and others. Email enables individuals to communicate electronically through the Internet or a data communications network. Although email is the most commonly used Internet application, it raises some security concerns. Specifically, email is usually cleartext, which means anyone can easily read it. Email can be spoofed to mask the true identity of the sender. Email also is a major conduit for spam, phishing, and viruses.

Users need to be made aware of potential problems and risks with email. Email carries a number of legal and regulatory requirements, which continue to grow. Email commonly uses two underlying services: Simple Mail Transfer Protocol (SMTP) and Post Office Protocol (POP). The following steps describe basic email operation:

Image

1. The user opens an email program such as Outlook to create an email message.

2. After the email is created and addressed to the recipient, the user sends the email.

3. The email is forwarded to an SMTP server, which provides a message transfer agent (MTA). Just as the postal service sorts mail using a zip code, email messages are sorted by domain. For example, in an email addressed to [email protected], the domain is thesolutionfirm, and it identifies where the message is to be forwarded.

4. The MTA forwards the email toward its final destination.

5. The email is delivered to the destination mail server, where it waits until the recipient user retrieves it.

6. The email is retrieved using Post Office Protocol version 3 (POP3) and is displayed via Outlook on the recipient’s computer.

Users must be educated about the fact that sensitive information (such as Social Security numbers) should not be sent by cleartext email. If an organization has policies that allow email to be used for sensitive information, encryption should be used. This requires an evaluation of the business needs. If only some information is to be encrypted, Pretty Good Privacy (PGP) might be the best option. However, if full-time encryption is needed, the company might want to use link encryption or standards such as Secure Multipurpose Internet Mail Extensions (S/MIME) or Privacy Enhanced Mail (PEM).

Business Intelligence

The objective of business intelligence (BI) is to reduce decision-making time and increase the value of a decision. Business intelligence is much like a crystal ball because the organization can use it to make better decisions in a shorter period of time. An organization can use BI to compare itself to its competitors. Business intelligence is also useful in helping understand customer needs as well as the capabilities of the firm. In addition, business intelligence is useful in risk management; it can help a business spot unusual trends, odd transactions, and statistics on loss and exposure. To properly implement an infrastructure to support business intelligence, the business must design and develop a data architecture.

These layers encompass the data architecture:

Images Data sources: The actual data sources reside here. For example, a grocery store might have customer reward card scanners at each checkout so that customers using the reward card have each item recorded on their account.

Images Data access: This layer is responsible for connecting the data sources with the data staging layer. For example, the grocery store might process sales to customers with reward cards to a local database.

Images Data staging: This layer is responsible for copying and formatting data into a standard format for the data warehouse layer.

Images Data warehouse: Data is captured by many databases and organized into subject-oriented usable groupings. For example, the grocery store collects all the data from various stores across the country into this one centralized database. It is then possible to drill up or drill down and obtain information by region or by item.

Images Data mining: Large volumes of data are searched for specific patterns. For example, if a grocery store examines paper plate sales, does it see that the same customers who purchase paper plates also purchase plastic cutlery?

Images Data mart: At this layer resides some type of relational database that enables the user to move the data around to extract specific components. At this point, the user can extract data about data.

Images Presentation layer: This is the top of the model, the point at which users interact with the system. This layer can include such applications as Microsoft Access and Excel.

Together these components provide the structure for a business intelligence system. Once developed, a BI system can be used in different ways, such as for scorecards, customer relationship management systems, decision support systems, document warehouses, data mining, and supply chain management.


Note

Several terms are used to describe large data stores that are used by BI tools. These are two common terms:

Images Data warehouse: A large store of data obtained from multiple sources that is generally used to guide management decisions.

Images Data lake: A large store of raw data stored in its native format until it is needed.

Whereas a data warehouse stores processed data in structured files formats, a data lake generally uses a flat architecture to store raw, unprocessed data.


Decision Support Systems

A decision support system (DSS) helps managers solve problems. A DSS uses models and mathematical techniques and usually is designed with fourth-generation programming (4GL) tools. DSS models share these common characteristics:

Images Used for decision making

Images Used for goal seeking

Images Perform simulation

Images Linkable

Images Perform “what if” modeling

Images Provide time series analysis

The true value of a DSS is its capability to help the user make a better decision. These systems must be flexible and adaptable, but they are not always as efficient as lower-level programming tools might be. DSS models include the following:

Images Model-driven DSS: Uses models based on statistics, finance, or simulation. These are designed to help users make a decision.

Images Communication-driven DSS: Designed to facilitate sharing so that more than one person can work on a task.

Images Data-driven DSS: Can access a variety of internal and external data to analyze outcomes. Companies such as Oracle, IBM, and Microsoft build products that support data warehousing and are some of the leaders in this field.

Images Document-driven DSS: Manipulates and manages unstructured information. eRoom is an example of a document-driven DSS.

Images Knowledge-driven DSS: Based on rules, facts, and knowledge. It is used for problem-solving and to provide answers.

Artificial Intelligence and Expert Systems

Auditors should understand artificial intelligence and expert systems, and they need to know that these systems are used to solve complex problems. An expert system is a computer program that contains the knowledge base and set of rules needed to extrapolate new facts from existing knowledge and inputted data. The Prolog and LISP programming languages, used most in developing such systems, are both considered 5GL languages. At the heart of these systems is the knowledge base.


Note

The difference between 4GL and 5GL can be best understood by how the code is developed.

4GL programming languages create code by defining algorithms and code logic under which the application processes data.

5GL programming languages create code by defining business constraints. The underlying code logic and algorithms are generated by the 5GL language. In some cases, a programmer is not needed to code these business language constraints.


An emerging field in the industry is artificial intelligence (AI), which extends expert systems through self-learning and cognitive processes. In other words, AI systems attempt to think like humans but with the speed of computers. While the definition of AI continues to evolve, and it’s questionable how close to human thoughts it will evolve, AI has made major breakthroughs in recent years. Consider IBM Watson, which today has medical applications. IBM Watson can read tens of thousands of medical journal articles daily and compare its understanding against a patient’s diagnosis. The ability to read and understand a medical journal article is a major AI breakthrough.

The challenges with these systems include ensuring that accurate data is entered into the system, that access controls are in place, that the proper level of expertise was used in developing the system, and that the knowledge base is secure.

Customer Relationship Management

Customer relationship management (CRM) refers to the tools, techniques, and software companies use to manage their relationships with customers. CRM solutions are designed to track and record everything you need to know about your customers. This includes items such as buying history, budget, timeline, areas of interest, and future planned purchases. Products designed as CRM solutions range from simple off-the-shelf contact management applications to high-end interactive systems that combine marketing, sales, and executive information. CRM typically involves three areas:

Images Sales automation: Automation of sales force management tasks

Images Customer service: Automation of customer service processes, such as requests, comments, complaints, and returns

Images Enterprise marketing: Automation of business enterprise information, such as trends, forecasts, business environment, and competition

Supply Chain Management

Supply chain management (SCM) is the science of matching buyers to sellers to improve the way businesses can acquire the raw materials they need for the products or services they sell to customers. SCM begins with raw materials and ends with finished goods that have been delivered to the customer. SCM involves five basic components:

Images Plan: Definition of the strategy used for managing resources and monitoring the supply chain

Images Source: The process of choosing suppliers

Images Make: The manufacturing process

Images Deliver: The logistics of moving goods and services to the customer

Images Return: The systems developed to return noncompliant products to the manufacturer

SCM has become more popular as companies have been moving to a global economy with increased competition. The opportunities SCM offers focus on key items in the supply chain process. First is the focus on keeping transportation costs as low as possible while also keeping enough raw material on hand—but no more than needed. With these two items handled properly, production improves as parts and raw materials are available as needed. This helps ensure that products are available to meet customer demand, thereby preventing loss of sales due to product shortages. The key to the SCM process is cooperation between companies in the supply chain and the business. Applying these principles can reduce inventory, increase transaction speed by causing data to be exchanged in real time, and produce higher revenue.

Social Media

Social media has become increasingly important to organizations. The value of an organization’s products and services is not just judged on what’s deliverable but also on what is said. Increasingly, an organization’s reputation is tied to what is said in social media (via Facebook, for example).

Social media departments are commonplace in large organizations. These teams drive value by doing the following:

Images Increasing brand awareness

Images Improving brand loyalty

Images Attempting to convert new customers

Images Enriching the customer experience

Images Performing damage control when negative events occur.

Auditing social media is challenging because what appears on social media is not under the direct control of an organization. Consequently, an auditor needs to keep social media assessments focused on the organization’s social media strategy, how it communicates expected behavior of its employees on social media, and what mechanisms it puts in place to react to social media events. An auditor should be looking for a well-thought-out and well-defined social media approach. Once the social media approach is understood, the auditor can focus on the effectiveness of execution against the strategy. Some key areas of a social media examination include the following:

1. Business social media strategy

2. Execution metrics against strategy such as conversion rates over social media

3. Employee policies and training on expected behavior

4. Capability to monitor social media for references to the organization

5. Response strategy when negative or false social media stories emerge

Chapter Summary

An auditor needs to know how to audit applications, systems, and related processes. This requires a broad understanding of business and convergence of technology to drive value and deliver business objectives.

This chapter explores a risk-based approach through the definition of an audit universe. It discusses how flowcharts help us learn the process and the functionality of an application. This chapter also discusses in detail application input, process, and output controls. The chapter reviews how to test and validate these controls in a variety of ways, including using snapshots, mapping, and tracing and tagging.

This chapter also considers e-commerce and social media risks to the business. It looks broadly at a number of common business applications to help you better understand how technology supports the business, including supply chain management (SCM), customer relationship management (CRM), decision support systems (DSSs), and business intelligence (BI). Collectively, these applications and tools provide a business with the ability to define how data is handled, access limits placed on the data, and understand how data is transformed into useful business information.

Exam Preparation Tasks

As mentioned in the section “How to Use This Book” in the Introduction, you have a couple choices for exam preparation: the exercises here; Chapter 10, “Final Preparation;” and the exam simulation questions on the book’s companion web page (www.informit.com/title/9780789758446).

Review All the Key Topics

Review the most important topics in this chapter, noted with the Key Topic icon in the outer margin of the page. Table 6-9 lists these key topics and the page number on which each is found.

Image

Table 6-9 Key Topics for Chapter 6

Key Topic Element

Description

Page Number

List

Criteria for assessing the risk to functions and processes

236

Table 6-2

Business process control examples

237

Table 6-3

Processing edit control examples

239

Table 6-4

Processing control techniques

240

List

Data category types

241

Table 6-5

Observation and testing

244

List

ACID test descriptions

245

Table 6-6

Continuous audit techniques

247

Table 6-7

Auditing applications

248

List

Steps an auditor should follow during the development process

250

Table 6-8

Audit controls and quality assurance checks

251

List

Basic email operation

255

Define Key Terms

Define the following key terms from this chapter and check your answers against the glossary:

ACID test

application controls

artificial intelligence (AI)

audit universe

automated controls

balance data

batch control

data lake

data warehouse

e-commerce

entity integrity

flowchart

general controls

hash total

long-term business goals

manual controls

short-term business goals

static data

system control parameters

transaction files

Exercises

6-1 Software Application Audit

Estimated time: 60 minutes

In this exercise, you perform a basic audit of a sample of source code.

1. Pick an open source software application on which to perform a security audit. You can use standard Linux utilities or download a simple C program from www.cis.temple.edu/~ingargio/cis71/code/. After you choose a program, download it and save the source code on your local computer.

2. Determine the lines of source code. Count the lines of code manually and record your result here:

________________________________________________________

3. Next, use a tool that automatically counts the lines of code for you. For instance, you can cut and paste the code into Excel and look at the last row number and enter the value here:

________________________________________________________

Do the numbers entered here agree with those you calculated in step 2?

4. Spend some time looking at the source code you downloaded. Look for anything that might be a problem or that you do not understand. Document any findings here:

________________________________________________________

________________________________________________________

________________________________________________________

5. Although a manual audit of a small program is possible, the task becomes much more difficult on larger programs. To ease that task, some programs automatically look at the code. One such tool is Codebrag tool, which you can download from http://codebrag.com. After you download the program, install it on your computer.

6. Codebrag is an open source scanning tool that enables an auditor to search for program trouble spots and can suggest remedies. Although it might not find every problem, it can detect potential buffer overflows, race conditions, and other common problems. Run Codebrag (or another tool) against the open source software application in step 1.

7. Did Codebrag (or another tool that you used) find any security holes or potential vulnerabilities? If so, describe them here:

________________________________________________________

________________________________________________________

________________________________________________________

This exercise demonstrates how manual methods of auditing, such as counting lines of code or examining code for potential errors, can be supplemented by automated tools that aid in the process.

Review Questions

1. Of the following options, which process is not an application system testing methodology?

a. Snapshots

b. Entity integrity

c. Mapping

d. Base case system evaluation

2. Which of the following is a continuous auditing technique that detects items that meet specific criteria?

a. Audit hooks

b. Snapshots

c. Integrated test facilities

d. Continuous and intermittent simulation

3. A decision support system should be used appropriately. A DSS is designed to do which of the following?

a. Use structured models to solve complex problems

b. Support nontraditional support activities

c. Answer rigidly structured problems

d. Answer less structured problems

4. You have been asked to recommend a control that can detect exceptions to the following: “An order is normally for no more than 20 items, yet this order is for 2,000.” Which control works best to detect this type of exception?

a. Validity check

b. Range check

c. Reasonableness check

d. Limit check

5. What type of programming language are decision support systems most commonly developed with?

a. 2GL

b. 3GL

c. 4GL

d. 5GL

6. What is the best way to describe the difference between a data warehouse and a data lake?

a. Data warehouses always contain customer information

b. Data warehouses always contain raw data, while data lakes always contain structure and highly processed data.

c. Data lakes always contain raw data, while data warehouses always contain structure and highly processed data.

d. There is no difference between a data warehouse and a data lake.

7. When referring to electronic data interchange (EDI), which of the following statements would be most accurate?

a. EDI has no impact on internal or external controls.

b. EDI reduces internal controls.

c. EDI increases internal controls.

d. EDI has no impact on internal controls.

8. What control is specifically used after data has been entered into a system but before it has been processed?

a. Editing

b. Sequence check

c. Balancing

d. Input authorization

9. You have been asked to recommend a continuous audit technique. Which of the following techniques is considered the least complex?

a. Audit hooks

b. Systems control audit review file and embedded audit modules

c. Snapshots

d. Continuous and intermittent simulation

10. All the following are required activities during the project management process in the design and development phase except for which one?

a. Studying system flowcharts

b. Examining proposed test plans

c. Evaluating output controls

d. Examining proposed audit trails

Suggested Readings and Resources

Images Auditing the software development life cycle: www.isaca.org/chapters3/Atlanta/AboutOurChapter/Documents/GW2014/Auditing%20SDLC%20_%20Van%20Stone%20Kamara.pdf

Images ACID compliance: www.fredosaurus.com/notes-db/transactions/acid.html

Images Continuous auditing: www.iia.org.uk/media/1042659/gtag03-continuous-auditing-2nd-edition.pdf

Images Project management basics: www.managementhelp.org/plan_dec/project/project.htm

Images Data mining case studies: www.dataminingcasestudies.com

Images Supply Chain Management Review: www.scmr.com

Images SQL Server database security audit: http://isaca-denver.org/Chapter-Resources/ISACA201601SQLServerSecurityAudit.pdf

Images IT audits of cloud and SaaS: www.isaca.org/Journal/archives/2010/Volume-3/Pages/IT-Audits-of-Cloud-and-Saas.aspx

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset