6

Tips and Techniques for Advanced Auditing

The cloud provides organizations with flexibility, scalability, increased collaboration, and speed. However, IT auditors need to be aware of the potential pitfalls that can increase the security risk for organizations. In this chapter, we want to equip IT auditors with a few tips and techniques they can leverage to make audits in the cloud more efficient. In addition, we will build awareness of considerations IT auditors need to know as they prepare for more advanced auditing requirements.

In this chapter, we’ll cover the following main topics:

• Common pitfalls
• Tips, tricks, and techniques
• Preparing for more advanced auditing
• Other clouds – IBM, Oracle, Alibaba

By the end of this chapter, we will be able to identify the common pitfalls IT auditors need to be cognizant of as they approach their audits. We will also have knowledge of some tips and techniques that can be utilized for more effective audits and the considerations for more advanced audits, including other cloud environments.

Common pitfalls

With the scale, speed, and flexibility of the cloud comes complexity. This complexity leads to inherent pitfalls. We will review two broad areas that are common pitfalls for organizations that the IT auditor should be aware of. The first area involves administrative pitfalls that include not managing resource usage, an inability to control shadow IT, and a lack of automation. The second area concerns technical pitfalls that include misconfiguration, providing overly permissive access to users, and the inadvertent exposure of data, such as credentials.

Let’s look at the most common pitfalls IT auditors should focus on during an audit, and recommendations they may consider providing to a cloud customer.

Inability to forecast resource usage and costs

Cloud service providers such as Amazon Web Services (AWS), Azure, and GCP have hundreds of services to choose from. Many cloud customers usually choose services not suited to their business needs due to poor planning including not understanding the requirements of the business.

Cloud service providers have complex pricing models with rates that change according to service, region, and many other parameters. If an organization doesn’t fully understand a cloud service provider’s pricing model, or how it will progress with the cloud customer usage of service, the organization may incur unexpected costs.

Unused resources left running can easily cause cloud costs to spiral out of control. Organizations need to know the type of resources their applications consume, their quantity, and their corresponding price dimension. In addition, organizations should automate resource provisioning with the various cloud-respective auto-scaling features.

An IT auditor should look to examine if the organization is utilizing tools for tracking and licensing their cost. Such tools include third-party tools, such as CloudCheckr and CloudHealth.

In addition, an IT auditor should evaluate if the organization is leveraging tags as a means of identification for forecasting. Tags can be defined as applying metadata to help describe and identify the resources running across an organization’s cloud environments. Therefore, utilizing tags is an essential tool for gaining visibility into an organization’s cloud consumption and expenditure.

The impact of shadow IT

Cloud computing has made it easier for users to bypass organizational procurement processes in order to access the cloud solutions they want. For instance, it’s very easy to spin up accounts in AWS, Azure, or GCP without the knowledge of the organization. This is also referred to as shadow IT.

Shadow IT is the practice of bypassing organizational processes and installing IT solutions without the knowledge or approval of the organization. The risk with shadow IT is that it creates a situation in which corporate data is placed outside of the protection provided by the organization’s security controls. As a result, shadow IT increases the risk of a data breach.

An IT auditor should evaluate whether an organization leverages tools to track which cloud services are being accessed, to ensure there are no cloud services being used that aren’t authorized or supported.

Avoiding automation

Many organizations execute processes in the cloud manually, including installation processes, configuring virtual servers, setting up a network, storage volumes, or other cloud resources. Manual processes are time-consuming, error-prone, and hard to scale.

Automation encompasses solutions and tools that help eliminate repetitive aspects managed by one or more manual processes in the cloud. Cloud automation can increase security and the efficiency of workflows and tasks in the cloud. Automating routine security tasks will mitigate the majority of the manual risks presented by human error.

An IT auditor should examine if the organization utilizes automation within its cloud environments. One popular way to implement cloud automation is by using infrastructure as code (IaC). IaC is a process of creating cloud infrastructure through templates defined by code. Once developed, IaC becomes the building blocks for creating compute, storage, networking, and security policy in a cloud environment.

Misconfiguration

The cloud is still a novel concept and many organizations are unfamiliar with securing cloud infrastructure. As a result, it is easy for a security oversight to leave an organization’s cloud-based resources exposed to attackers. An example of a misconfiguration is mistakenly making a cloud-based repository public when you meant to make it private. This makes the repository accessible to anyone on the internet. Tools exist specifically for searching the internet for these unsecured cloud deployments.

To address cloud misconfiguration, the IT auditor should evaluate if the organization scans and reviews its cloud workloads for common vulnerabilities, such as exposed access points, resources labeled as public, and so on. This can be done by using cloud security posture management tools. Cloud security posture management tools are automated solutions that identify misconfiguration issues and compliance risks in cloud environments.

The inadvertent exposure of credentials

Credentials are the keys used to access cloud services. Credentials include user credentials, passwords, access keys, encryption, and decryption keys among others.

Software developers with poor security practices often embed credentials into their code to save time during the code development process. The code containing the credentials may then be uploaded into a public repository service. This can be considered the same as closing the entrance door of a house and forgetting the key in the lock: this is the most straightforward and obvious way to cause a data breach. Tools exist that enable adversaries to find credentials in public cloud accounts.

The IT auditor should examine whether the organization has enforced secure coding standards, along with a secrets management strategy. This is to ensure that software developers code their applications securely, minimizing any vulnerabilities that may be exploited.

Overly permissive access

Cloud environments usually include both human and non-human identities. Cloud environments are often created with overly broad permissions that allow unregulated access to cloud resources. Threat actors who have managed to get initial entry into a cloud environment might be able to leverage these broad permissions to escalate access and move laterally inside the cloud infrastructure.

Identity access management (IAM) is a framework of policies, processes, and technologies that enable organizations to manage digital identities and control user access to an organization’s data.

Some practices an IT auditor should look out for ways to minimize access risks are as follows:

• Implementing IAM wherever feasible
• Utilizing role-based access control and the least privilege principle
• Enabling multi-factor authentication
• Performing regular reviews of all identity roles and policies

Now that we’ve looked at the common pitfalls organization face in cloud adoption, let’s look at tips and tricks to perform an effective audit.

Tips, tricks, and techniques

Cloud environments are complex and have low visibility. In a traditional data center, there was a finite number of assets that IT auditors could examine and report on. However, in a cloud environment, there can be exponential growth in the number of assets, which may include virtual machines (VMs), virtual networks, containers, serverless functions, and so on. It can be very challenging to inventory what is running in an organization’s cloud.

Asset inventory plays such a foundational role in a cyber security program, that CIS Critical Security Controls and the NIST Cybersecurity Framework list the need to inventory and control infrastructure assets as their first security controls.

The first thing the IT auditor needs to do is to understand the cloud asset inventory to be able to perform an effective audit. The IT auditor needs to understand what applications are running within the cloud and whether they are approved by the organization, or if they represent shadow IT. In this section, we will demonstrate how to leverage native tools from each cloud service provider to get inventory and a quick snapshot security posture of the cloud environment.

It is important to note that the tips and tricks provided in this chapter should be incorporated and aligned based on the cloud customer’s cloud security policies, standards, and procedures.

AWS

In AWS, you can use the AWS Tag Manager to inventory cloud resources. AWS allows you to assign metadata to a cloud customer’s AWS resources using tags. All tags serve as simple labels that consist of a user-defined key and an optional value. The Tag Editor can be found under AWS Resource Groups. To list all resources using the Tag Editor, you can either manually select specific regions, or all of the regions listed under Region by selecting All supported resource types, and not specifying a tag. Then, click Find resources and this will show all resources that were created, as shown in Figure 6.1:

Figure 6.1 – Finding resources to tag

All the resources are identified, as shown in Figure 6.2:

Figure 6.2 – Resource search results

To get a snapshot of the AWS security posture, we can use AWS Security Hub. According to the AWS documentation, “AWS Security Hub is a cloud security posture management service that performs security best practice checks, aggregates alerts, and enables automated remediation.”

To use AWS Security Hub, we first have to enable the service. We can simply search for the service using the AWS search function, as seen in Figure 6.3:

Figure 6.3 – AWS Security Hub

Before you can enable Security Hub standards and controls, you must first enable AWS Config, as seen in Figure 6.4:

Figure 6.4 – Enabling AWS Config

To enable AWS Security Hub, select the security standards you would like it to perform security checks against as seen in Figure 6.5. AWS Security Hub security checks are currently mapped to AWS Foundational Security Best Practices, the CIS AWS Foundation Benchmark, and Payment Card Industry Data Security Standard (PCI DSS).

Figure 6.5 – Security standards

Once you have enabled Security Hub, you may have to wait up to two hours for it to populate. There are three important tabs for the IT auditor to review:

• Summary
• Findings
• Insights

Let’s start by reviewing the Summary tab. It has different information such as security standards passed and failed, Resources with the most failed security checks, findings by region, and so on:

Figure 6.6 – The Summary tab

You can view findings across multiple regions, as seen in Figure 6.7. As you can see, we have 1 moderate finding and 15 low findings:

Figure 6.7 – Findings by region

Another tab the IT auditor should review in AWS Security Hub is the Findings tab. Here, you can get the details of a finding and what security checks it failed, as seen in Figure 6.8:

Figure 6.8 – Findings

Another useful tab the IT auditor should review in AWS Security Hub is the Insights tab. Here, you can get insight into the type of finding. The information presented on this tab include: AWS resources with the most findings, S3 buckets with public write or read permissions, AMIs that are generating the most findings, and so on, as seen in Figure 6.9:

Figure 6.9 – The Insights tab

The second tool an IT auditor should leverage is AWS Config. As per the AWS documentation, “AWS Config is a service that enables you to assess, audit, and evaluate the configurations of your AWS resource.” AWS Config provides AWS-managed rules, which are predefined, customizable rules that AWS Config uses to evaluate whether your AWS resources align with prescribed best practices. AWS Config can be quite valuable to an IT auditor as the service lets you define your own rules. To get to AWS Config, search for it on the AWS console, as seen in Figure 6.10:

Figure 6.10 – AWS Config search

You can enable AWS Config with 1-click setup, as shown in Figure 6.11:

Figure 6.11 - Enabling AWS Config

Select General settings | Rules, then click Confirm, as seen in Figure 6.12:

Figure 6.12 – Setting up AWS Config

Once AWS Config has launched, the three most valuable tabs for an IT auditor are the Dashboard, Rules, and Resources tabs. In the Dashboard tab, you can find information such as Configuration Items, Compliance status, and AWS Config success metrics, as seen in Figure 6.13:

Figure 6.13 – The Dashboard tab

The Rules tab displays whether your resource configurations align with the relevant rules identified, as seen in Figure 6.14:

Figure 6.14 – The Rules tab

The Resources tab displays an inventory of supported resources and their compliance status, as shown in Figure 6.15:

Figure 6.15 – The Resources tab

In our example, you can see we have the IAM User, EC2 Volume, and S3 Bucket types that are Noncompliant. You can click on each resource identifier for more information. The IT auditor would have to investigate relevant findings and collaborate with the cloud customer to come up with appropriate remediation.

A third useful tool for an IT auditor is AWS Trusted Advisor.

AWS Trusted Advisor

AWS Trusted Advisor provides real-time best practice guidance to help provision, monitor, and maintain AWS resources. You can then follow AWS Trusted Advisor recommendations to optimize your services and resources. These best practice recommendations span five categories:

• Cost optimization
• Performance
• Security
• Fault tolerance
• Service limits

To launch AWS Trusted Advisor, search for the service in the AWS console, as seen in Figure 6.16:

Figure 6.16 – Searching for AWS Trusted Advisor

The following screenshot is from AWS documentation and shows an example of the AWS Trusted Advisor interface, as seen in Figure 6.17:

Figure 6.17 – The AWS Trusted Advisor interface

Azure

A core tool an IT auditor should leverage in the Azure environment is Microsoft Defender for Cloud.

Microsoft Defender for Cloud

Microsoft Defender for Cloud (formerly known as Azure Security Center) is Azure’s native solution. The service helps measure, maintain, and improve the level of security by continuously assessing resources and providing recommendations. You can use Microsoft Defender to determine the population of cloud resources. To launch Microsoft Defender, search for it in the search bar. The Overview tab has information such as Security posture, Azure Subscriptions, and Active Recommendations, as seen in Figure 6.18:

Figure 6.18 – The Overview tab

The Inventory tab displays the total resources running in Azure. It gives information such as resource type and health status, as seen in Figure 6.19:

Figure 6.19 – The Inventory tab

In our example, we have 1 unhealthy resource. The auditor would need to investigate this resource further.

The Recommendations tab provides Azure recommendations with related severity of issues. As you can see in our example, we have wide open network ports and Microsoft Defender recommends we should restrict them:

Figure 6.20 – The Recommendations tab

Under the Cloud Security section is Security posture, which provides a holistic view and a secure score as seen in Figure 6.21:

Figure 6.21 – The Security posture tab

Another item under the Cloud Security section is Regulatory compliance. In this section, you can add and track customized regulations that you want your organizations to align with, as seen in Figure 6.22:

Figure 6.22 – Regulatory compliance

Another tool an IT auditor can leverage is Microsoft Purview.

Microsoft Purview

Microsoft Purview (formerly Azure Purview) is a centralized data governance and risk management service that helps manage data. To set up Microsoft Purview, you can search for it on any browser, or go to https://azure.microsoft.com/en-us/products/purview/#overview, as seen in Figure 6.23:

Figure 6.23 – Azure Purview

You will need to set up a Microsoft Purview account, as shown in Figure 6.24:

Figure 6.24 – Creating the Microsoft Purview account

With the account created, launch the Microsoft Purview workspace from the Azure portal, as shown in Figure 6.25.

Figure 6.25 – Selecting an account

With Microsoft Purview launched, you are able to navigate to the Browse assets and Manage glossary tabs, as seen in Figure 6.26:

Figure 6.26 – Microsoft Purview Home page

If you click on Browse assets, it displays the population discovered, as seen in Figure 6.27.

Figure 6.27 – Browse assets

Before Microsoft Purview scans your data, Microsoft Purview will need to be given access to data sources. You can do this by assigning Microsoft Purview managed identity access rights.

Once configured, Purview can create a holistic, up-to-date map of your data landscape with automated data discovery, sensitive data classification, and many other data insights, as shown in Figure 6.28:

Figure 6.28 – Management | Overview

GCP

IT auditors can leverage a tool called Security Command Center for GCP.

Security Command Center

Security Command Center provides centralized visibility into the security posture and status of the GCP environment. According to GCP’s documentation, “Security Command Center ingests data about new, modified, and deleted assets from Cloud Asset Inventory, which continuously monitors assets in your cloud environment. Security Command Center supports a large subset of Google Cloud assets. For most assets, configuration changes, including IAM and organization policies, are detected in near-real time.” You can get to Security Command Center from the Google Cloud Console, as shown in Figure 6.29:

Figure 6.29 – Security Command Center

Security Command Center requires an organization resource that is associated with a domain. As a result, you will need to create an organization, as shown in Figure 6.30.

Figure 6.30 – The Admin console

Note

For detailed information on setting up Security Command Center, please view the Google Cloud Documentation at https://cloud.google.com/security-command-center/docs/set-up.

Once the organization is set up, you need to enable the Security Command Center dashboard, as shown in Figure 6.31:

Figure 6.31 – The Security tab

Next, select the services you would like to set up, as shown in Figure 6.32:

Figure 6.32 – Services setup

Once Security Command Center is fully configured, we can go and review the security posture of the GCP environment. There are five important tabs to review; they include OVERVIEW, VULNERABILITIES, ASSETS, FINDINGS, and SOURCES:

• OVERVIEW: Provides a snapshot of the security state, as seen in Figure 6.33:

Figure 6.33 – Security Command Center | OVERVIEW

• VULNERABILITIES: Provides a list of the vulnerabilities found when last scanned. In our example, we have an MFA not enforced vulnerability with a High severity, as seen in Figure 6.34:

Figure 6.34 – Security Command Center | VULNERABILITIES

• ASSETS: Displays the Google Cloud resources, as shown in Figure 6.35:

Figure 6.35 – Security Command Center | ASSETS

• FINDINGS: Provides detail around the security risks identified and what organizations they have been found under, as seen in Figure 6.36:

Figure 6.36 - Security Command Center | FINDINGS

• SOURCES: Displays a summary of assets and findings from the relevant security sources that are enabled, as seen in Figure 6.37:

Figure 6.37 – Security Command Center | SOURCES

A second tool an IT auditor can leverage in GCP is Cloud Asset Inventory.

Cloud Asset Inventory

Cloud Asset Inventory is a service that allows one to view, monitor, and analyze GCP assets. To access Cloud Asset Inventory, you can either go to the Google Console under IAM & Admin or go through a browser, as shown in Figure 6.38:

Figure 6.38 – Cloud Asset Inventory

Three tabs are important for the IT auditor to review; OVERVIEW, RESOURCE, and IAM POLICY. The OVERVIEW tab provides a snapshot of resources in use, as shown in Figure 6.39:

Figure 6.39 – The Asset Inventory OVERVIEW tab

The RESOURCES tab displays the resource name, type, and location, as shown in Figure 6.40:

Figure 6.40 – The Asset Inventory RESOURCES tab

IAM policies display all the IAM policies across services, and resource types with information such as roles, permissions, and so on:

Figure 6.41 – The Asset Inventory IAM policy

A third tool an IT auditor can leverage to assess GCP is Cloud Overview.

Cloud Overview

Cloud Overview has three tabs; DASHBOARD, ACTIVITY, and RECOMMENDATION. The DASHBOARD tab has a summary of Resources, CPU usage, APIs, Billing, and Monitoring, as seen in Figure 6.42:

Figure 6.42 – The Cloud Overview DASHBOARD

The ACTIVITY tab provides a log of recent activities. In our example, we can see a VM and a network were deployed recently, as seen in Figure 6.43:

Figure 6.43 – The Cloud Overview ACTIVITY tab

The RECOMMENDATIONS tab provides GCP recommendations for the project. Currently, we either have no recommendations or the tab hasn’t populated, as seen in Figure 6.44:

Figure 6.44 – The Cloud Overview RECOMMENDATIONS tab

In this section, we’ve reviewed some tips and techniques to utilize for a more effective audit. This included leveraging cloud-native solutions in AWS, Azure, and GCP to review the security posture of an organization. Next, we will talk about how IT auditors can become prepared for more advanced auditing.

Preparing for more advanced auditing

So far we have been leveraging native graphical user interface (GUI) tools to perform audits. To prepare for more advanced auditing, the IT auditor should familiarize themselves with the command-line interface (CLI). A GUI permits users to interact with the cloud platform using graphical features such as icons, windows, and menus. In contrast, a CLI allows users to interact with the system via commands. A CLI gives the IT auditor more autonomy to write custom scripts that could be used to automate processes.

Let’s start by looking at the AWS CLI.

Note

Instructions on getting started and installing the AWS CLI can be found at https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html.

Once you have the AWS CLI installed, you will need to configure the basic settings that the AWS CLI uses to interact with AWS. These include security credentials, Default region name, and Default output format, as shown in Figure 6.45:

Figure 6.45 – The AWS CLI configuration settings to interact with AWS

Now that we have the AWS CLI configured, we can run simple commands to perform audit actions. I will demonstrate two commands that an IT auditor can leverage. To list the number of users within AWS, we type in the following command:

aws iam-list users

Note that UserId, CreateDate, and PasswordLastUsed are displayed, as shown in Figure 6.46:

Figure 6.46 – AWS list users in IAM

As another example, if you need to know which groups are connected to AWS VPCs, you can type the following command:

aws ec2 describe-security-groups

This is shown in Figure 6.47:

Figure 6.47 – AWS describe-security-groups

VPC security groups and their respective attributes are shown in Figure 6.48:

Figure 6.48 – AWS security groups and their attributes

In Chapter 4, Network, Infrastructure, and Security Controls, we stated that all AWS accounts come with a default VPC. A default VPC comes with a public subnet. As an IT auditor, it is important to ensure that sensitive company data is not placed in a public subnet.

AWS has a command library that can be found at https://awscli.amazonaws.com/v2/documentation/api/latest/index.html. In addition, you can type in aws help on the AWS CLI for information about a specific command.

Next, we will look at the Azure CLI.

Note

Instructions on getting started and installing the Azure CLI can be found at https://docs.microsoft.com/en-us/cli/azure/get-started-with-azure-cli.

Once you have the Azure CLI set up, you need to log in to Azure:

Figure 6.49 – Logging in to Azure

I will demonstrate two commands an IT auditor can leverage to perform an audit. To list all the resource groups we have, use the following command:

az resource list

Figure 6.50 – The resource list

To get a list of IP addresses associated with a VM, we can use the following command:

az vm list-ip-addresses

Figure 6.51 – Listing IP addresses associated with a VM

The full Azure CLI command reference can be found at https://docs.microsoft.com/en-us/cli/azure/reference-index?view=azure-cli-latest. In addition, you can type in az help on the Azure CLI for information on a specific command.

Next, we will look at the Google Cloud CLI.

Note

Instructions on getting started and installing the Google Cloud CLI can be found at https://cloud.google.com/sdk/docs/install.

Once you have installed the CLI, you need to authenticate using Google credentials:

Figure 6.52 – Authentication

I will demonstrate two commands an IT auditor can leverage to perform an audit. To list all credential accounts, type the following command:

gcloud auth list

Figure 6.53 – Listing credential accounts

To list your project’s logs, type the following command:

gcloud logging logs list

Figure 6.54 – Listing logs

The Google Cloud CLI cheat sheet can be found at https://cloud.google.com/sdk/docs/cheatsheet. In addition, you can type in the following command on the Google Cloud CLI for information on a specific command:

gcloud help

In this section, we’ve reviewed how to leverage CLI for more advanced auditing. All the tools we have talked about in this chapter are native cloud tools.

It is important to acknowledge there are open source tools that can assist IT auditors in advanced auditing. One such tool is Cloud Custodian. Cloud Custodian is an open source tool developed by Capital One for implementing automated security, compliance, and governance. IT auditors can use Cloud Custodian to monitor cloud environments as it generates a set of outputs for any given security policies. In the next section, we will briefly touch on other cloud platforms on the market.

Other clouds

Although the book focuses on the “big three” cloud providers, we recognize that there are other cloud infrastructure solutions and want to briefly highlight these and options for learning more outside of this book.

Oracle Cloud Infrastructure

Oracle Cloud Infrastructure or (OCI) has a unique attribute in the concept of compartments, which are used to segment and manage access as well as develop logical groupings of resources. OCI also offers features such as Security Zones, which are compartments where security can’t be disabled. Similar to other clouds, OCI has available features for finance and change management, tagging of resources for management, as well as options for logging and notification of log events through a feature known as Service Connector Hub.

Learn more at https://education.oracle.com/.

IBM Cloud

One of the more unique things about IBM Cloud is the amount of open source technology that is used throughout the platform, and the availability of product solutions known as Cloud Paks, which range from security and compliance tooling to AI and chatbot. Like other cloud environments, IBM Cloud offers an interface for viewing security and compliance-related information. However, the creation and configuration of a “collector” may be required in order for the IBM Cloud Security and Compliance center to display this information. As part of the discovery and assessment for audit, this should be reviewed.

Learn more at https://www.ibm.com/training/cloud.

Alibaba Cloud

Alibaba Cloud is another global cloud option and has a strong presence primarily in Asia. Alibaba Cloud is also known as Aliyun. In terms of structure, Alibaba Cloud is comparable in many ways to AWS. Some of the more interesting and unique features include the ability to configure both user and role-based SSO and the availability of a service known as Cloud Config, which tracks and records configuration changes. Similar to other cloud providers, logging is available (through a feature known as ActionTrail). However, it will need to be assessed to ensure it’s been activated and is logging all in-scope components.

Learn more at https://edu.alibabacloud.com/.

For other cloud providers, it is critical to understand if there are any sub-service organization dependencies that need to be taken into consideration. A sub-service organization is a supporting vendor that is engaged by the cloud service provider to perform some services for the cloud service provider. For example, a cloud service provider may utilize a sub-service organization for infrastructure hosting. In these instances, the IT auditor will also have to evaluate controls at the sub-service organization. The IT auditor will also want to verify that the scope of any assessments performed on the sub-service organization is sufficient for the services used by the cloud customer.

In this section, we’ve reviewed the options for other clouds outside the “big three” cloud service providers (AWS, Azure, and GCP).

Summary

In this chapter, we looked at tips, tricks, and techniques that you can utilize for the three major cloud providers AWS, Azure, and GCP. We covered how to identify the common pitfalls IT auditors need to be cognizant of as they approach their audits; tips and techniques to utilize for more effective audits, and considerations for more advanced audits, including other cloud environments.

In our next chapter, we’ll review tools for monitoring and assessing the cloud.