Chapter 4
Compute

THE AWS CERTIFIED SYSOPS ADMINISTRATOR - ASSOCIATE EXAM TOPICS COVERED IN THIS CHAPTER MAY INCLUDE, BUT ARE NOT LIMITED TO, THE FOLLOWING:

  • Domain 2.0 High Availability
  • images2.1 Implement scalability and elasticity based on scenario
  • Content may include the following:
    • Which AWS compute service to use
    • Scalability of specific AWS compute service
    • Elasticity of specific AWS compute service
  • images2.2 Ensure level of fault tolerance based on business needs
  • Content may include the following:
    • What AWS Cloud services are Availability Zone based and the implication for high availability
    • What AWS Cloud services are region-based services and the implication for high availability
  • Domain 4.0 Deployment and Provisioning
  • images4.1 Demonstrate the ability to build the environment to conform to architectural design
  • Content may include the following:
    • Choosing the appropriate AWS Cloud service to meet the business need
    • How to achieve high availability in compute services
  • images4.2 Demonstrate the ability to provision cloud resources and manage implementation automation
  • Content may include the following:
    • Steps in deployment of various AWS Cloud services and key decisions regarding scaling and availability
  • Domain 6.0 Security
  • images6.3 Demonstrate understanding of the shared responsibility model
  • Content may include the following:
    • Use of security groups to control access to compute instances
    • Use of public/private keys to control access to guest operating system

images

Introduction to AWS Compute Services

Amazon Elastic Compute Cloud (Amazon EC2) was the third service introduced by AWS. It followed Amazon Simple Queue Service (Amazon SQS) and Amazon Simple Storage Service (Amazon S3). Amazon EC2 was announced in 2006, as a way to provide on-demand computing. Since that initial introduction, AWS has worked to expand Amazon EC2 and has incorporated a Graphical User Interface (GUI)-based management console, persistent storage in the form of both magnetic drives and Solid State Drives (SSDs), CPUs optimized for different types of compute loads, and enhanced networking capabilities.

AWS has also expanded the meaning of compute. Today, compute includes the ability to spin up any number of instances in the cloud on a pay-as-you-go basis, and it also incorporates some of the following services:

AWS Lambda This is a compute service that runs code without the need to provision or manage servers. Essentially, AWS Lambda acquires CPU cycles without acquiring the actual CPU. AWS Lambda executes code on demand and scales automatically. This scaling can range from a few requests per day to thousands per second.

AWS Elastic Beanstalk This is a service that enables you to quickly deploy and manage applications on the AWS Cloud without worrying about the infrastructure that runs those applications. AWS Elastic Beanstalk reduces management complexity without restricting choice or control. You upload your application, and AWS Elastic Beanstalk handles the details of capacity provisioning, load balancing, scaling, and application health monitoring.

Amazon EC2 Container Service (Amazon ECS) This is a highly scalable and fast container management service that makes it easy to run, stop, and manage Docker containers on a cluster of Amazon EC2 instances.

Amazon Lightsail This service is for developers who need virtual private servers. Amazon Lightsail includes everything you need to launch your project quickly—a virtual machine, SSD-based storage, data transfer, Domain Name System (DNS) management, and a static IP address—for a low, predictable price.

AWS Batch This service enables you to run batch computing workloads on the AWS Cloud. Batch computing is a common way for developers, scientists, and engineers to access large amounts of compute resources. AWS Batch removes the undifferentiated heavy lifting of configuring and managing the required infrastructure.

In this chapter, you learn about the various compute products and services described here, how to implement them, how to manage them, and how to secure them. It focuses on the following:

  • Launching an Amazon EC2 instance
  • The pricing model for Amazon EC2
  • What is Amazon EC2 instance metadata and how to find it
  • What is user data, how is it loaded into the Amazon EC2 instance, and how to find it
  • How to create an Amazon Machine Image (AMI)
  • Knowing when you would use a dedicated host and when you would use a dedicated instance
  • The different CPU types available in Amazon EC2 and how to change instance types on running instances
  • The different storage options available for Amazon EC2
  • Understanding how instance size affects network speed
  • Understanding how to log in remotely to an Amazon EC2 instance
  • Knowing what Amazon EC2 metrics are monitored in Amazon CloudWatch
  • How to configure an AWS Lambda function
  • The pricing model for Amazon Lambda
  • What AWS Lambda metrics Amazon CloudWatch monitors
  • Understanding how to control CPU choice in AWS Lambda
  • Understanding the use cases for AWS Elastic Beanstalk, Amazon Lightsail, and AWS Batch
  • What is an AWS Lambda event and what are supported AWS Lambda event sources
  • What services discussed in this chapter are considered highly available and what services are not
  • What services AWS Elastic Beanstalk spins up by default
  • What tools are available to configure the services mentioned in this chapter
  • Knowing the different tools available to monitor Amazon EC2, Amazon ECS, and AWS Lambda
  • Knowing the different tools available to manage security for Amazon EC2
  • Understanding the shared responsibility model and how it applies to these services: Amazon EC2, Amazon ECS, AWS Lambda, and AWS Elastic Beanstalk
  • Understanding what security groups do and how they protect instances
  • Understanding the basic process for setting up Amazon ECS
  • Understanding when you would use Amazon EC2 and when you would use Amazon Lightsail
  • Knowing what services Amazon Lightsail spins up by default

Amazon Elastic Compute Cloud (Amazon EC2)

Amazon EC2 provides scalable computing capacity in the AWS Cloud. Amazon EC2 eliminates your need to invest in hardware, so that you can develop and deploy applications faster. You can use Amazon EC2 to launch as many or as few virtual servers as you need, configure security and networking, and manage storage. You launch your instances in the AWS Region and Availability Zone that you specify.

Amazon EC2 provides the following features:

  • Virtual computing environments, known as instances
  • Preconfigured templates for your instances, known as AMIs, that package the components you need for your server (including the operating system and additional software)
  • Various configurations of CPU, memory, storage, and networking capacity for your instances, known as instance types
  • Secure login information for your instances using key pairs (AWS stores the public key, and you store the private key in a secure place.)
  • Storage volumes for temporary data that’s deleted when you stop or terminate your instance, known as instance store volumes
  • Persistent storage volumes for your data using Amazon Elastic Block Store (Amazon EBS), known as Amazon EBS volumes
  • Multiple physical locations for your resources, such as instances and Amazon EBS volumes, known as regions and Availability Zones
  • A firewall that enables you to specify the protocols, ports, and source IP ranges that can reach your instances using security groups
  • Static IPv4 addresses for dynamic cloud computing that can be assigned to an individual instance, known as Elastic IP addresses
  • IPv6 addresses that can also be assigned to instances
  • Virtual networks that you can create that are logically isolated from the rest of the AWS Cloud and that you can optionally connect to your own network, known as Amazon Virtual Private Clouds (Amazon VPCs)

Implementation

In implementing an Amazon EC2 instance, you go through the following steps:

  1. Decide on the AMI that you want to use.
  2. Choose the instance type and size that you need.
  3. Provide other needed configuration details.

This section goes through each of these steps in greater detail.

Choosing an AMI

Decide on the AMI that you want to use. An AMI provides the information required to launch an instance. An AMI includes the following:

  • A template for the root volume for the instance (perhaps an operating system, an application server, and applications)
  • Launch permissions that control which AWS accounts can use the AMI to launch instances
  • A block device mapping that specifies the volumes to attach to the instance when it is launched

You can obtain AMIs from one of four places:

  • Provided by AWS
  • AWS Marketplace
  • You can create and manage your own AMIs.
  • A shared AMI

With AMIs provided by AWS, you have a variety of choices of either Linux or Windows operating systems. With the Windows AMI, the cost of the operating system is included in the hourly charge for the instance.

AWS Marketplace offers a wide variety of AMIs from a number of third parties. You can choose AMIs in categories such as software infrastructure, developer tools, or business software from a number of vendors, including Juniper, Bitnami, and SAP.

AWS also gives you the ability to create and manage your own AMIs. For example, you can take an existing AWS-provided Linux AMI, modify it to meet your needs, and save that AMI. You can then use that AMI as the basis for compute infrastructure. This is the route most companies take when choosing an AMI.

A shared AMI is an AMI that a developer created and made available for other developers to use. You can create your own AMIs and share them with others. Note that you use a shared AMI at your own risk—Amazon can’t vouch for the integrity or security of AMIs shared by other Amazon EC2 users. Therefore, you should treat shared AMIs as you would any foreign code that you might consider deploying in your own datacenter and perform the appropriate due diligence. Only use an AMI from a trusted source.

Choosing an Instance Type

After choosing an AMI, you then need to choose an instance type. The instance type that you specify determines the hardware of the host computer used for your instance. Each instance type offers different compute, memory, and storage capabilities. Instance types are grouped in instance families based on these capabilities.

Amazon EC2 provides each instance with a consistent and predictable amount of CPU capacity, regardless of its underlying hardware. Amazon EC2 dedicates some resources of the host computer (for example, CPU, memory, and instance storage) to a particular instance. Amazon EC2 shares other resources of the host computer (such as the network and the disk subsystem) among instances.

The choice of the instance type affects the following attributes:

  • CPU specialization
  • Virtualization types
  • Network features
  • Storage features
  • Instance limits

With CPU specialization, CPUs can be classified as being optimized in one of the following ways:

  • General Purpose (T2, M4)
  • Compute-Optimized (C3, C4)
  • Memory-Optimized (X1, R3, R4)
  • Storage-Optimized (I3, D2)
  • Accelerated Computing (P2, G2, F1)

As AWS continues to introduce new instance families and new generations of instance families, the most up-to-date information can be found in the Amazon EC2 User Guide.

Each instance type supports one or both of the following types of virtualization: paravirtual or Hardware Virtual Machine (HVM). The virtualization type of your instance is determined by the AMI that you use to launch it.

For best performance, you may want to use an HVM AMI. In addition, HVM AMIs are required for enhanced networking. HVM virtualization uses hardware-assisted technology provided by the AWS platform. With HVM virtualization, the guest Virtual Machine (VM) runs as if it were on a native hardware platform, except that it still uses paravirtual network and storage drivers for improved performance.

Network features are also influenced by the instance type you choose. Network speed is primarily a function of the size of the instance. Larger instances provide greater network speed than smaller instances in the same family. Certain instance types offer support for enhanced networking. Enhanced networking uses Single Root I/O Virtualization (SR-IOV) to provide high-performance networking capabilities. There are also instance families that offer 10G connectivity. Support of jumbo frames (frames greater than 1,500 Maximum Transmission Unit [MTU] in size) is also dependent on the instance type chosen. More information about networking on AWS can be found in Chapter 5, “Networking.”

Storage features are also influenced by the instance type you choose. Some instance types support Amazon EBS volumes and instance store volumes, while other instance types support only Amazon EBS volumes. Some instances that support instance store volumes use SSDs to deliver very high random I/O performance. To obtain additional, dedicated capacity for Amazon EBS I/O, you can launch some instance types as Amazon EBS-optimized instances. Some instance types are Amazon EBS-optimized by default. Refer to Chapter 6, “Storage,” for a more comprehensive look at storage on AWS.

There is a limit on the total number of instances that you can launch in an AWS Region, and there are additional limits on some instance types. You can use the Amazon EC2 Service Limits page in the Amazon EC2 console to view the current limits for resources provided by Amazon EC2 and Amazon VPC on a per-region basis.

Other Configuration Details

Once you choose the AMI and the instance type, there are a number of other configuration details that you need to provide. These details include the following:

  • Number of instances
  • Purchasing options
  • Amazon VPC to be used
  • Subnet to be located in
  • Public IP address
  • AWS Identity and Access Management (IAM) role
  • Shutdown behavior
  • Monitoring
  • Tenancy
  • User data
  • Storage type and volume
  • Tagging
  • Security groups
  • Public/private key

You are able spin up as many instances as you want simultaneously, subject to the service limits mentioned.

AWS purchasing options allow you to request Spot Pricing. If you request Spot Pricing, you are sent to an additional screen where you can configure Availability Zone, maximum price, launch group (if applicable), and whether this is a persistent request. If it’s not a persistent request, you specify what time and date the request is valid to and from.

All Amazon EC2 instances need to be spun up in an Amazon VPC. You can choose the Amazon VPC in which the instance is spun up. If you have not already created an Amazon VPC, then the instance will be spun up in the default Amazon VPC. Similarly, because Amazon EC2 is an Availability Zone-based service, you need to choose the Availability Zone in which to locate the Amazon EC2 instance. The private IP addresses assigned to the Amazon EC2 instance is determined by what Amazon VPC and what subnet you locate the instance in.

You can choose to assign a public IP address to your instance. Doing so will potentially make this Amazon EC2 instance accessible from the Internet (the other requirement is that the Amazon VPC containing the Amazon EC2 instance needs to have an Internet gateway). Public IP addresses are associated with specific interfaces, so if you have multiple interfaces on your Amazon EC2 instance, you can have multiple public IP addresses. The public IP address remains attached to the interface as long as the Amazon EC2 instance is running. If the Amazon EC2 instance is rebooted, it keeps the public IP address. If the Amazon EC2 instance is turned off, the public IP address is removed. If the Amazon EC2 instance is turned back on, a new public IP address is assigned.

Both IPv4 and IPv6 are supported in Amazon EC2.

IAM roles are discussed in more details in Chapter 3, “Security and AWS Identity and Access Management (IAM).” You can assign an IAM role to an Amazon EC2 instance as part of a configuration. This IAM role would give the Amazon EC2 instance permission to access other AWS Cloud services (such as when the Amazon EC2 instance needs to access an Amazon S3 bucket).

Shutdown behavior controls the conduct of the Amazon EC2 instance after it receives a shutdown command. Amazon EC2 instances that have instance storage can only be in a running state or be terminated—they cannot be stopped. Amazon EC2 instances that have Amazon EBS storage can be running, stopped, or terminated. The default behavior of these instances is to delete the attached storage after the instance is terminated.

Termination protection, when enabled, prevents the accidental termination of instances. After termination protection is enabled, you must disable it if you wish to terminate an instance. This gives you an extra level of protection against accidental termination.

Amazon EC2’s default mode is shared tenancy. The other choices are to run as a dedicated host or a dedicated instance. Dedicated hosts allow you to use your existing per-socket, per-core, or per-VM software licenses. Rebooting a dedicated host will reboot the instance on the same physical AWS infrastructure. Dedicated instances are Amazon EC2 instances that run in an Amazon VPC on hardware that is dedicated to a single customer. Your dedicated instances are physically isolated at the host hardware level from instances that belong to other AWS accounts.

User data is that which you can pass to the instance both upon initial boot up and, if desired, in subsequent boots. User data allows you to specify parameters for configuring your instance or attach a simple script. You can also use this data to build more generic AMIs that can be modified by configuration files supplied at launch time. For example, you can change a basic AWS Linux AMI to become a web server using user data. User data is limited to 16 KB, but it can have links to locations (like an Amazon S3 bucket) to load larger files.

Different instance types will support different storage options. The three major types of storage for Amazon EC2 are instance storage, Amazon EBS, and Amazon Elastic File System (Amazon EFS).

Instance storage is where the disks are physically attached to the host computer. Instance storage provides temporary block-level storage for instances. The data on an instance store volume persists only during the life of the associated instance; if you stop or terminate an instance, any data on instance store volumes is lost. Note that not all Amazon EC2 instance types support instance storage. Instance storage cannot be modified in terms of the amount of storage. Pricing for instance storage is included in the pricing for the Amazon EC2 instance.

Amazon EBS provides block-level storage volumes for use with Amazon EC2 instances. Amazon EBS volumes are highly available and reliable storage volumes that can be attached to any running instance that is in the same Availability Zone. Amazon EBS volumes that are attached to an Amazon EC2 instance are exposed as storage volumes that persist independently from the life of the instance. Amazon EBS volumes can be in a variety of media (magnetic or SSD), can be in a variety of sizes, can be encrypted, and have specified I/O speeds (with Provisioned IOPS). Because Amazon EBS exists as a separate service, it is priced independently of the Amazon EC2 instance to which it is attached.

Amazon EBS is recommended when data must be quickly accessible and requires long-term persistence. Amazon EBS volumes are particularly well-suited for use as the primary storage for file systems, databases, or for any applications that require fine granular updates and access to raw, unformatted, block-level storage. Amazon EBS is well suited to both database-style applications that rely on random reads and writes and to throughput-intensive applications that perform long, continuous reads and writes.

Another key difference between instance store-backed Amazon EC2 instances and Amazon EBS-backed Amazon EC2 instances is the ability to stop and start the Amazon EC2 instance. With Amazon EBS-backed Amazon EC2 instances, you can stop and start the Amazon EC2 instance. With instance store-backed Amazon EC2 instances, you can only delete the instance. Figure 4.1 illustrates the differences between an instance store-backed Amazon EC2 instance and an Amazon EBS-backed Amazon EC2 instance.

Chart shows instance store backed and EBS backed AmazonEC2 instance starts with AMI launch, pending, running, shutting-down. Running has options: stop at right, start or terminate, and reboot on its left.

FIGURE 4.1 Differences between an instance store-backed Amazon EC2 instance and an Amazon EBS-backed Amazon EC2 instance

Amazon EFS provides scalable file storage for use with Amazon EC2. You can create an Amazon EFS file system and configure your instances to mount the file system. You can use an Amazon EFS file system as a common data source for workloads and applications running on multiple instances. Because Amazon EFS exists as a separate service, it is priced independently of the Amazon EC2 instance to which it is attached. It is important to note that Amazon EFS can only be used with Linux-based Amazon EC2 instances.

Tags enable you to categorize your Amazon EC2 instances in different ways (for example, by purpose, owner, or environment). This is useful when you have many resources of the same type—you can quickly identify a specific resource based on the tags you have assigned to it. Each tag consists of a key and a value, both of which you define. You can work with tags using the AWS Management Console, the AWS Command Line Interface (AWS CLI), and the Amazon EC2 Application Programming Interface (API).

Security groups are applied to the interface of an Amazon EC2 instance. Security groups allow access via port, protocol, and source. A source can be an IP address, a range of IP addresses, or another security group. You cannot set up a security group with a deny rule. Security groups control both inbound and outbound access, but the recommended practice is to use them mostly to control inbound access. By default, security groups allow all outbound access. In addition, security groups are stateful; if you send a request from your instance, the response traffic for that request is allowed to flow in regardless of inbound security group rules. You must attach a security group with either Secure Shell (SSH) or Remote Desktop Protocol (RDP) access if you intend to be able to log in to the guest operating system.

In order to protect your Amazon EC2 instance, Amazon EC2 uses public-key cryptography to encrypt and decrypt login information. To log in to your instance, you must create a key pair, specify the name of the key pair when you launch the instance, and provide the private key when you connect to the instance. Linux instances have no password, and you use a key pair to log in using SSH. With Windows instances, you use a key pair to obtain the administrator password and then log in using RDP. You can use Amazon EC2 to create your key pair, or you could use a third-party tool and then import the public key to Amazon EC2.

Management

The decision to pick an instance store-backed Amazon EC2 instance or an Amazon EBS-backed Amazon EC2 instance has implications regarding both behavior and performance. Table 4.1 summarizes some of these differences.

TABLE 4.1 Instance Store-Backed Amazon EC2 Instance vs. an Amazon EBS-Backed Amazon EC2 Instance

Feature Amazon EBS-Backed Amazon EC2 Instance Instance Store-Backed Amazon EC2 Instance
Boot times Boots faster Takes longer to boot
Ability to reboot Can reboot Can reboot
Ability to stop Can stop instance Not able to stop instance, only terminate
Ability to resize Able to resize by stopping instance and choosing different instance size Not able to resize (not able to stop instance)
Ability to change CPU family Able to change CPU family by stopping instance and choosing different CPU family size Not able to change CPU family (not able to stop instance)
Ability to add or remove security groups from interfaces Able to do so while the Amazon EC2 instance is stopped, running, or rebooting Able to do so while the Amazon EC2 instance is running or rebooting

Amazon EC2 instances that use instance storage take longer to boot than Amazon EC2 instances that use Amazon EBS. Another impact to boot times is the use of user data when initially booting an instance. Having user data slows the boot process down. Using a fully configured AMI (which you can create) speeds up the boot process. Note that a fully configured AMI, because it is hard coded, is less flexible in terms of modification (you have to create a completely new AMI), so the trade-off is between speed of booting the instance and the flexibility of configuration.

Instance metadata is data about your instance that you can use to configure or manage the running instance. Metadata can be accessed via http://169.254.169.254/. Amazon EC2 instances can also include dynamic data, such as an instance identity document that is generated when the instance is launched and exposed to the instance through instance metadata. It validates the attributes of the instances, such as the subscribed software, instance size, instance type, operating system, and AMI. It can be accessed via http://169.254.169.254/latest/dynamic/instance-identity/document.

You can also access the user data that you supplied when launching your instance. For example, you can specify parameters for configuring your instance, downloading patches, installing software, or attaching a simple script. You can also use this data to build more generic AMIs that can be modified by configuration files supplied at launch time. For example, if you run web servers for various small businesses, they can all use the same AMI and retrieve their content from the Amazon S3 bucket that you specify in the user data at launch.


Snapshots

You can back up the data on your Amazon EBS volumes to Amazon S3 by taking point-in-time snapshots. Snapshots are incremental backups, which means that only the blocks on the device that have changed after your most recent snapshot are saved. This minimizes the time required to create the snapshot and saves on storage costs. When you delete a snapshot, only the data unique to that snapshot is removed. Active snapshots contain all of the information needed to restore your data (from the time the snapshot was taken) to a new Amazon EBS volume.


Monitoring the Status of Your AWS Cloud Services

AWS provides two dashboards that show the status of AWS Cloud services. The first is the AWS Service Health Dashboard. The dashboard displays service health across the entire AWS infrastructure, and you can personalize it to track your AWS Cloud services. With the AWS Service Health Dashboard, you can view the service health for all services across all regions (with the exception of the AWS GovCloud [US] Region and China [Beijing] Region). You are also able to subscribe to RSS feeds for chosen services across regions. In addition, you can view past system statuses for all services in all regions (again with the exception of the AWS GovCloud [US] Region and China [Beijing] Region).

The second dashboard is the AWS Personal Health Dashboard. The AWS Personal Health Dashboard provides information about AWS health events that can affect your account. The information is presented in two ways: a dashboard that shows recent and upcoming events organized by category and a full event log that shows all events from the past 90 days. AWS Personal Health Dashboard uses the Health API, which you can use to create your own health dashboards or feed into any current management systems you may have.

Monitoring with Amazon CloudWatch

Amazon CloudWatch monitoring is turned on by default for Amazon EC2 instances. Basic monitoring, which collects metrics every five minutes, is provided by default. You can enable detailed metrics that would collect metrics every minute. Amazon CloudWatch monitors the following metrics for Amazon EC2:

  • CPU utilization
  • Disk reads
  • Disk writes
  • Network in
  • Network out
  • Instance status check
  • System status check
  • CPU credit usage (T2 instances only)
  • CPU credit balance (T2 instances only)

You can view Amazon CloudWatch metrics based on a particular instance, a particular image, membership in an Auto Scaling group, or as an aggregate. Amazon CloudWatch can be used to trigger an alarm that can stop, terminate, reboot, or recover an Amazon EC2 instance.

Using AWS CloudTrail

You can use AWS CloudTrail to get a history of AWS API calls and related events for your account. This includes calls made using the AWS Management Console, AWS Software Development Kits (SDKs), command-line tools, and higher-level AWS Cloud services. When AWS CloudTrail logging is enabled, calls made to Amazon EC2 are tracked in log files, along with any other AWS Cloud service records. AWS CloudTrail determines when to create and write to a new file based on a specified time period and file size. All of the Amazon EC2 actions are logged. Every log entry contains information about who generated the request. The user identity information in the log helps you determine whether the request was made with root or IAM user credentials, with temporary security credentials for a role or federated user, or by another AWS Cloud service.


Amazon EC2 Systems Manager

Amazon EC2 Systems Manager is a management service that helps you automatically collect software inventory, apply operating system patches, create system images, and configure Windows and Linux operating systems. Amazon EC2 Systems Manager can be used for both Amazon EC2 instances and for compute instances located in your own datacenters. These capabilities help you define and track system configurations, prevent drift, and maintain software compliance of your Amazon EC2 and on-premises configurations.

IP Addresses

When an Amazon EC2 instance is spun up, a private IP address is assigned to the interface of that Amazon EC2 instance. What private IP address is used for this assignment is determined by what Amazon VPC and what subnet the Amazon EC2 instance is spun up in. The private IP address will be assigned from that subnet block. That private IP address remains assigned to that interface, and it is only released once the Amazon EC2 instance is deleted. Stopping the instance will have no effect on the private IP address. Public and Elastic IP addresses behave differently. Both are also assigned to an interface and can be assigned to only one interface (though an interface can have multiple public and Elastic IP addresses assigned to it). When an Amazon EC2 instance with a public IP address assigned to it is deleted, the public IP is returned to AWS so it can be reassigned to another Amazon EC2 instance (most likely in another AWS account). In contrast, when an Amazon EC2 instance with an Elastic IP address is deleted, the Elastic IP address is not returned to AWS; it is kept by the account, available for reassignment within that particular AWS account.

Purchasing Options

There are a number of options for purchasing Amazon EC2 instances: On-Demand, Reserved, or Spot. On-Demand is straightforward—you pay for the Amazon EC2 instance as you use it. There are no up-front costs or commitments, the minimum billing increment is one hour, and you do not pay for Amazon EC2 instances that are stopped.

Reserved instances provide you with a significant discount compared to On-Demand instance pricing. Reserved instances are not physical instances; they are a billing discount applied to the use of On-Demand instances in your account. With Reserved instances, you make a commitment for a specific family of instances in a specific region or Availability Zone. You can pay for Reserved instances monthly (like On-Demand), or you can prepay the full or partial amount.

Spot instances enable you to bid on unused Amazon EC2 instances, which can lower your Amazon EC2 costs significantly. The hourly price for a Spot instance (of each instance type in each Availability Zone) is set by Amazon EC2, and it fluctuates depending on the supply of and demand for Spot instances. Your Spot instance runs whenever your bid exceeds the current market price. Spot instances are a cost-effective choice if you can be flexible about when your applications run and if your applications can be interrupted.

Estimating Costs

AWS provides two tools for estimating costs: the AWS Simple Monthly Calculator and the AWS Total Cost of Ownership (TCO) Calculator. The AWS Simple Monthly Calculator incorporates a wide array of pricing calculations across all AWS Cloud services in all AWS Regions. It also shows the breakdown of features for each service in each region. The AWS Simple Monthly Calculator has a mechanism to select a service in a particular region and add it to the total bill.

The AWS TCO Calculator allows you to estimate the cost savings when using AWS and provides a detailed set of reports that can be used in executive presentations. The AWS TCO Calculator also gives you the option to modify assumptions that best meet your business needs.

Tracking Costs

AWS provides AWS Trusted Advisor and My Billing Dashboard for managing and tracking costs. The tools can be found in the AWS Management Console. AWS Trusted Advisor gives you recommendations on how to potentially reduce your spending with AWS. The My Billing Dashboard is specifically dedicated to your charges and bills. With My Billing Dashboard, you can view and download bills, explore your bills (graph, visualize, and analyze), create budgets, and receive notifications based on those budgets. You can also create Cost Allocation Tags, which allow you to associate costs with resources that you create.

Creating Your Own AMIs

As discussed in the implementation section, you can use your own AMIs when choosing an AMI. The process of creating an AMI is different depending on whether the AMI you are creating is an instance store-backed AMI or if it is an Amazon EBS-backed AMI.

To create an instance store-backed AMI, first launch an instance from an AMI that is similar to the AMI that you would like to create. You can connect to your instance and customize it. When the instance is set up the way you want it, you can bundle it. It takes several minutes for the bundling process to complete. After the process completes, you have a bundle, which consists of an image manifest (image.manifest.xml) and files (image.part.xx) that contain a template for the root volume. Upload the bundle to your Amazon S3 bucket and then register your AMI.

To create an Amazon EBS-backed AMI, first launch an instance from an AMI that is similar to the AMI that you would like to create. You can connect to your instance and customize it. After the instance is configured correctly, stop the instance (stopping the instance ensures data integrity of the instance; a snapshot will be taken of the instance). Next create the image. When you create an Amazon EBS-backed AMI, AWS automatically registers it for you.

Once you have created an AMI, you can control who has access to it. You can share the AMI among AWS accounts, and you can copy the AMI to other AWS Regions. Note that if the AMI is encrypted, you are not able to share it.

Placement Groups

A placement group is a logical grouping of instances in a single Availability Zone. Placement groups are recommended for applications that benefit from low network latency, high network throughput, or both. High Performance Computing (HPC) is an example of an application that would potentially benefit from being in a placement group. Because all instances are within a single Availability Zone, Amazon EC2 instances in the same placement group cannot, by definition, be either in different Availability Zones or in different regions. Amazon EC2 instances that are in the same placement group communicate with each other via either the private IPv4 address or the IPv6 address.

Security

Security in the Amazon EC2 environment is a responsibility shared by both the end user and AWS. This is because, within this environment, there are specific parts that AWS controls and specific parts that are controlled by the end user. Physical security and security of the Hypervisor is AWS responsibility. The end user, on the other hand, is responsible for securing the operating systems running on their instances as well as the applications running on those operating systems. You retain control of what security you choose to implement to protect your content, platform, applications, systems, and networks, no differently than you would for applications in an on-site datacenter.

Regarding Amazon EC2, you are responsible for controlling who has access to the machine (both network access and administrative access) and for keeping the guest operating system up-to-date and patched properly. If you so desire, you can apply additional layers of security (for example, running IP Tables or Windows Defender on the guest operating system) without needing authorization from AWS. AWS does not have any access rights to your instances or to the guest operating system.

Controlling Network Access

AWS provides a number of tools regarding network access: placement of the Amazon EC2 instance in an Amazon VPC (Amazon VPC security was discussed in Chapter 3) and security groups to control access on a specific interface. With administrative access, you again can control access with placement of Amazon EC2 instances in an Amazon VPC, the use of security groups, and the use of public/private key pairs both to encrypt traffic and control access.

Controlling Administrative Access

AWS provides a number of tools to manage the security of your instances. These tools include the following:

  • IAM
  • AWS Trusted Advisor
  • AWS Service Catalog
  • Amazon Inspector

IAM allows you to create policies that control access to APIs and apply those policies to users, groups, and roles.

AWS Trusted Advisor, which comes as part of AWS Support, looks at things like open ports on security groups and level of access allowed to Amazon EC2 instances, and it makes recommendations to improve the security of your AWS infrastructure.

AWS Service Catalog allows IT administrators to create, manage, and distribute portfolios of approved products to end users who can then access the products they need in a personalized portal. AWS Service Catalog allows organizations to manage commonly deployed IT services centrally, and it helps organizations achieve consistent governance and meet compliance requirements while enabling users to deploy quickly only the approved IT services they need within the constraints the organization sets.

Amazon Inspector is a security vulnerability assessment service that helps improve the security and compliance of your AWS resources. Amazon Inspector automatically assesses resources for vulnerabilities or deviations from best practices and then produces a detailed list of security findings prioritized by level of severity.


Amazon EC2 Container Service (Amazon ECS)

Amazon ECS is a highly scalable container management service. Amazon ECS makes it easy to run, stop, and manage Docker containers on Amazon EC2 instances.

With Amazon ECS, you can launch and place containers across your cluster using API calls. You schedule the placement of containers based on your resource needs, isolation policies, and availability requirements. You can use Amazon ECS both for cluster management and for scheduling deployments of containers onto hosts.

Amazon ECS eliminates the need for you to operate your own cluster management and configuration management systems. Amazon ECS can be used to manage and scale both batch and Extract, Transform, Load (ETL) workloads. It can also be used to build application architectures based on the microservices model.

Amazon ECS is a regionally-based service that can be used to run application containers in a highly available manner across all Availability Zones within an AWS Region.

Implementation

To implement Amazon ECS, you need to install an Amazon ECS agent on your Amazon EC2 instances. If you use Amazon ECS-optimized AMIs, that agent is already installed. Additionally, the container instance needs to have an IAM role that authenticates to your account and will need external network access to communicate with the Amazon ECS service endpoint.

Clusters are the logical grouping of container instances that you can place tasks on. Clusters are region specific and can contain multiple, different instance types (though an instance can only be a member of a single cluster).

Amazon ECS obtains a Docker image for repository. This repository can be on AWS or on other infrastructure.

Deploying a Container

To deploy a container, you need to do the following:

  1. Define a task. This is where you assign the name, provide the image name (important for locating the correct image), and decide on the amount of memory and CPU needed.
  2. Define the service. In this step, you decide how many instances of the task you want to run in the cluster and any Elastic Load Balancing load balancers that you want to register the instances with.
  3. Create the Amazon ECS cluster. This is where the cluster is created and also where you specify the number of instances to run. The cluster can run across multiple Availability Zones.
  4. Create the stack. A stack of instances is created based on the configuration information provided. You can monitor the creation of the stack in the AWS Management Console. Creation of the stack is usually completed in less than five minutes.

Management

Amazon ECS can be configured using the AWS Management Console, the AWS CLI, or the Amazon ECS CLI.

Monitoring Amazon ECS

The primary tool used for monitoring your Amazon ECS clusters is Amazon CloudWatch. Amazon CloudWatch collects Amazon ECS metric data in one-minute periods and sends them to Amazon CloudWatch. Amazon CloudWatch stores these metrics for a period of two weeks. You can monitor the CPU and memory reservation and utilization across your cluster as a whole and the CPU and memory utilization on the services in your cluster. You can use Amazon CloudWatch to trigger alarms, set notifications, and create dashboards to monitor the services.

Once it’s set up, you can view Amazon CloudWatch metrics in both the Amazon ECS console and the Amazon CloudWatch console. The Amazon ECS console provides a maximum 24-hour view while the Amazon CloudWatch console provides a fine-grained and customizable view of running services.

The other tool available is AWS CloudTrail, which will log all Amazon ECS API calls.

AWS Trusted Advisor is another source for monitoring all of your AWS resources, including Amazon ECS, to improve performance, reliability, and security.

There is no additional cost for using Amazon ECS. The only charges are for the Amazon EC2 instances or AWS Lambda requests and compute time.

Security

With Amazon ECS, you need to do the following:

  1. Control who can create task definitions.
  2. Control who can deploy clusters.
  3. Control who can access the Amazon EC2 instances.

IAM is the tool used for the first two necessities. For controlling access to Amazon EC2 instances, the tools described in the Amazon EC2 section still apply. You can use IAM roles, security groups, and (because these Amazon EC2 instances are located in an Amazon VPC) network Access Control Lists (ACLs) and route tables to control access to the Amazon EC2 instances.

AWS Elastic Beanstalk

AWS Elastic Beanstalk enables you to deploy and manage applications in the AWS Cloud without worrying about the infrastructure that runs those applications. AWS Elastic Beanstalk reduces management complexity without restricting choice or control. You simply upload your application, and AWS Elastic Beanstalk automatically handles the details of capacity provisioning, load balancing, scaling, and application health monitoring.

Languages Supported in AWS Elastic Beanstalk

AWS Elastic Beanstalk supports applications developed in the following languages:

  • Java
  • PHP
  • .NET
  • Node.js
  • Python
  • Ruby

Services that AWS Elastic Beanstalk Deploys

AWS Elastic Beanstalk automates the deployment of Amazon VPC, multiple compute instances (across multiple Availability Zones), security groups (both for instances and load balancers), Auto Scaling, Amazon S3 buckets, Amazon CloudWatch alarms, and domain names. Even though AWS Elastic Beanstalk automates the deployment of your environment, you can make modifications as you see fit.

Implementation

There are a number of tools that you can use to create and manage the AWS Elastic Beanstalk environment. These tools are as follows:

  • AWS Management Console
  • AWS Elastic Beanstalk CLI
  • AWS SDK for Java
  • AWS Toolkit for Eclipse
  • AWS SDK for .NET
  • AWS Toolkit for Visual Studio
  • AWS SDK for Node.js
  • AWS SDK for PHP (requires PHP 5.2 or later)
  • Boto (AWS SDK for Python)
  • AWS SDK for Ruby

You deploy a new application on AWS Elastic Beanstalk by uploading a source bundle. This source bundle can be a ZIP or a WAR file (you can include multiple WAR files inside of your ZIP file). This source bundle cannot be larger than 512 MB.

Management

AWS Elastic Beanstalk is by definition highly available. It creates multiple compute instances, locates those compute instances in two Availability Zones, and creates a load balancer (a Classic Elastic Load Balancing load balancer) to spread the compute load across these multiple instances.

You can configure Auto Scaling to establish a minimum number of instances, a maximum number of instances, and what parameters would trigger either an increase or decrease in the number of instances. These triggers can either be time-based or event-based.

When you go into the AWS Elastic Beanstalk console, it displays all of the environments running in that region, sorted by application. You can further drill down by application name to see the environments, application versions, and saved configurations. There are limits on the number of applications, application versions, and environments that you can have.

As with Amazon EC2, you are responsible for patching and updating both the guest operating system and your application with AWS Elastic Beanstalk.

Making Changes in Your AWS Elastic Beanstalk Environment

It is recommended to use the AWS Elastic Beanstalk console and CLI when making changes to your AWS Elastic Beanstalk environments and configuration. Although it is possible to modify your AWS Elastic Beanstalk environment using other service consoles, CLI commands, or AWS SDKs, those changes will not be reflected in the AWS Elastic Beanstalk console and you will not be able to monitor, change, or save your environment. It will also cause issues when you attempt to terminate your environment without using the AWS Elastic Beanstalk console.

Monitoring Your AWS Elastic Beanstalk Environment

You can monitor the overall health of your environment using either the basic health reporting or enhanced health reporting and monitoring systems. Basic health reporting gives an overall status (via color key) of the environment. It does not publish any metrics to Amazon CloudWatch. Enhanced health reporting and monitoring not only provides status via color key, but it also provides a descriptor that helps indicate the severity and cause of the problem. You can also configure enhanced health reporting and monitoring to publish metrics to Amazon CloudWatch as custom metrics.

You can retrieve log files from the individual instances or have those log files uploaded to an Amazon S3 bucket for more permanent storage.

Security

When you create an environment with AWS Elastic Beanstalk, you are prompted to create two IAM roles:

Service role This is used by AWS Elastic Beanstalk to access other AWS Cloud services.

Instance profile This is applied to the instances created in your environment, and it allows them to upload logs to Amazon S3 and perform other tasks.

In addition to these two roles, you can create polices in IAM and attach them to users, groups, and roles. Just as in Amazon EC2, in AWS Elastic Beanstalk you need to create an Amazon EC2 key pair to access the guest operating system within your compute instances. Access would be via either Port 22 (for SSH) or Port 3389 (for RDP).


AWS Lambda

AWS Lambda is a serverless compute service. It runs your code in response to events. AWS Lambda automatically manages the underlying compute resources with no server configuration from you. There is no need to provision or manage servers.

The execution of code (called an AWS Lambda function) can be triggered by events internal to your AWS infrastructure or external to it.

AWS Lambda is a regionally-based service running across multiple Availability Zones. This makes it highly available and highly scalable. Because the code is stateless, it is executed almost automatically without deployment or configuration delays.

Implementation

Implementation for AWS Lambda involves the following steps:

  1. Configure an AWS Lambda function.
  2. Create a deployment package.
  3. Upload the deployment package.
  4. Create an invocation type.

The languages available to write your code are Node.js, Java, Python, and C#. The language that you choose also controls what tools are available for authoring your code. These tools can include the AWS Lambda console, Visual Studio, .NET Core, Eclipse, or your own authoring environment. You use these languages to create a deployment package.

Once a deployment package has been created, you then upload it to create an AWS Lambda function. The tools you can use to do so are the AWS Lambda console, CLI, and AWS SDKs.

The code that you have written is part of what is called an AWS Lambda function. The other elements of an AWS Lambda function are as follows:

  • Memory
  • Maximum execution time
  • IAM role
  • Handler name

The amount of memory that you specify controls the CPU power. The default amount of memory allocated per AWS Lambda function is 512 MB, but this can range from 128 MB to 1,536 MB in 64 MB increments.

Because you pay for the resources that are used to run your AWS Lambda function, the purpose of a timeout is to prevent your AWS Lambda function from running indefinitely. You can choose a value ranging between 1 and 300 seconds (5 minutes). When the specified timeout is reached, the AWS Lambda function is terminated.

The IAM role controls the level of access that the specific AWS Lambda function has. This is important when using AWS Lambda to access other AWS resources.

The handler refers to the method in your code where AWS Lambda begins execution. AWS Lambda passes any event information that triggered the invocation (further information on invocation and invocation types follows) as a parameter to the handler method.

When an AWS Lambda function is invoked, AWS Lambda launches a container based on your configuration specifications. Because it is launching a container, there may be some latency the first time that the AWS Lambda function is invoked. AWS Lambda will maintain the container for some time after it has been invoked to speed up response time for subsequent invocations.

How AWS Lambda Functions Are Invoked

AWS Lambda functions can be invoked either synchronously or asynchronously. If you are using AWS Cloud services to invoke AWS Lambda functions, those AWS Cloud services control whether the function is invoked synchronously or asynchronously. For example, if Amazon S3 invokes an AWS Lambda function, it does so asynchronously. In contrast, Amazon Cognito invokes an AWS Lambda function synchronously.

AWS Lambda functions are invoked by the following:

Push event model Some event sources can publish events to AWS Lambda and directly invoke your AWS Lambda function. The push model applies to Amazon S3, Amazon SNS, Amazon Cognito, Amazon Echo, and user applications, where each event triggers the AWS Lambda function.

Pull event model Some event sources publish events, but AWS Lambda must poll the event source and invoke your AWS Lambda function when events occur. This model applies when AWS Lambda is used with streaming event sources such as Amazon Kinesis and Amazon DynamoDB Streams. For example, AWS Lambda polls your Amazon Kinesis stream, or Amazon DynamoDB Stream, and it invokes your AWS Lambda function when it detects new records on the stream.

Direct invocation model This invocation type causes AWS Lambda to execute the function synchronously and returns the response immediately to the calling application. This invocation type is available for custom applications.


Management

You can test your AWS Lambda function either in the AWS Lambda console or by using the CLI. Amazon CloudWatch provides metrics for AWS Lambda. These metrics include, but are not limited to, the following:

  • Invocation count
  • Invocation duration
  • Number of requests
  • Latency per request
  • Duration of code execution

AWS CloudTrail will record the API calls used for the AWS Lambda functions. This provides another source of information for troubleshooting.

The AWS Lambda function is executed once it is invoked. If the execution fails, two more attempts are made. You can, as an option, configure AWS Lambda to forward a failed payload to a Dead Letter Queue (DLQ). This DLQ can take the form of an Amazon Simple Notification Service (Amazon SNS) topic or an Amazon Simple Queue Service (Amazon SQS) queue.

Security

The underlying compute infrastructure for AWS Lambda is the Amazon Linux AMI. Because AWS is responsible for maintaining this infrastructure (in terms of patching and upgrading), you are not able to log in to these compute instances or customize the operating system or language runtimes.

Each AWS Lambda function has a role associated with it. This role controls what AWS resources this specific AWS Lambda function has access to. As such, you need to grant the necessary level of permissions to that role.

You also manage permissions using an AWS Lambda function policy. AWS Lambda function policies allow you to have an event (for example, an event notification on an Amazon S3 bucket) to invoke an AWS Lambda function. AWS Lambda function policies are also used to invoke an AWS Lambda function when the source event is from outside the AWS account containing the AWS Lambda function.


Amazon Lightsail

Amazon Lightsail is designed to give you the ability to launch and manage a Virtual Private Server (VPS) easily with AWS. When you launch an Amazon Lightsail instance, you pick the operating system, the developer stack, or application, and the size of the instance that you wish to run. Amazon Lightsail instances come with SSD-backed storage, a publicly accessible IP address, DNS management, and data transfer capability.

Amazon Lightsail is designed to make it easy to spin up single servers. Amazon Lightsail is especially helpful for developers (both professional and hobbyist) because it enables them to focus on development instead of installing software or frameworks.

Implementation

You can access Amazon Lightsail via a console or the CLI. Access via the console can be done either through the AWS Management Console or through the separate Amazon Lightsail console.

Implementing Amazon Lightsail involves creating your instance by choosing an operating system, an application, or a developer stack.

OS Choices

The operating system choices for Amazon Lightsail are Amazon Linux or Ubuntu.

Applications and Developer Stack Choices

The following are application choices for Amazon Lightsail:

  • Drupal
  • Joomla
  • Redmine
  • GitLab
  • WordPress
  • Magneto
  • Nginx

The following are the developer stacks choices for Amazon Lightsail:

  • LAMP
  • LEMP
  • MEAN
  • Node.js

After choosing an application or developer stack, you pick your instance plan, which determines the number of Virtual CPUs (vCPUs), amount of RAM, amount of disk space, and expected data transfer amounts. Assign a unique name (the name needs to be unique within your account), decide on the number of instances you want to spin up (the maximum is 20), and create your instance.

Your Amazon Lightsail instance is spun up in its own Amazon VPC.

Management

Once the instance is running, you can stop the instance, reboot it, or delete it. Billing stops once an instance is deleted.

Managing Amazon Lightsail involves managing the instance itself and managing the guest operating system and application or developer stack. Managing the instance can be done either via the Amazon Lightsail console or the CLI. Managing the other aspects can be done by accessing the instance via SSH connection offered in the Amazon Lightsail console or via your own SSH client. Accessing via your own SSH client involves downloading a private key from your AWS account page.

The Amazon Lightsail console provides you with access to five management features: Connect, Metrics, Networking, Snapshots, and History. Connect gives you the ability to log in to the guest operating system via the Amazon Lightsail console (versus a separate SSH client).

Available Metrics

Metrics provides you with information on the following:

  • CPU utilization
  • Network in
  • Network out
  • Status check failed instance (indicates if the specific instance is failing)
  • Status check failed system (indicates if the underlying AWS hardware is failing)

All of these metrics are offered in one-hour, six-hour, one-day, one-week, and two-week increments.

Networking

Networking gives you information on your IP addresses, both the publicly accessible IP address (the address accessible from the Internet) and the private IP address (the address accessible internally). You can change the public IP to become a static IP address. Networking also lists the ports open on the firewall and provides the ability to open or close ports as needed, giving you control over who has access to your instance.

Snapshots is where you create and store snapshots. This is also where you can create new instances based on snapshots. There is no limit to the number of snapshots you can take and store. Note that you are charged for storing snapshots.

History lists when an instance was created, stopped, and rebooted; provides information on snapshots taken; and provides information on attached or detached static IP addresses.

With Amazon Lightsail, you can keep the public IP address assigned to the instance, assign a static public IP address, or create a DNS zone. Public IP addresses in Amazon Lightsail behave very similarly to public IP addresses in Amazon EC2; when you stop an Amazon Lightsail instance, the public IP address is removed; when you restart that instance, a new IP address is assigned. If you need IP address persistence, you should assign a static public IP address to the instance. A DNS zone supports A, CNAME, MX, and TXT records. You can use Amazon Route 53 to register your domain.

Costs

Costing for Amazon Lightsail is done on an hourly basis, with the minimum billing increment being one hour. While there is a monthly maximum for each different instance size, it is possible to have additional charges based on the following:

  1. Storage costs of any snapshots that you take and keep
  2. Charges for data transfer in excess of the amount allocated
  3. Any public IP addresses that are not assigned to an instance

Changing Instance Size

Changing the instance size for Amazon Lightsail involves taking a snapshot of the existing instance and spinning up a new instance in the larger size. It is not possible to modify the number of vCPUs, RAM, or SSD size of an individual instance.

Security

Security with Amazon Lightsail follows the AWS shared responsibility model. Access to the compute instance is controlled via AWS tools (specifically IAM), although access to the guest operating system, application, or developer stack is controlled by you. Keeping the guest operating system, application, or developer stack current and patched is your responsibility.

AWS facilitates control over access to the guest operating system, application, and developer stack by only opening relevant ports for that particular operating system, application, or developer stack. Additional ports can be opened and existing ports can be modified.

AWS Batch

AWS Batch enables you to run batch computing workloads on AWS. These workloads can be at any scale. AWS Batch allows the end user to plan, schedule, and execute batch jobs while being able to control costs.

Batch jobs, by their nature, run asynchronously and have interdependencies between jobs. These interdependencies make the ability to control scheduling and sequencing very important. AWS Batch provides the tools to control the scheduling and sequencing of jobs.

Implementation

AWS Batch uses terminology that should be familiar to users of on-premises batch systems:

  • Job
  • Job definition
  • Job queue
  • Scheduler
  • Compute environment

Jobs are the unit of work executed by AWS Batch. Jobs can be executed as AWS Lambda functions or as containerized applications running on Amazon EC2.

Job definitions are a similar concept to Amazon ECS task definitions—they specify how jobs are to be run. Some of the attributes specified in a job definition include vCPU requirements, memory requirements, mount points, and environmental variables, among other items.

Job queue is where jobs reside until the point that they are scheduled to a compute resource. You can have multiple job queues and set priorities using job queues. For example, you can specify having all jobs in a specific queue complete before jobs in another queue are attempted.

The scheduler evaluates the jobs in the job queue and makes decisions on when, where, and how to run those jobs. Currently, the algorithm is First In, First Out (FIFO), as long as all dependencies on other job resources have been met.

The compute environment can be of two types: managed and unmanaged. In a managed compute environment, you describe the needed requirements (such as instance types, minimum/maximum/desired vCPUs, use of Spot instances), and AWS launches and scales your resources. You specify in which Amazon VPC and subnet the compute instances are launched. In an unmanaged compute environment, you launch and manage the compute resources and need to include the Amazon ECS agent and run supported versions of Linux and Docker.

Implementing an AWS Batch Job

Steps to implementing a batch job are as follows:

  1. Define a job.
  2. Configure the compute environment.
  3. Configure the job queue.
  4. Submit a job.

Once a job is submitted it will go through the following states:

Submitted Submitted to the queue but has not yet been evaluated by the scheduler.

Pending Submitted to the queue and evaluated but not yet running due to a dependency on another job or resource.

Runnable In the queue with no outstanding dependencies and is ready to be scheduled.

Starting Scheduled to a host and container initiation operations are underway.

Running The job is running.

Succeeded The job has finished successfully.

Failed The job has failed all available attempts.

If a job fails, you can automate retry attempts up to 10 times.

Management

There are service limits to AWS Batch. You can have up to 10 compute environments, 5 job queues, and 3 compute environments per job queue. These limits can be changed via the AWS Management Console or the AWS CLI. You can also have up to 20 job dependencies and 1,000,000 jobs in a submitted state. The maximum job size is 20 KB. These three limits cannot be changed.

Monitoring

You can monitor your AWS Batch environment either via the CLI or the AWS Batch dashboard. Logs for your jobs (for example, STDERR and STDOUT) are available in the AWS Management Console and can be written to Amazon CloudWatch Logs. You can use SSH to access the Amazon EC2 instances that have been spun up to process your AWS Batch job.

Costs

There are no additional costs for AWS Batch. The only costs are for the Amazon EC2 resources or the AWS Lambda resources used. It is possible to use Spot pricing for all of your compute instances in order to minimize costs.

Security

AWS Batch uses IAM to control access to AWS resources that are used. Through IAM, you can define policies for different users in your organization to have different levels of access. Because you control the Amazon VPC in which you create the compute resources, you can implement additional layers of security with the use of private subnets, network ACLs, and security groups.

Summary

In this chapter, we discussed the following services:

  • Amazon EC2
  • Amazon ECS
  • AWS Elastic Beanstalk
  • AWS Lambda
  • Amazon Lightsail
  • AWS Batch

It is important to understand what each service does and does not do, even though the exam will not ask you about actual commands to launch an instance or to code an AWS Lambda function.

Exam Essentials

Know the basics about launching an Amazon EC2 instance. Launching an Amazon EC2 instance involves choosing an AMI (and knowing what are the sources for an AMI), choosing an instance type (and knowing what an instance type controls), and configuring user data.

Understand the pricing model for Amazon EC2. With Amazon EC2, there are three different pricing models: On-Demand, Reserved instances, and Spot. Understand the circumstances that would lead you to choose one pricing model over another.

Know what Amazon EC2 instance metadata is and how to find it. Instance metadata is data about your instance that you can use to configure or manage the running instance. Examples of instance metadata include AMI-id, public IP address, any roles assigned to the Amazon EC2 instance, and hostname. Metadata can be retrieved by logging in to the instance and using the following Uniform Resource Identifier (URI): http:169.254.169.254/latest/meta-data.

Know what user data is, how it is loaded into the Amazon EC2 instance, and how to find it. Instance user data is loaded on the initial boot of the Amazon EC2 instance, and it can be loaded in subsequent boots. User data is limited to 16 KB. User data can be used to load additional software and perform other functions. User data can be retrieved by logging into the instance and using the following URI: http:169.254.169.254/latest/user-data.

Know how to create an AMI. Although there are four sources for AMIs, the major source will be AMIs that you create. Creating your own AMIs gives you greater control over the actual instance. This control extends to boot times (instances spun up from AMIs boot faster than those that need to load user data), operating systems, and instance size.

Know when you would use a dedicated host and when you would use a dedicated instance. An Amazon EC2 Dedicated Host is a physical server with Amazon EC2 instance capacity fully dedicated to your use. Dedicated hosts allow you to use your existing per-socket, per-core, or per-VM software licenses. Rebooting a dedicated host will reboot the instance on the same physical AWS infrastructure. Dedicated instances are Amazon EC2 instances that run in an Amazon VPC on hardware that is dedicated to a single customer. Your dedicated instances are physically isolated at the host hardware level from instances that belong to other AWS accounts.

Understand the different CPU types available in Amazon EC2 and how to change instance types on running instances. The CPU types offered allow you to pick CPUs that are optimized for compute, storage, I/O, and graphics, among other considerations.

Know the different storage options available for Amazon EC2. Depending on the instance type chosen, you can have instance storage, Amazon EBS storage, or Amazon EBS-optimized storage.

Understand how instance size affects network speed. With the exception of certain instance types (specifically S and X instance types), network speed is a function of the size of the instance—larger instances have higher network speed. For greater network speed, some instance types support enhanced networking. Enhanced networking involves installing the correct elastic network interface and configuring the correct drivers.

Understand what an Amazon EBS-optimized instance is and when you would use it. An Amazon EBS-optimized instance uses an optimized configuration stack and provides additional, dedicated capacity for Amazon EBS I/O. This optimization provides improved performance for your Amazon EBS volumes by minimizing contention between Amazon EBS I/O and other traffic from your instance. Amazon EBS-optimized instances deliver dedicated throughput between Amazon EC2 and Amazon EBS, with options between 500 Mbps and 1,000 Mbps, depending on the instance type used.

Understand how to log in remotely to an Amazon EC2 instance. Refer to the Amazon EC2 User Guide for complete instructions.

Know what Amazon EC2 metrics are monitored in Amazon CloudWatch. Amazon CloudWatch collects metrics on CPU utilization, disk reads and writes, network in and out, status check failed instance, and status check failed system. It is important to remember that by default Amazon CloudWatch does not collect metrics beyond the Hypervisor level. Amazon CloudWatch collects log files, by default, in five-minute intervals.

Know how to configure an AWS Lambda function. AWS Lambda functions can be written in Node.js, Java, Python, and C# with a variety of tools available to create and edit code. An AWS Lambda function also has allocated memory, a maximum execution time set, an IAM role assigned (if it needs to access other AWS resources), and a handler name.

Understand the pricing model for AWS Lambda. Pricing for AWS Lambda has two components: total number of requests and duration of time your code executes. Duration is calculated from the time your code begins executing until it returns or otherwise terminates.

Know what AWS Lambda metrics Amazon CloudWatch monitors. Amazon CloudWatch will monitor AWS Lambda invocation count, invocation duration, number of requests, latency per request, and duration of code execution among other metrics.

Understand how to control CPU choice in AWS Lambda. CPU power is allocated by AWS based on the amount of memory allocated. The default amount of memory allocated is 512 MB, but that can be configured from 128 MB to 1,536 MB in 64 MB increments.

Understand the use cases for AWS Elastic Beanstalk, Amazon Lightsail, and AWS Batch. AWS Elastic Beanstalk provisions necessary infrastructure resources, such as the load balancer, Auto Scaling group, security groups, and the database (optional), and provides a unique domain name for your application infrastructure stack. AWS Elastic Beanstalk is oriented toward web developers who want to launch highly scalable, highly available web applications quickly and easily. Amazon Lightsail is most suitable for projects that require a few VPSs and for developers looking for a no-nonsense, simple management interface. AWS Batch enables developers, scientists, and engineers to run hundreds of thousands of batch computing jobs easily and efficiently on AWS.

Know what an AWS Lambda event is and what are supported AWS Lambda event sources. AWS Lambda events are events that trigger the execution of an AWS Lambda function. These events can be internal to AWS (for example, an Amazon CloudWatch notification, adding an object to an Amazon S3 bucket) or external to AWS.

Understand what services discussed in this chapter are considered highly available and what services are not. Services that are Availability Zone based (like Amazon EC2 or Amazon Lightsail) are not considered highly available. Services that are AWS Region-based (like Amazon ECS and AWS Lambda) are considered highly available. Some services (like AWS Elastic Beanstalk) may or may not be highly available depending on how they have been configured.

Know what services AWS Elastic Beanstalk spins up by default. AWS Elastic Beanstalk spins up a default Amazon VPC (with an Internet gateway attached), subnets in multiple Availability Zones, compute instances in those Availability Zones, and Elastic Load Balancing with a public DNS name.

Know the different tools available to configure the services mentioned in this chapter. For all of the services mentioned in this chapter, with the exception of Amazon Lightsail, you can use the AWS Management Console to configure and manage them. Amazon Lightsail has its own management console that should be used for configuration, which can be accessed either directly or via the AWS Management Console. The CLI can be used to configure and manage all services mentioned in the chapter. Additionally, AWS Elastic Beanstalk and Amazon ECS have their own CLIs.

Know the different tools available to monitor Amazon EC2, Amazon ECS, and AWS Lambda. The two important tools for monitoring Amazon EC2, Amazon ECS, and AWS Lambda are Amazon CloudWatch and AWS CloudTrail. Amazon CloudWatch can receive log data from Amazon EC2, Amazon ECS, and AWS Lambda. This log data can be viewed in the AWS Management Console, the Amazon CloudWatch console, or a third-party console. Based on that log data, you can create dashboards, send notifications, and trigger alarms. AWS CloudTrail records the calls made to the AWS APIs using the AWS Management Console, the AWS Command Line Interface (AWS CLI), your own applications, and third-party software, and then publishes the resulting log files to the Amazon S3 bucket of your choice. AWS CloudTrail can also issue a notification to an Amazon SNS topic of your choice each time a file is published. Each call is logged in JSON format.

Know the different tools available to manage security for Amazon EC2. The principal tool for controlling access to AWS Cloud services is IAM. Access to interfaces on compute instances in Amazon EC2, Amazon ECS (when using Amazon EC2 instances), and AWS Elastic Beanstalk (which also uses Amazon EC2 instances) is controlled via port, protocol, and source using security groups. Interfaces on compute instances in Amazon Lightsail are protected by firewalls, which operate in a similar manner to security groups. Remote access (either via RDP or SSH) is controlled by both security groups (the proper protocol must be enabled on the interface) and the use of public/private keys.

Understand the AWS shared responsibility model and how it applies to Amazon EC2, Amazon ECS, AWS Lambda, and AWS Elastic Beanstalk. The AWS shared responsibility model means that AWS is responsible for security of the cloud while the AWS customer is responsible for security in the cloud. For Amazon EC2, Amazon ECS, AWS Elastic Beanstalk, and Amazon Lightsail, the customer has access to and control over the guest operating system running on the instance; therefore, the AWS customer is responsible for making sure that the guest operating system is properly patched and kept upgraded. For AWS Lambda, the customer does not have access to the operating system running the compute instance; therefore, AWS is responsible for making sure that the operating system is properly patched and upgraded.

Understand what security groups do and how they protect instances. Security groups control access on an interface level via port, protocol, and source. A source can be a single IP address, a range of IP addresses, or another security group. Security groups can be configured only to allow a connection, not to deny a connection.

Understand when you would use Amazon EC2 and when you would use Amazon Lightsail. Amazon Lightsail gives you a limited number of configuration options in terms of number of CPUs, amount of memory, and amount of storage. Amazon Lightsail is more applicable when you need a system that is preconfigured and preassembled and is ready to run your web applications with no system-building effort on your part. Amazon EC2, on the other hand, gives you many more options in terms of number and types of CPUs, amount of memory, and amount of storage. Amazon EC2 is more applicable when you want to (or need to) take the time to hand-select individual AWS components (for example, servers, storage, or IP addresses) and put them together on your own.

Know what services Amazon Lightsail spins up by default. With Amazon Lightsail, you choose an operating system (either AWS Linux or Ubuntu), then you can add a developer stack (such as LAMP) or an application (WordPress, for example). The plan you choose determines the number of CPUs, amount of memory, and amount of storage. A firewall is created to protect your instance with only the needed ports open. You can also add static IP addresses and DNS-hosted zones.


Resources to Review

Exercises

By now you should have set up an account in AWS. If you haven’t, now would be the time to do so. It is important to note that these exercises are in your AWS account and thus are not free.

Use the Free Tier when launching resources. The AWS Free Tier applies to participating services across the following AWS regions: US East (Northern Virginia), US West (Oregon), US West (Northern California), Canada (Central), EU (London), EU (Ireland), EU (Frankfurt), Asia Pacific (Singapore), Asia Pacific (Tokyo), Asia Pacific (Sydney), and South America (Sao Paulo). For more information, see https://aws.amazon.com/s/dm/optimization/server-side-test/free-tier/free_np/.

If you have not yet installed the AWS Command Line utilities, refer to Exercise 2.1 (Linux) or Exercise 2.2 (Windows).









Review Questions

  1. Because of a change in your web traffic, you realize that you need a larger instance size for your web server. What is the best way to make this change?

    1. Take a snapshot of your current instance, and use that as the basis for creating a new web server in the correct size.
    2. Create a new Amazon Machine Image (AMI), and use that to spin up a new web server with the correct instance type.
    3. Stop the instance, and then restart it with the correct instance type.
    4. Assign an Elastic IP to the instance interface. Create a new instance with the right instance size. Reassign the Elastic IP from the old instance to the new instance.
  2. You have configured a batch job on AWS Batch and you need it to complete. What should you do in order to make sure that the batch job completes?

    1. Configure the pricing to be Spot pricing, but set the bid price at two times the On-Demand price for those instances.
    2. Configure the pricing to be On-Demand.
    3. Configure the pricing to be Spot pricing, but set the bid price for one-tenth of the On-Demand price for those instances.
    4. There is nothing that you can do to guarantee completion of a batch job.
    5. Configure the number of instances needed for the job as two times what is actually needed.
  3. With AWS Batch, what values are you allowed to specify when configuring your instances?

    1. Minimum number of Virtual CPUs (vCPUs), maximum number of vCPUs, and desired number of vCPUs
    2. Minimum and maximum number of vCPUs
    3. Desired number of vCPUs
    4. You are not able to specify the number of vCPUs. AWS will assign the number based on current demand and availability.
  4. You attempt to use Secure Shell (SSH) to access an Amazon Elastic Compute Cloud (Amazon EC2) instance but are unable to do so. What would NOT be a reason why you are unable to do so successfully?

    1. You have an incorrect key.
    2. The security group attached to the interface does not allow SSH.
    3. You have an incorrect role assigned to your Amazon EC2 instance.
    4. The Amazon EC2 instance does not have a public IP address attached to it.
  5. You need to create an Amazon Elastic Compute Cloud (Amazon EC2) instance with the fastest possible boot time. What combination will give you an Amazon EC2 instance with the fastest boot time?

    1. An instance store-backed Amazon Machine Image (AMI) with user data
    2. An Amazon Elastic Block Store (Amazon EBS)-backed AMI with user data
    3. An instance store-backed AMI with no user data
    4. An Amazon EBS-backed AMI with no user data
  6. Your boss has asked you to provide a report that shows your current monthly spending with AWS and an estimate of how much you will be spending this entire month. Which of these methods would be the best way to get those amounts?

    1. Use the AWS Total Cost of Ownership (TCO) Calculator to build a model of your current infrastructure, and use that to create an estimate.
    2. Use the My Billing Dashboard to see what your current spending is and also what your estimated total monthly spending would be.
    3. Depending on the level of support you have with AWS, open a trouble ticket, and ask to be provided with this information.
    4. Use the AWS Simple Monthly Calculator to build a model of your current infrastructure, and use that to create an estimate.
  7. What would be a good reason NOT to use Amazon Lightsail?

    1. You need to run a Windows instance.
    2. You want to spin up a compute instance quickly for some basic code configuration work.
    3. You want a high level of control over the cost of your compute instance.
    4. You have a website that does not need to be highly available.
  8. You have an application that needs to be available at all times. This application, once started, needs to run for about 35 minutes. You must also minimize the application’s cost. What would be a good strategy for achieving all of these objectives?

    1. Create an AWS Lambda function and have an Amazon Simple Notification Service (Amazon SNS) notification kick it off.
    2. Set up a dedicated instance for this application.
    3. Set up a dedicated host for this application.
    4. Create an Amazon Elastic Compute Cloud (Amazon EC2) Reserved instance.
  9. Which of the following would NOT be an appropriate event to trigger an AWS Lambda function?

    1. Performing a GET operation in an Amazon Simple Storage Service (Amazon S3) bucket
    2. Publishing a message to the Amazon Simple Notification Service (Amazon SNS) topic
    3. An entry in an Amazon DynamoDB table
    4. A message in an Amazon Simple Queue Service (Amazon SQS) queue
  10. Which of the following would be a good use case for AWS Elastic Beanstalk?

    1. You need to make your current Amazon Relational Database Service (Amazon RDS) setup highly available.
    2. You want to spin up a website to serve internal customers, but you don’t want to have to think about making it highly available and highly scalable.
    3. You need a compute instance to spin up in response to a specific network event.
    4. You are building a minimal website for test purposes.
  11. You have a relational database that is running on a Windows Amazon Elastic Compute Cloud (Amazon EC2) instance. What would be the best choice for instance type?

    1. An instance type that uses instance storage
    2. An instance type that is Amazon Elastic Block Store (Amazon EBS)-optimized
    3. An instance type with a general-purpose CPU
    4. An instance type that allows you to use Amazon Elastic File System (Amazon EFS)
  12. You have a relational database that is running on a Windows Amazon Elastic Compute Cloud (Amazon EC2) instance. What would be the best choice for storage?

    1. Instance store volumes
    2. Magnetic-based Amazon Elastic Block Store (Amazon EBS)
    3. Solid State Drive (SSD)-based Amazon EBS
    4. Amazon Elastic File System (Amazon EFS)
    5. Provisioned IOPS
  13. You have an instance store-backed Amazon Elastic Compute Cloud (Amazon EC2) instance with an interface that has a private IP address and a public IP address attached to it. You stop the instance. What happens to the IP addresses?

    1. Both the public and private IP addresses are removed from the interface.
    2. The public IP address is removed, but the private IP address remains associated with the interface.
    3. The public IP address remains associated with the interface, but the private IP address is removed.
    4. Both the public IP address and the private IP address remain associated with the interface.
    5. None of the above. You cannot stop instance store-backed Amazon EC2 instances.
  14. You have an Amazon Elastic Block Store (Amazon EBS) storage-backed Amazon Elastic Compute Cloud (Amazon EC2) instance with an interface that has a private IP address and a public IP address attached to them. You stop the instance. What happens to the IP addresses?

    1. Both the public and private IP addresses are removed from the interface.
    2. The public IP address is removed, but the private IP address remains associated with the interface.
    3. The public IP address remains associated with the interface, but the private IP address is removed.
    4. Both the public IP address and the private IP address remain associated with the interface.
    5. None of the above. You cannot stop instance storage-backed Amazon EC2 instances.
  15. You have an Amazon Elastic Compute Cloud (Amazon EC2) instance running in one AWS Region that you want to be able to run in another AWS Region. What do you need to do in order to accomplish that?

    1. You have to build the new Amazon EC2 instance from scratch because Amazon Machine Images (AMIs) are unique to an AWS Region.
    2. You can copy an AMI from one AWS Region to another but only if they are under the same AWS account.
    3. You cannot copy an AMI across AWS Regions because then you would have two AMIs with the same AMI ID.
    4. You can copy AMIs across AWS Regions using the CopyImage Application Programming Interface (API).
    5. You can copy AMIs across AWS Regions using the CopyImage API, but you need first to remove launch permissions and user defined tags.
  16. You need to spin up an Apache Web Server on an Amazon Elastic Compute Cloud (Amazon EC2) instance. What is the best way to do so?

    1. Spin up an Amazon EC2 instance, use Secure Shell (SSH) to access it after it has booted up, and configure it to be an Apache Web Server.
    2. In the metadata field, load the necessary software to spin up an Apache Web Server.
    3. In the user data field, load the necessary software to spin up an Apache Web Server.
    4. Go to AWS Marketplace and find an Apache Web Server Amazon Machine Image (AMI).
  17. You need an Amazon Elastic Compute Cloud (Amazon EC2) instance to be able to access an object in an Amazon Simple Storage Service (Amazon S3) bucket. What is the most secure way to do this?

    1. Make sure that the Amazon EC2 instance has the necessary user permission to be able to access the Amazon S3 bucket.
    2. Create an AWS Identity and Access Management (IAM) role. Make sure that role has the necessary level of permission to access the Amazon S3 bucket. Assign that role to the Amazon EC2 instance.
    3. Create an IAM role. Make sure that role has the necessary level of permission to access the Amazon S3 bucket. Assign that role to the Amazon S3 bucket.
    4. Make sure that the route table assigned to the Amazon EC2 instance has a route to the Amazon S3 bucket.
  18. Why would you use AWS Elastic Beanstalk to deploy a web application that uses multiple Availability Zones?

    1. You might run out of compute instances if you ran everything in a single Availability Zone.
    2. Using multiple Availability Zones improves the response time of your web application.
    3. Using multiple Availability Zones makes your application highly available.
    4. You can’t run AWS Elastic Beanstalk in a single Availability Zone.
  19. What are some of the aspects of Amazon Elastic Compute Cloud (Amazon EC2) that are controlled by your choice of instance type?

    1. Operating system
    2. User data
    3. Amazon Machine Image (AMI)
    4. CPU family availability
  20. You have just spun up an M4.2xlarge Amazon Elastic Compute Cloud (Amazon EC2) instance. What does the “4” stand for?

    1. Indicates the amount of RAM associated with the instance
    2. Indicates the generation of M class family instance
    3. Indicates the baseline number of CPUs associated with the instance
    4. Has no meaning at all and is only an AWS naming convention
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset