Understanding EBS volumes

We briefly touched base on EBS volumes back in the previous chapter where we were comparing EBS-backed and instance store-backed images. In this section, you are going to learn a bit more about EBS volumes, their features, benefits, different types, along with steps on how to create, attach, and delete them as well. So, what are we waiting for? Let's get started!

First up, let's understand EBS volumes a bit better. EBS volumes are nothing more than block-level storage devices that you can attach to your EC2 instances. They are highly durable and can provide a host of additional functionalities to your instances, such as data persistence, encryption, snapshotting capabilities, and so on. Majority of the time, these EBS volumes are used for storing data for a variety of applications and databases, however you can use it just as a normal hard drive as well. The best part of EBS volumes is that they can persist independently from your instances. So powering down an instance or even terminating it will not affect the state of your EBS volumes. Your data will stay on it unless and until you explicitly delete it.

Let's look at some of the key features and benefits that EBS volumes have to offer:

  • High availability: Unlike your instance store-backed drives, EBS volumes are automatically replicated by AWS within the availability zone in which they are created. You can create an EBS volume and attach it to any instance present in the same availability zone; however, one EBS volume cannot be attached to multiple instances at the same time. A single instance, however, can have multiple EBS volumes attached to it at any given time.
  • Encryption capabilities: EBS volumes provide an add-on feature using which you can encrypt your volumes using standard encryption algorithms, such as AES-256, and keys as well. These keys are autogenerated the first time you employ encryption on a volume using the AWS Key Management Service (KMS). You can additionally even use IAM to provide fine-grained access control and permissions to your EBS volumes.
  • Snapshot capabilities: The state of an EBS volume can be saved using point-in-time snapshots. These snapshots are all stored incrementally on your Amazon S3 account and can be used for a variety of purposes, such as creating new volumes based on an existing one, resizing volumes, backup and data recovery, and so on.

Tip

EBS volumes cannot be copied from one AWS region to another. In such cases, you can take a snapshot of the volume and copy the snapshot over to a different region using the steps mentioned at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-copy-snapshot.html.

EBS volume types

There are three different types of EBS volumes available today, each with their own sets of performance characteristics and associated costs. Let's briefly look into each one of them and their potential uses:

  • General purpose volumes (SSD): These are by far the most commonly used EBS volume types as they provide a good balance between cost and overall performance. By default, this volume provides a standard 3 IOPS per GB of storage, so a 10 GB general purpose volume will get approximately 30 IOPS and so on so forth, with a max value of 10,000 IOPS. You can create general purpose volumes that can range in size from 1 GB to a maximum of 16 TB. Such volumes can be used for a variety of purposes, such as instance root volumes, data disks for dev and test environments, database storage, and so on.
  • Provisioned IOPS volumes (SSD): These are a specialized set of SSDs that can consistently provide a minimum of 100 IOPS burstable up to 20,000 IOPS. You can create Provisioned IOPS Volumes that range in size from a minimum of 4 GB all the way up to 16 TB. Such volumes are ideally suited for applications that are IO intensive, such as databases, parallel computing workloads such as Hadoop, and so on.
  • Magnetic volumes: Very similar to traditional tape drives and magnetic disks, these volumes are a good match for workloads where data is accessed infrequently, such as log storage, data backup and recovery, and so on. On an average, these volumes provide up to a 100 IOPS with an ability to burst up to 1,000 IOPS. You can create Magnetic volumes that range in size from a minimum of 1 GB all the way up to 1 TB.

Getting started with EBS Volumes

Now that we have a fair idea of what an EBS Volume is, let's look at some simple ways that you can create, attach, and manage these volumes.

To view and access your account's EBS Volumes using AWS Management Console, simply select the Volumes option from the EC2 dashboard's navigation pane, as shown here:

Getting started with EBS Volumes

This will bring up the Volume Management dashboard as shown here. In my case, I already have a volume present here that is shown as in use. This is our first instance's root device volume that we launched in Chapter 3, Images and Instances. Each EBS-backed instance's volume will appear here in the Volume Management dashboard. You can use this same dashboard to perform a host of activities on your volumes, such as create, attach, detach, and monitor performance, to name a few.

Getting started with EBS Volumes

You can view any particular EBS Volume's details by simply selecting it and viewing its related information in the Description tab, as shown. Here, you can view the volume's ID, Size, Created date, the volume's current State as well as its Attachment information, which displays the volume's mount point on a particular instance. Additionally, you can also view the volume's health and status by selecting the Monitoring and Status Checks tab, respectively. For now, let's go ahead and create a new volume using the volume management dashboard.

Creating EBS volumes

From the Volume Management dashboard, select the Create Volume option. This will pop up the Create Volume dialog box as shown here:

Creating EBS volumes

Fill in the details as required in the Create Volume dialog box. For this tutorial, I went ahead and created a simple 10-GB general purpose volume:

  • Type: From the Type drop-down list, select either General Purpose (SSD), Provisioned IOPS (SSD), or Magnetic as per your requirements.
  • Size (GiB): Provide the size of your volume in GB. Here, I provided 10 GB.
  • IOPS: This field will only be editable if you have selected Provisioned IOPS (SSD) as the volume's type. Enter the max IOPS value as per your requirements.
  • Availability Zone: Select the appropriate availability zone in which you wish to create the volume. Remember, an EBS volume can span availability zones, but not regions.
  • Snapshot ID: This is an optional field. You can choose to populate your EBS volume based on a third party's snapshot ID. In this case, we have left this field blank.
  • Encryption: As mentioned earlier, you can choose whether or not you wish to encrypt your EBS Volume. Select Encrypt this volume checkbox if you wish to do so.
  • Master Key: On selecting the Encryption option, AWS will automatically create a default key pair for the AWS's KMS. You can make a note of the KMS Key ID as well as the KMS Key ARN as these values will be required during the volume's decryption process as well.

Once your configuration settings are filled in, select Create to complete the volume's creation process. The new volume will take a few minutes to be available for use. Once the volume is created, you can now go ahead and attach this volume to your running instance.

Attaching EBS volumes

Once the EBS volume is created, make sure it is in the available state before you go ahead and attach it to an instance. You can attach multiple volumes to a single instance at a time, with each volume having a unique device name. Some of these device names are reserved, for example, /dev/sda1 is reserved for the root device volume. You can find the complete list of potential and recommended device names at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/device_naming.html.

The following screenshot shows the option of attaching a volume:

Attaching EBS volumes

To attach a volume, select the volume from the Volume Management dashboard. Then select the Actions tab and click on the Attach Volume option. This will pop up the Attach Volume dialog box, as shown:

Attaching EBS volumes

Type in your instance's ID in the Instance field and provide a suitable name in the Device field as shown. In this case, I provided the recommended device name of /dev/sdf to this volume. Click on Attach once done. The volume attachment process takes a few minutes to complete. Once done, you are now ready to make the volume accessible from your instance.

Accessing volumes from an instance

Once the volume is attached to an instance, you can basically format it and use it like any other block device. In this case, I'll be using the same Amazon Linux instance that we created back in Chapter 3, Images and Instances. You can attach and mount volumes to Windows-based instances as well.

To get started, first up connect to your running instance using putty or any other SSH client of your choice. Next, type in the following command to check the current disk partitioning of your instance:

# sudo df -h

You should see a /dev/xvda1 like filesystem mounted on the root (/) partition along with few other temp filesystems, as shown here. This is more or less the standard disk partitioning scheme that your instances will have.

Accessing volumes from an instance

Next, run the following command to list out partitions on your current instance. You should see a default /dev/xvda partition along with its partition table and an unformatted disk partition with the name /dev/xvdf as shown in the following screenshot. The /dev/xvdf command is the newly added EBS volume that we will need to format in the upcoming steps:

# sudo fdisk -l
Accessing volumes from an instance

Once you have verified the name of your newly added disk, you can go ahead and format with a filesystem of your choice. In this case, I have gone ahead and used the ext4 filesystem for my new volume:

# sudo mkfs -t ext4 /dev/xvdf

Now that your volume is formatted, you can create a new directory on your Linux instance and mount the volume to it using your standard Linux commands:

# sudo mkdir /my-new-dir
# sudo mount /dev/xvdf /my-new-dir

Here is the screenshot of creating new directory using the preceding commands:

Accessing volumes from an instance

Here's a useful tip! Once you have mounted your new volume, you can optionally edit the Linux instance's fstab file and add the volume's mount information there. This will make sure that the volume is mounted and available even if the instance is rebooted. Make sure you take a backup copy of the fstab file before you edit it, just as a precautionary measure.

Detaching EBS volumes

Detaching EBS volumes is a fairly simple and straightforward process. You will first need to unmount the volume from your instance and then simply detach it using Volume Management dashboard.

Run the following command to unmount the EBS volume from the instance:

# sudo umount /dev/sdf

Note

Make sure you are unmounting the correct volume from the instance. Do not try and unmount the /dev/sda or any other root partitions.

Once the volume is successfully unmounted from the instance, detach the volume by selecting the Detach Volume option from the Actions tab, as shown here:

Detaching EBS volumes

Managing EBS volumes using the AWS CLI

You can create, attach, and manage EBS volumes using the AWS CLI as well. Let's go ahead and create a new EBS volume using the AWS CLI. Type in the following command:

# aws ec2 create-volume 
--size 5 --region us-west-2 --availability-zone us-west-2a 
--volume-type gp2

Note

The --volume-type attribute accepts any one of these three values:

  • gp2: General Purpose instances (SSD)
  • io1: Provisioned IOPS volumes (SSD)
  • standard: Magnetic volumes

The following code will create a 5 GB General Purpose volume in the us-west-2a availability zone.

The new volume will take a couple of minutes to be created. You should see a similar output as the following screenshot. Make a note of the new volume's Volume ID before proceeding to the next steps.

Managing EBS volumes using the AWS CLI

Now that the new volume is created, we can go ahead and attach it to our instance. Type in the following command:

# aws ec2 attach-volume 
--volume-id vol-40993355 
--instance-id i-53fc559a 
--device /dev/sdg 

The following command will attach the volume with the volume ID vol-40993355 to our instance (i-53fc559a), and the device name will be /dev/sdg. Once again, you can supply any meaningful device name here, but make sure that it abides by AWS's naming conventions and best practices as mentioned in http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/device_naming.html.

Once the volume is attached to your instance, the next steps are pretty easy and straightforward. First format the new volume with a suitable filesystem of your choice. Next up, create a new directory inside your instance and mount the newly formatted volume on it. Voila! Your volume is now ready for use.

You can detach and delete the volume as well using the AWS CLI. First up, we will need to unmount the volume from the instance. To do so, type in the following command in your instance:

# unmount /dev/sdg

Make sure you are unmounting the correct volume and not the root partition. Once the volume is unmounted, simply detach it from the instance using the following AWS CLI code:

# aws ec2 detach-volume 
--volume-id vol-40993355

The output of the preceding command is as follows:

Managing EBS volumes using the AWS CLI

Finally, go ahead and delete the volume using the following AWS CLI code:

# aws ec2 delete-volume 
--volume-id vol-40993355

Remember that you cannot delete volumes if they are attached or in use by an instance, so make sure that you follow the detachment process before deleting it.

Backing up volumes using EBS snapshots

We do know for a fact that AWS automatically replicates EBS volumes so that your data is preserved even in case the complete drive fails. But this replication is limited only to the availability zone in which the drive or EBS volume was created, which means if that particular availability zone was to go down for some reason, then there is no way for you to back up your data. Fortunately for us, AWS provides a very simple yet highly efficient method of backing EBS volumes, called as EBS snapshots.

An EBS snapshot in simple terms is a state of your volume at a particular point in time. You can take a snapshot of a volume anytime you want. Each snapshot that you take is stored incrementally in Amazon S3, but, you will not be able to see these snapshots in your S3 buckets; they are kind of hidden away and stored separately.

You can achieve a wide variety of tasks using snapshots. A few are listed as follows:

  • Create new volumes based on existing ones: Snapshots are a great and easy way to spin up new volumes. A new volume spawned from a snapshot is an exact replica of the original volume, down to the last detail.
  • Expand existing volumes: Snapshots can also be used to expand an existing EBS Volume's size as well. It is a multistep process, which involves you taking a snapshot of your existing EBS volume and creating a larger new volume from the snapshot.
  • Share your volumes: Snapshots can be shared within your own account (private) as well publicly.
  • Backup and disaster recovery: Snapshots are a handy tool when it comes to backing up your volumes. You can create multiple replicates of an existing volume within an AZ, across AZs that belong to a particular region, as well as across regions, using something called an EBS Snapshot copy mechanism.

To create a snapshot of your volumes, all you need to do is select the particular volume from the Volume Management dashboard. Click on the Actions tab and select the Create Snapshot option, as shown here:

Backing up volumes using EBS snapshots

Tip

It is really a good practice to stop your instance before taking a snapshot if you are taking a snapshot of its root volume. This ensures a consistent and complete snapshot of your volume at all times.

You should see the Create Snapshot dialog box as shown in the following screenshot. Provide a suitable Name and Description for your new snapshot. An important thing to note here is that this particular snapshot is not supporting encryption, but why? Well, that's simple! Because the original volume was not encrypted, neither will the snapshot be encrypted. Snapshots of encrypted volumes are automatically encrypted. Even new volumes created from an encrypted snapshot are encrypted automatically. Once you have finished providing the details, click on Create to complete the snapshot process:

Backing up volumes using EBS snapshots

You will be shown a confirmation box, which will display this particular snapshot's ID. Make a note of this ID for future reference.

The new snapshot will take a good 3–4 minutes to go from Pending to Completed. You can check the status of your snapshot by viewing the Status as well as the Progress fields in the Description tab, as shown here:

Backing up volumes using EBS snapshots

Once the snapshot process is completed, you can use this particular snapshot and Create Volume, Copy this snapshot from one region to another, and Modify Snapshot Permissions to private or public as you see fit. These options are all present in the Actions tab of your Snapshot Management dashboard:

Backing up volumes using EBS snapshots

But for now, let's go ahead and use this snapshot to create our very first AMI. Yes, you can use snapshots to create AMIs as well. From the Actions tab, select the Create Image option. You should see the Create Image from EBS Snapshot wizard as shown here. Fill in the required details and click on Create to create your very first AMI:

Backing up volumes using EBS snapshots

The details contain the following options:

  • Name: Provide a suitable and meaningful name for your AMI.
  • Description: Provide a suitable description for your new AMI.
  • Architecture: You can either choose between i386 (32 bit) or x86_64 (64 bit).
  • Root device name: Enter a suitable name for your root device volume. Ideally, a root device volume should be labelled as /dev/sda1 as per EC2's device naming best practices.
  • Virtualization type: You can choose whether the instances launched from this particular AMI will support Paravirtualization (PV) or Hardware Virtual Machine (HVM) virtualization.

    Tip

    You can read more about the various Virtualization types supported by EC2 at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/virtualization_types.html.

  • RAM disk ID, Kernel ID: You can select and provide your AMI with its own RAM disk ID (ARI) and Kernel ID (AKI); however, in this case I have opted to keep the default ones.
  • Block Device Mappings: You can use this dialog to either expand your root volume's size or add additional volumes to it. You can change the Volume Type from General Purpose (SSD) to Provisioned IOPS (SSD) or Magnetic as per your AMI's requirements. For now, I have left these to their default values.

Click on Create to complete the AMI creation process. The new AMI will take a few minutes to spin up. In the meantime, you can make a note of the new AMI ID from the Create Image from EBS Snapshot confirmation box, as shown:

Backing up volumes using EBS snapshots

You can view your newly created AMI under the AMIs option from the EC2 dashboard's navigation pane:

Backing up volumes using EBS snapshots

So, here we have it! You very own AMI, created and ready to use.

An important point to note here is that you will not be able to delete this particular EBS Snapshot now as it is in use by your AMI. You will have to deregister your AMI first from the AMI Management dashboard and then try and delete the snapshot.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset