Chapter 5. Building Your Own Private Clouds Using Amazon VPC

So, in the previous chapter, we covered a lot of different things! We started off with some introductions and working examples of security groups and later on continued with understanding how EC2 networking really works, with a brief look at Elastic IP addresses. To top this off, we also learnt a lot about EBS volumes and their types and how we can create, attach, and manage them.

This chapter, however, is going to be a lot different and interesting as in this chapter, we will explore and learn about an awesome service provided by AWS called Virtual Private Cloud (VPC)! We will learn about the different VPC concepts and terminologies, deployment strategies, and a whole lot more, so stick around; we are just getting started!

An overview of Amazon VPC

So far we have learnt a lot about EC2, its features, and uses, and how we can deploy scalable and fault tolerant applications using it, but EC2 does come with its own sets of minor drawbacks. For starters, you do not control the IP addressing of your instances, apart from adding an Elastic IP address to your instance. By design, each of your instances will get a single private and public IP address, which is routable on the Internet—again, something you cannot control. Also, EC2 security groups have the capability to add rules for inbound traffic only; there is no support for providing any outbound traffic rules. So, although EC2 is good for hosting your applications, it is still not that secure. The answer to all your problems is Amazon VPC!

Amazon VPC is a logically isolated part of the AWS cloud that enables you to build and use your own logical subnets and networks. In a simpler sense, you get to build your own network topology and spin up instances within it. But what actually separates VPC from your classic EC2 environment is the ability to isolate and secure your environment. Using VPCs, you can choose which instances are accessible over the Internet and which are not. You can create a bunch of different subnets within a single VPC, each with their own security policies and routing rules. VPCs also provided an added layer of protection by enforcing something called as Network Access Control Lists (ACLs) besides your standard use of security groups. This ensures that you have total control over what traffic is routed in and out of your subnets and the VPC as well.

VPCs also provide an added functionality using which you can connect and extend your on-premise datacenters to the AWS cloud. This is achieved using an IPsec VPN tunnel that connects from your on premise datacenter's gateway device to the VPC's Virtual Private Gateway, as shown in the following image:

An overview of Amazon VPC

An important point to note here is that a VPC is still a part of the AWS Cloud. It is not physically separate hosting provided by AWS, it simply is a logically isolated part of the EC2 infrastructure. This isolation is done at the network layer and is very similar to a traditional datacenter's network isolation; it's just that we as end users are shielded from the complexities of it.

Note

To know more about VPN and virtual private gateways, refer to http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_VPN.html.

With this brief overview in mind, let's look at some of VPC's key concepts and terminologies to get a better understanding of how things work.

VPC concepts and terminologies

By now you must have understood that VPC is nothing more than a network service provided by AWS using which you can create logically isolated environments for your EC2 instances. And just like any other network service, VPC too works on some key concepts, explained as follows.

Subnets

Perhaps the most important part of the VPC, the subnets are nothing more than a range of valid IP addresses that you specify. VPC provides you with two different subnet creation options: a publically or Internet routed subnet called as a public subnet and an isolated subnet called as a private subnet. You can launch your instances in either of these subnets depending on whether you wish your instances to be routed on the Internet or not.

How does it all work? Pretty simple! When you first create a VPC, you provide it with a set of IP addresses in the form of a CIDR, for example, 10.0.0.0/16. The /16 here indicates that this particular VPC can support up to 65,536 IP addresses (2^(32-16) = 65,536, IP address range 10.0.0.0 - 10.0.255.255)! Now that's a lot! Once the VPC's CIDR block is created, you can go ahead and carve out individual subnets from it. For example, a subnet with the CIDR block 10.0.1.0/24 for hosting your web servers and another CIDR block 10.0.5.0/24 for your database servers and so on and so forth.

The idea here is that from the 65,536 IP address block, we carved out two subnets, each supporting 256 IPs (/24 CIDR includes 256 IP addresses in it). Now you can specify the subnet for the web servers to be public, as they will need to be routed to the Internet and the subnet for the database servers to be private as they need to be isolated from the outside world.

Tip

To know more about CIDRs and how they work, refer to https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing.

There is one additional thing worth mentioning here. By default, AWS will create a VPC for you in your particular region the first time you sign up for the service. This is called as the default VPC. The default VPC comes preconfigured with the following set of configurations:

  • The default VPC is always created with a CIDR block of /16, which means it supports 65,536 IP addresses in it.
  • A default subnet is created in each AZ of your selected region. Instances launched in these default subnets have both a public and a private IP address by default as well.
  • An Internet Gateway is provided to the default VPC for instances to have Internet connectivity.
  • A few necessary route tables, security groups, and ACLs are also created by default that enable the instance traffic to pass through to the Internet. Refer to the following figure:
    Subnets

You can use this default VPC just as any other VPC by creating additional subnets in it, provisioning route tables, security groups, and so on. In fact, the instances that we launched back in Chapter 3, Images and Instances, were based out of the default VPC!

Note

Any other VPC that you create besides the default VPC is called as the non-default VPC. Each non-default VPC in turn contains non-default subnets, and so on and so forth.

Security groups and network ACLs

We have talked a lot about security groups in the past two chapters already. We do know that security groups are nothing but simple firewall rules that you can configure to safeguard your instances. You can create a maximum of 100 security groups for a single VPC, with each Security Group containing up to 50 firewall rules in them. Also, it is very important to remember that a Security Group does not permit inbound traffic by default. You have to explicitly set inbound traffic rules to allow traffic to flow to your instance. However, all outbound traffic from the instance is allowed by default.

Network ACLs are something new. These provide an added security measure over security groups as they are instance specific, whereas Network ACLs are subnet specific. Unlike your security groups, however, you can both allow and restrict inbound and outbound traffic using ACL rules. Speaking of ACL rules, they are very much similar to your Security Group rules, however, with one small exception. Each ACL rule is evaluated by AWS based on a number. The number can be anything from 100 all the way up to 32,766. The ACL rules are evaluated in sequence starting from the smallest number and going all the way up to the maximum value. The following is a small example of how ACL rules look:

Inbound ACL rules

Rule No.

Source IP

Protocol

Port

Allow/Deny

100

0.0.0.0/0

All

All

ALLOW

*

0.0.0.0/0

All

All

DENY

Outbound ACL rules

Rule No.

Dest IP

Protocol

Port

Allow/Deny

100

0.0.0.0/0

all

all

ALLOW

*

0.0.0.0/0

all

all

DENY

These are the ACL rules created by AWS for your default VPC; as a result, this particular ACL is called as the default Network ACL as well. What do these rules mean? For starters, the rule number 100 for both the inbound and outbound ACL specifies the traffic to flow from any protocol running on any port in and out of the subnet. The * is also considered as a rule number and is a must in all ACLs. It basically means that you drop any packets that do not match the ACL's rules. We will be checking out ACLs and security groups in action later on in this chapter when we create our very own VPC for the first time.

Routing tables

Route tables are pretty straightforward and easy to implement in a VPC. They are nothing but simple rules or routes that are used to direct network traffic from a subnet. Each subnet in a VPC has to be associated with a single route table at any given time; however, you can attach multiple subnets to a single route table as well.

Remember the default VPC and the default subnets? Well, a similar default route table is also created when you first start using your VPC. This default route table is called as the main route table and it generally contains only one route information that enables traffic to flow within the VPC itself. Subnets that are not assigned to any route tables are automatically allocated to the main route table. You can edit and add multiple routes in the main route table as you see fit; however, you cannot modify the local route rule. The following an example of a main route table viewed from the VPC Management dashboard:

Routing tables

As you can see, there are a couple of entries made in this table. The first is the local route rule that allows traffic to flow within this particular subnet (10.0.0.0/16). The second route is something called as a route for VPC endpoints. This is a private connection made between your VPC and some other AWS service; in this case, the service is S3. Let's look VPC endpoints a little closely.

VPC endpoints

VPC endpoints basically allow you to securely connect your VPC with other AWS services. These are virtual devices that are highly available and fault tolerant by design. They are scaled and managed by AWS itself, so you don't have to worry about the intricacies of maintaining them. All you need to do is create a VPC endpoint connection between your VPC and an AWS service of your choice, and voila! Your instances can now communicate securely with other AWS services. The instances in the VPC communicate with other services using their private IP addresses itself, so there's no need to route the traffic over the Internet.

Note

AWS currently only supports VPC endpoint connections for Amazon S3. More services are planned to be added shortly.

When an endpoint is created, you first need to select either of your VPC's route tables. The traffic between your VPC instances and the AWS service will be routed using this particular route table. Similar to any other route information, a VPC endpoint route also contains a Destination field and a Target field. The Destination filed contains the AWS service's prefix list ID, which is generally represented by the following ID: pl-xxxxxxxx. The Target field contains the endpoint ID, which is represented in the following format: vpce-xxxxxxxx.

In the following route table example, the prefix list ID (pl-68a54001) represents the S3 service whereas the target vpce-80cd2be9 represents the endpoint ID:

VPC Endpoint Route

Destination

Target

10.0.0.0/16

Local

pl-68a54001

vpce-80cd2be9

Endpoints also provided an additional feature using which you can control and govern access to the remote AWS service. This is achieved using something called as endpoint policies.

Endpoint policies are nothing more than simple IAM-based resource policies that are provided to you when an endpoint is first created. AWS creates a simple policy document that allows full access to the AWS service by default. The following is a sample endpoint policy that is created by AWS for the S3 service:

{
  "Statement": [
    		{
"Action": "*",
"Effect": "Allow",
"Resource": "*",     	
"Principal": "*"
}
  ]
}  

To know more about the endpoint policies and features, refer to http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-endpoints.html.

Internet Gateways

Internet Gateways, as the name suggest, are primarily used to provide Internet connectivity to your VPC instances. All you have to do is create and attach an Internet Gateway device to your VPC and add a route entry in your public subnet's route table to point to the Internet Gateway! That's it! The default VPC comes with an Internet gateway already deployed in it. So, any instance that you launch from the default subnet obtains Internet connectivity automatically. This does not apply for non-default VPCs, however, as an instance launched in a non-default subnet does not receive a public IP address by default. You would have to either assign one to the instance during the launch phase or modify your non-default subnet's public IP address attributes.

Once you have created and attached an Internet Gateway to your VPC, you will also have to make sure that the public subnet's route table has an entry for this gateway.

Plus, you will also have to create the correct set of security groups and network ACL rules to allow your subnet's traffic to flow through the Internet. The following is an example of a VPC's route table showing the route for a subnet's traffic to the Internet Gateway (igw-8c3066e9):

Internet Gateways

Besides the Internet connectivity, Internet Gateways also perform NAT on the instance's private IPs. The instances in a subnet are only aware of their private IP addresses that they use to communicate internally. The Internet Gateway maps the instance's private IP with an associated public or Elastic IP and then routes traffic outside the subnet to the Internet. Conversely, the Internet Gateway also maps inbound traffic from the Internet to a public or Elastic IP and then translates it to the instance's private IP address. This is how your instances receive Internet from within a VPC, which brings us to yet another interesting topic called as NAT instances.

NAT instances

So, we have just learnt that the Internet Gateway NATs the IP addresses of instances placed out in the public subnet so that they can communicate with the Internet, but what about instances in the private subnets? How do they communicate with the Internet without having direct Internet connectivity via the gateway?

That's where a NAT instance comes into play. A NAT Instance is a special instance created inside your public subnet that NATs outbound traffic from instances based in your private subnet to the Internet. It is important to note here that the NAT instance will only forward the outbound traffic and not allow any traffic from the Internet to reach the private subnets, similar to a one way street.

You can create a NAT Instance out of any AMI you wish; however, AWS provides few standard Linux-based AMIs that are well suited for such purposes. These special AMIs are listed out in the community AMIs page and all you need to do is filter out the amzn-ami-vpc-nat AMI from the list and spin up an instance from it.

The following example depicts the traffic flow from a private subnet (10.0.1.0/24) to the NAT instance inside a public subnet (10.0.0.0/24) via a route table:

NAT instances

In the preceding example, outbound traffic from the public subnet's route table is routed to the Internet Gateway (igw-8c3066e9) while the outbound traffic from the private subnet's route table is routed to the NAT instance. Along with the route tables, it is also essential that you correctly populate the Security Group for your NAT instance. The following is a simple NAT instance Security Group example for your reference:

NAT instance - inbound security Rules

Source

Protocol

Port

Remarks

10.0.1.0/24

TCP

80

Permit inbound HTTP traffic from private subnet

10.0.1.0/24

TCP

443

Permit inbound HTTPS traffic from private subnet

<HOSTIP>*

TCP

22

Permit SSH login to NAT instance from remote N/W

Note

The * replace the <HOSTIP> field with the IP address of your local desktop machine.

The following are the outbound security rules:

NAT Instance - outbound security rules

Source

Protocol

Port

Remarks

0.0.0.0/0

TCP

80

Permit HTTP access to Internet for the NAT instance

0.0.0.0/0

TCP

443

Permit HTTPS access to Internet for the NAT instance

DNS and DHCP Option Sets

VPCs provide an additional feature called as DHCP Option Sets using which you can set and customize the DNS and DHCP for your instances. The default VPC comes with a default DHCP Options Set that is used to provide the instances with a dynamic private IP address and a resolvable hostname. Using the DHCP Options Set, you can configure the following attributes for your VPC:

  • Domain Name Servers (DNS): You can list down up to four DNS servers here of your own choice or even provide the Amazon DNS server details. The Amazon DNS server is provided in your VPC and runs on a reserved IP address. For example, if your VPC has the subnet of 10.0.0.0/16, then the Amazon DNS Server will probably run on the IP 10.0.0.2. You can additionally provide the Amazon DNS Server's IP address, 169.254.169.253, or the value AmazonProvidedDNS as required. Values entered here are automatically added to your Linux instances /etc/resolv.conf file for name resolution.

Domain name: You can either provide your own domain name value or choose to use the default AWS domain name values using this option. The default AWS domain names can be provided only if you have selected AmazonProvidedDNS as your DNS server. For example, instances launched in the US West region with the Amazon DNS server value will get a resolvable private DNS hostname as us-west-2.compute.internal.

  • NTP servers: You can list up to four NTP server IP addresses using the DHCP Options Set wizard. Note, however, that this will only accept IP address values and not FQDNs such as pool.ntp.org.
  • NetBIOS name server: You can list down up to four NetBIOS name servers as well; however, this field is optional.
  • NetBIOS node type: You can specify the NetBIOS node value, which can either be 1, 2, 4, or 8. AWS recommends that you specify 2 as broadcast, and multicasts are not currently supported.

You can create and attach only one DHCP option set with a VPC at a time. AWS uses the default DHCP option if you do not specify one explicitly for your VPC. Instances either running or newly launched will automatically pick up these DNS and DHCP settings, so there is no need for you to restart or relaunch your existing instances.

VPC limits and costs

Okay, so far we have understood a lot about how the VPC works and what its components are, but what is the cost of all this? Very simple, it's nothing! VPC is a completely free of cost service provided by AWS; however, you do have to pay for the EC2 resources that you use, for example, the instances, the Elastic IP addresses, EBS volumes, and so on. Also, if you are using your VPC to connect to your on premise datacenter using the VPN option, then you need to pay for the data transfers over the VPN connection as well as for the VPN connection itself. AWS charges $0.05 per VPN connection hour.

Besides this, VPCs also have a few limits set on them by default. For example, you can have a maximum of five VPCs per region. Each VPC can have a max of one Internet gateway as well as one virtual private gateway. Also, each VPC has a limit of hosting a maximum of up to 200 subnets per VPC. You can increase these limit by simply requesting AWS to do so. To view the complete list of VC limits, refer to http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Appendix_Limits.html.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset