The Play by Play for IPsec
Let’s start the play-by-play discussion assuming that both routers have been correctly configured to be VPN peers and that they have default routes pointing to the Internet and that they were both just powered up. With a site-to-site VPN, as shown in our topology, each of the peers could also be called a VPN gateway, which is serving the customers on the 10.0.0.0/24 and 172.16.0.0/24 networks. The two routers will become IPsec peers with each other to form the IPsec tunnel over the Internet.
The first thing the router on the left (R1) is going to do, if it has been told to encrypt and protect traffic that is sourced from the 10.0.0.0 network and destined for the 172.16.0.0 network, is wait for that traffic to show up. Let’s say a user on the 10.0.0.0 network sends a packet to a server on the 172.16.0.0 network, and now R1 sees this packet and needs to encrypt it and protect it before sending it on its way. Unfortunately, the router has not yet established any VPN tunnels between itself and the router on the far right. So, if traffic did show up at R1 and it needed to be encrypted based on the policy, R1 would initiate negotiations with the router on the right. In this case, R1 would be the initiator of the VPN.
Step 1: Negotiate the IKEv1 Phase 1 Tunnel
What these two routers first negotiate is something called an Internet Key Exchange (IKE) Phase 1 tunnel. This can be done in one of two modes: main mode or aggressive mode. Main mode uses more packets for the process than aggressive mode, but main mode is considered more secure. Most current VPN implementations default to using main mode. This first tunnel (the IKE Phase 1 tunnel) is used between the two routers to speak directly to each other. This tunnel (once established) is not going to be used to forward user packets, but rather only to protect management traffic related to the VPN between the two routers. Packets such as a keepalive message to verify that the VPN tunnel is still working are an example of traffic that these two routers send across the IKE Phase 1 tunnel directly to each other.
Because the router on the left (R1) first received traffic that needed to be encrypted and there was no IKE Phase 1 tunnel in place, the router on the left becomes the initiator for the negotiations. The initiator sends over all of its configured/default parameters that it is willing to use for the IKE Phase 1 tunnel. Five basic items need to be agreed upon between the two VPN devices/gateways (in this case, the two routers) for the IKE Phase 1 tunnel to succeed, as follows:
Hash algorithm: This could be message digest 5 algorithm (MD5) or Secure Hash Algorithm (SHA) on most devices.
Encryption algorithm: This could be Digital Encryption Standard (DES) (bad idea, too weak), Triple DES (3DES) (better) or Advanced Encryption Standard (AES) (best) with various key lengths. (Longer is better for keys.)
Diffie-Hellman (DH) group to use: The DH “group” refers to the modulus size (length of the key) to use for the DH key exchange. Group 1 uses 768 bits, group 2 uses 1024, and group 5 uses 1536. More secure DH groups are part of the next-generation encryption (NGE):
Group 14 or 24: Provides 2048-bit DH
Groups 15 and 16: Support 3072-bit and 4096-bit DH
Group 19 or 20: Supports the 256-bit and 384-bit ECDH groups, respectively
The purpose of DH is to generate shared secret keying material (symmetric keys) that may be used by the two VPN peers for symmetrical algorithms, such as AES. It is important to note that the DH exchange itself is asymmetrical (and is CPU intensive), and the resulting keys that are generated are symmetrical.
Authentication method: Used for verifying the identity of the VPN peer on the other side of the tunnel. Options include a pre-shared key (PSK) used only for the authentication or RSA signatures (which leverage the public keys contained in digital certificates).
Lifetime: How long until this IKE Phase 1 tunnel should be torn down. (The default is one day, listed in seconds.) This is the only parameter that does not have to exactly match with the other peer to be accepted. If all other parameters match and the lifetime is different, they agree to use the smallest lifetime between the two peers. A shorter lifetime is considered more secure because it gives an attacker less time to calculate keys used for a current tunnel.
Step 2: Run the DH Key Exchange
Now having agreed to the IKE Phase 1 policy of the peer, the two devices run the DH key exchange. They use the DH group (DH key size for the exchange) they agreed to during the negotiations, and at the end of this key exchange, they both have symmetrical keying material (which is a fancy way of saying they both have the same secret keys that they can use with symmetrical algorithms).
DH, as you learned in a previous chapter, allows two devices that do not yet have a secure connection to establish shared secret keying material (keys that can be used with symmetrical algorithms, such as AES).
Step 3: Authenticate the Peer
The last piece of IKE Phase 1 is to validate or authenticate the peer on the other side. For authentication, they use whatever they agreed to in the initial HAGLE, and if they successfully authenticate with each other, we now have an IKE Phase 1 tunnel in place between the two VPN gateways. This tunnel is bidirectional, meaning that either device can send or receive on that IKE Phase 1 tunnel. The authentication could be done either using a PSK or using RSA digital signatures (depending on what they agreed to use in Step 1).
What About the User’s Original Packet?
Now here is the challenge: After all the work that went in to building the IKE Phase 1 tunnel, this tunnel is used only as a management tunnel so that the two routers can securely communicate with each other directly. This IKE Phase 1 tunnel is not used to encrypt or protect the end user’s packets. To protect the end user’s packets, (which is the entire goal for IPsec), the two VPN devices build a second tunnel for the sole purpose of encrypting the end-user packets. This second tunnel is called the IKE Phase 2 tunnel; it is also commonly referred to as (drum roll, please) the IPsec tunnel. This IKE Phase 2 tunnel is the tunnel used to protect the end-user packets as those packets cross untrusted networks between the VPN peers.
Leveraging What They Have Already Built
The two routers, with a beautiful IKE Phase 1 tunnel in place, can use that IKE Phase 1 tunnel to securely negotiate and establish the IPsec or IKE Phase 2 tunnel. In my years of working with students, this is where the confusion sometimes creeps in, because during the configuration the students say to themselves, “Didn’t I already specify the details for encryption and hashing? Why is it asking for them again in the configuration?” The answer is that we have to set up specific commands to specify the IKE Phase 1 policies, and we set up a different set of similar commands for the IKE Phase 2 policy (including the component called a transform set).
Immediately after the IKE Phase 1 tunnel is established (the two different modes to set up the IKE Phase 1 tunnel are main mode, which takes more packets, or aggressive mode, which takes fewer packets and is considered less secure), the routers immediately begin to establish the IKE Phase 2 tunnel.
The IKE Phase 1 tunnel is their management tunnel. The entire conversation and negotiation of the IKE Phase 2 tunnel are completely done in private because of the IKE Phase 1 tunnel protection the negotiated traffic. The IKE Phase 2 tunnel includes the hashing and encryption algorithms. The name of the mode for building the IKE Phase 2 tunnel is called Quick mode.
Now IPsec Can Protect the User’s Packets
Once the IKE Phase 2 tunnel is built, the routers can then begin to encrypt the user’s traffic and send those encrypted packets directly to the peer on the far side. From the Internet’s perspective, it looks like packets sourced from the IP address of R1 are being sent to the IP address of R2. The encrypted payload of these packets contains the original IP addresses and contents of the user who is forwarding a packet to a server or vice versa. If these packets are eavesdropped upon, the eavesdropper sees the IP addresses involved between the two routers; the payload (the original packets) has been encrypted and encapsulated inside, and is cipher text and unreadable to the individual who does not have the symmetric keys to decrypt the contents.
Traffic Before IPsec
If the packet is captured before IPsec is used, as it crosses the cloud (untrusted network), the eavesdropper sees the packet as it appears in Figure 6-2.
In this figure, any eavesdropper can see and determine the entire conversation between the client and the server. Because Telnet offers no encryption capabilities on its own, the at-tacker could learn the username and password used for initiation of this Telnet session and each command the Telnet user issued and its results.
Traffic After IPsec
After you configure R1 and R2 to become VPN peers/gateways, and tell them that all packets between the two networks of 10.0.0.0/24 and 172.16.0.0/24 should be protected by IPsec, R1 and R2 negotiate and build their VPN tunnels (IKE Phase 1 and IKE Phase 2), and then any traffic from either network and destined for the other is protected. Let’s consider the packet shown in the earlier figure. When R1 sees this same packet heading out to 172.16.0.4, and because its source IP address is on the 10.0.0.0/24 network, R1 uses the IKE Phase 2 tunnel and encrypts the packet and encapsulates the encrypted packet with a new IP header that shows the source IP address as R1 and the destination address as R2. The Layer 4 protocol would show as being Encapsulating Security Payload (ESP), which is reflected in the IP header as protocol #50. When R2 receives this, R2 de-encapsulates the packet, sees that it is ESP, and then proceeds to decrypt the original packet. Once decrypted, R2 forwards the plaintext packet to the server at 172.16.0.4. The encrypted packet, as it crosses over the untrusted network between R1 and R2, appears, as shown Figure 6-3.
