Internet Connectivity

Internet connectivity is the most basic method of connecting to Microsoft Azure. As shown in Figure 9-2, all the Azure PaaS services utilize public IP addresses, which means that the DNS for the workload and the connectivity to it is available to anyone on the Internet. This also means that all Azure PaaS services can be consumed over the Internet. Azure IaaS services need to be NAT’d (network address translation) from the private IP address the workload has on the Azure virtual network to a public IP address on Microsoft Azure’s “Edge,” or external network, connected to the Internet.

Internet-based workloads in Azure work well for many use cases. When you don’t need the workload to traverse a private or internal network, if the target user population of the workload is coming from the Internet, if the user population is a different organization, making the workload Internet-facing can have many advantages, including security. If you are making a workload external facing to “untrusted” systems and users, meaning they aren’t managed by your organization, not coupling the Azure environment to your on-premises environment provides a security boundary. If a bad actor or malware were able to breach your workload, then the workload is all they would have access to, not on-premises.

Another advantage of Internet connectivity is the bandwidth available to the workload. This bandwidth is part of Microsoft’s global network infrastructure, allowing high-speed, low-latency communication. Lower-end VMs typically see over 1Gbps down, over 500Mbps up, and extremely low latencies, as shown in Figure 9-3. This takes the burden off an on-premises Internet connection and reduces your on-premises exposure. When exposing workloads directly to the Internet, you get the full benefit of Microsoft’s cybersecurity infrastructure, but you can still deploy your own security solutions if you want.

Azure VPN

VPN connectivity is frequently used when dedicated private circuits don’t exist between locations that want to share IT services, such as Branch Offices or datacenters. VPN tunnels can either be initiated from the individual user’s device from anywhere in the world, such as a coffee shop or house, known as Point-to-Site (P2S) VPN, or from an on-premises location to your on-premises datacenter. This is commonly referred to as a site-to-site (S2S) VPN tunnel, frequently used for “branch office” or low-cost office to office network connectivity. The key difference between P2S and S2S VPN tunnels is P2S VPN tunnels connect one resource to a remote datacenter or network, whereas S2S VPN tunnels connect many systems and users to a remote datacenter or network.

P2S and S2S VPN connections are designed to run over the Internet but can run on top of a private network connection. Both use the industry-standard protocols Internet Protocol Security (IPsec) and Internet Key Exchange (IKE). These protocols ensure the security of the connection. If the connection is broken or suspected of being tampered with, the connection drops as a security precaution. The connection then needs to be reestablished to continue communication to the remote endpoints or services. A P2S VPN connection usually requires the user to reinitiate the connection. You are likely familiar with this from having experienced this behavior on unreliable networks at hotels or cafés or having to use your smartphone as an Internet hotspot. S2S VPN connections usually have their tunnel reestablished automatically as soon as the first packets attempt to traverse the tunnel.

Azure VPN is provided by a service called a VPN gateway , which encrypts and sends all traffic to the on-premises location over the Internet, over Microsoft’s ExpressRoute network connection, or between VNets in Azure (VNet-to-VNet). Azure virtual networks can only have one VPN gateway, just like a subnet can only have one default gateway address. Multiple networks can connect to a single Azure VPN gateway, thus allowing an administrator to create a virtual datacenter in the cloud and connect all their on-premises locations to it. This is the classic hub-and-spoke architecture, with the Azure virtual network as the hub. Figure 9-4 illustrates how different VPN connections can be used to join several different external resources to several cloud services.

A virtual network gateway is made up of VMs that are deployed to a specific subnet you create called the gateway subnet . The deployment of these VMs is the reason why standing up a VPN connection to Azure can take up to 45 minutes. The VMs provisioned when you create the virtual network gateway contain routing tables and run specific gateway services, which cannot be configured by administrators. Like all Azure services, if high availability is desired, administrators can deploy VPN gateways to Azure Availability Zones. This creates multiple gateways in an Azure region, which are physically and logically separated.

Azure VPN gateways come in several sizes, with varying performance metrics such as the number of concurrent connections, the connection speeds, and whether Zone Redundancy is an option for high availability. Azure VPN gateways range in speed from 100 Mbps to 10 Gbps. When Azure VPN gateways are used from on-premise to Azure, regardless of whether it is P2S or S2S, the Internet carrier is primarily responsible for the network latency and throughput; the ISP is usually the bottleneck.

In addition to the ISP being the bottleneck, assuming you’ve chosen a fast Azure VPN gateway, the ISP’s Internet connection has no performance service-level agreement (SLA). Because the Internet has so many parties that could cause problems on an ISP’s network (and because they’re outside the ISP’s control), the negative performance and lack of an SLA is a major reason why VPN-over-Internet is not used when private dedicated circuits like ExpressRoute are available. For a detailed list of all Azure VPN gateways and architectural illustrations, please refer to https://docs.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-about-vpngateways.

ExpressRoute

Azure ExpressRoute is a private, dedicated, high-bandwidth, low-latency, SLA’d connection into the Microsoft Azure global network. ExpressRoute allows connectivity not only to Azure resources, but also Office 365 and Dynamics 365 at speeds from 50 Mbps to 100 Gbps. ExpressRoute uses private peering for connectivity to an Azure virtual network, and Microsoft peering for connectivity to PaaS and SaaS workloads, such as Office 365 and Dynamics 365. Peering is a connection between two different networks over the Internet for the sake of exchanging data between them. Peering allows customers to have more control over the routing, performance, capacity, and redundancy. Therefore, the peering process is key to Azure services’ connectivity when SLA high bandwidth or low latency is desired.

Each of the ExpressRoute routers can have ExpressRoute Private and Microsoft peering enabled for the various, IaaS, PaaS, and SaaS workloads, as shown in Figure 9-5. These IP ranges listed previously are added to the routers on the customer side of the connections illustrated in Figure 9-5. These routers are set up in an Active/Passive architecture. If they are utilized in Active/Active, the SLA is not supported.

In addition, ExpressRoute supports VPN failover for a redundant network connection. This allows customers to failover from ExpressRoute to a redundant VPN, with the understanding that the SLA on VPN is on the VPN gateway, not on the bandwidth.

In Figure 9-5, the ExpressRoute circuit connects the partner’s Edge network to Microsoft’s Edge network. This is an important point for clarification addressed earlier in the book. ExpressRoute connects one network to another; it does not connect a customer’s enterprise network to an Azure “region.” Route filters applied to the ExpressRoute connection determine what traffic can flow to and from Azure. Now that you understand the logical connections and routing domains to Microsoft Azure over ExpressRoute, we’ll review how that works physically, in conjunction with the Internet and firewalls. Figure 9-6 depicts how logical connections work in parallel with the Internet.

In Figure 9-6, Microsoft peering from on-premises connects on the outside of the Internet-facing firewall, using a public IP address to peer with Microsoft’s PaaS and SaaS services, which also use public IPs because they are Internet-facing. The private peering uses private IPs, as the name depicts, and comes from inside the environment to the IaaS VNet(s). Each of these peering types can be run in parallel, with multiple instances of each, or by themselves.

ExpressRoute locations are frequently referred to as peering locations, meet-me locations, or points-of-presence (PoP). These locations are managed by several Microsoft ExpressRoute partners, known as connectivity providers . AT&T, CenturyLink, Orange, and Tata Communications are a few of the connectivity providers.

At the time this chapter was written, there were 108 connectivity providers around the globe. Connectivity providers are usually ISPs, but many ISPs are not ExpressRoute connectivity providers. Each connectivity provider has specific locations that they make the Azure ExpressRoute service available, which is a subset of their actual public footprint. A detailed list of Azure ExpressRoute partners and peering locations is at https://docs.microsoft.com/en-us/azure/expressroute/expressroute-locations-providers. In Figure 9-7, Azure’s public and enterprise ExpressRoute locations are different from the ones in the four Azure Government national clouds.

When you use an ISP not listed as a connectivity provider on the “Azure ExpressRoute partners and peering locations” page listed previously, customers can still connect to Azure through exchange providers. Exchange providers provide ExpressRoute at layer 2, or the data link layer, on the OSI model. We’ll review Azure layer 2 ExpressRoute connections later in this chapter.

If ExpressRoute connectivity providers or exchanges aren’t available in your geography, you may also connect to Azure through satellite operators. This should only be considered for Azure connectivity if lower latency options are not available. Satellite connectivity for ExpressRoute makes Azure one hop away but with a latency subject to the many hundreds of miles each way to communicate with the satellite. ExpressRoute over satellite may be the only option in locations like at sea or very remote locations of the globe.

ExpressRoute ports can be ordered in both a metered plan and an unlimited data plan. Both plans’ costs include high-availability dual ports and are billed by the port speed monthly. ExpressRoute charges cited online and through billing reflect two ports on two routers. Both ExpressRoute data plans included unlimited ingress traffic, but only the unlimited data plan includes unlimited outbound traffic. The cost of egress traffic on the Metered Data plan varies by location. For example, North America is $0.025 per GB. A detailed breakout of ExpressRoute pricing is at https://azure.microsoft.com/en-us/pricing/details/expressroute/. ExpressRoute ports can be upgraded or downgraded at any time, but only upgrading the circuit can be done with no service interruption at this time.

Layer 2 ExpressRoute

Layer 2 ExpressRoute connections terminate a private circuit in a cloud exchange, where a cross-connect is established, usually between a customer’s cage and the cloud exchange’s cage for Microsoft Azure. Exchange providers (also referred to as network service providers, fiber hotels, Internet exchange points, or peering points) are where customers can bring in a circuit to their own infrastructure to exchange traffic with another network. The exchange providers’ facilities are like colocations where circuits coming in from a customer, come into a cage for security reasons. The exchange provider runs cables from your cage to the cage where the Microsoft Azure global network is. This process of connecting through an exchange provider can drastically reduce connectivity and deployments times to approximately 30 days in some locations. Figure 9-8 depicts an Azure connection through an exchange provider.

The previous graphic illustrates the topology of the various pieces of a layer 2 ExpressRoute connection. Figure 9-8 does not depict the physical architecture for clarity reasons. For ExpressRoute connections to have an SLA, they must have two circuits with redundant routers on each end of the connection.

ExpressRoute has two connectivity models: layer 2 and layer 3. Layer 2 connectivity models are best used when a customer wants more granular control over the routing to and from the cloud or when they don’t have a layer 3 network already in place. Layer 2 ExpressRoute connections require the customer to provide the networking equipment, and the expertise to configure that equipment.

Layer 3 ExpressRoute

Layer 3 ExpressRoute connections are usually part of a telco providers’ Multiprotocol Label Switching (MPLS) network provided to the customer. The connections to Azure via ExpressRoute are merely ports on the mesh.

Think of an MPLS network as a spider web. Each connection on the spider web is a location on the WAN. If a single connection on the spider web is broken, that represents that location being offline, but no other location is affected. The other advantage of MPLS networks is the ability to create virtual direct connections across the WAN from one location to another, making them appear as a single hop, which you should notice by now, is what ExpressRoute does to Azure.

Figure 9-9 depicts an ExpressRoute connection from an MPLS WAN to Azure; note the differences between Figure 9-8 for layer 2 vs. this one for layer 3.

Layer 3 networks are usually managed by the telco, and the telco provides both the equipment and expertise for the configuration. Both the layer 2 and layer 3 ExpressRoute models require the use of two connections, between two routers on both the customer’s and Microsoft’s network edge, for a total of four identically configured routers, for Microsoft to guarantee the SLA. Point-to-point Ethernet connections are available to both layer 2 and layer 3 ExpressRoute customers, depending upon geography and the telco being used.

ExpressRoute Premium

For more information on the thresholds of Azure services such as ExpressRoute, please refer to the “Azure subscription and service limits, quotas, and constraints” page at https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits. This is a page worth saving since the various service thresholds continually improve. For more documentation on ExpressRoute Premium, please refer to https://docs.microsoft.com/en-us/azure/expressroute/expressroute-faqs#expressroute-premium.

ExpressRoute Direct

ExpressRoute Direct is a special, turbo-charged version of ExpressRoute with ultra-fast port speeds. ExpressRoute Direct is only available from very select ExpressRoute locations, which is at https://docs.microsoft.com/en-us/azure/expressroute/expressroute-locations. The ports offered at these locations are dual 10 Gbps or 100 Gbps into the Microsoft global network. The ExpressRoute Locations page referenced here provides the most up to date list of which network Connectivity Provides support ExpressRoute Direct. ExpressRoute Direct may have the Premium add-on applied to it, and all other metrics are the same as standard ExpressRoute regarding egress and redundancy. The billing of ExpressRoute Direct is different than standard ExpressRoute ports since the billing starts 45 days after the port is requested, or once it is first used, whichever comes first. ExpressRoute Direct is provisioned through specific service providers at the defined locations, which enables fast onboarding. For a detailed list of the ExpressRoute Direct requirements and deployment workflow, please refer to https://docs.microsoft.com/en-us/azure/expressroute/expressroute-erdirect-about.

ExpressRoute Global Reach

ExpressRoute Global Reach is an Azure service that allows customers to network various on-premises locations across the Microsoft Azure global network. The various Microsoft networking partners provide the “last mile” to your locations. ExpressRoute Global Reach lets Azure be your WAN backbone. Figure 9-10 depicts how ExpressRoute Global Reach can connect your on-premises infrastructures. Without ExpressRoute Global Reach, customers connecting to Azure over ExpressRoute would only be able to connect to each other’s Azure resources, not the on-premises resources.

Note how in the previous figure, even the Internet is accessed via the connection to Microsoft Azure. This is not required for ExpressRoute Global Reach but was illustrated as an example of the networking possibilities when using Microsoft Azure. This Internet model could be applied to any of the ExpressRoute architectures if desired.

Implementing ExpressRoute

Creating an Azure ExpressRoute circuit in the Azure portal is quite easy, but remember that’s only part of it. Follow these steps to create the ExpressRoute Circuit.

1. 1.

   In the Azure portal, navigate to ExpressRoute Circuits and click Add.

    
2. 2.

   Name the circuit.

   Note Naming conventions should make it easy to understand which circuit to pick when selecting from several of them.

    
3. 3.

   Choose the peering location.

    
4. 4.

   Select the bandwidth that you plan to use. Remember that you can always upgrade with no service interruption, so start conservatively.

    
5. 5.

   Select whether ExpressRoute Premium is required.

    
6. 6.

   Select whether you want the unlimited egress traffic data plan.

    
7. 7.

   Select the subscription.

    
8. 8.

   Select the resource group.

    
9. 9.

   Select the location in which you will be peering the Azure ExpressRoute circuit.

    
10. 10.

    Click Create.

     

Figure 9-11 illustrates the form filled out to create the ExpressRoute circuit. Of course, like everything in Azure, this could have also been done as infrastructure as code.

Once the ExpressRoute circuit is created, the next step is to contact your network connectivity provider and provide them with the Service Key. There are a few key items you can view from the ExpressRoute circuit properties shown in Figure 9-12.

The Subscription ID is needed for several things. This may be an individual subscription that your organization has set up for networking, to make it easy to track the charges or you may need to provide it to Microsoft for various support requests, such as enabling Microsoft peering for Office 365. The subscription can be scoped in Azure Cost Management and Billing. The service key is the GUID that must be provided to the connectivity provider to establish the connection. The connectivity provider may need other details such as VPN ID, VLAN ID, ASN, routing configuration if the provider manages, and so forth. In Figure 9-12, none of the peerings are provisioned. This is where administrators can check the status of their circuit as they work with their connectivity providers establishing the ExpressRoute connection. The ExpressRoute circuit status in the portal is the Microsoft side of the ExpressRoute connection.

Azure Virtual WAN

Azure Virtual WAN is a new Microsoft-managed cloud networking service that allows customers to connect multiple on-premises locations to Azure and network each of the on-premises locations together over the Microsoft Azure backbone in an automated and optimized fashion. Think of Virtual WAN as a competitor solution to an MPLS WAN, with the key differentiator being you still need the Telco for the last mile. Azure Virtual WAN allows Microsoft customers to use Microsoft as their WAN telco to the cloud and across their own WAN. Azure Virtual WAN implements a classic hub-and-spoke topology, where Azure is the hub, and each customer location is the end of a spoke.

Spokes can be

• On-premises locations

• Remote users

• VNets

• Internet

Azure Virtual WAN supports

• ExpressRoute

• S2S VPN

• P2S VPN

• VNet to VNet VPN

Azure Virtual WAN supports the following traffic flows.

• Branch to VNet

• Branch to branch

• Remote user to VNet

• Remote user to branch

• VNet to VNet

• Branch to hub to hub to branch

• Branch to hub to hub to VNet

• VNet to hub to hub to VNet

Azure Virtual WAN allows customers to choose their VPN endpoints are and which VPN concentrator in the Azure VNet is used. Azure Virtual WAN, illustrated in Figure 9-13, is like the MPLS network commonly used across enterprises today. Connections from on-premise don’t connect to the VNet that the hub is associated with; they connect with the Hub itself. Virtual WAN endpoints must conform to the Azure VPN requirements of supporting IKEv2/IKEv1 IPsec. Azure Virtual WAN uses Azure Firewall to secure traffic. If customers want a third-party solution, they can use any of the Virtual WAN partner solutions for their software-defined WAN (SDWAN) provider. Azure Virtual WAN supports network virtual appliances (NVA), such as a Fortinet, in line from the Virtual WAN Hub, as a spoke that all traffic is routed through.

Azure Virtual WAN hub gateways are highly available and are defined by what Microsoft calls scale units . A virtual WAN gateway hub scale unit is a metric for how much throughput the gateway can handle. It is twice the number represented because, since it is highly available, there are two instances, and each has that throughput allocated to it. This means you should consider the size you choose as twice what you select. Figure 9-14 shows the Azure portal when defining the gateway scale unit.

Azure Virtual WAN P2S VPN connections can be performed via OpenVPN, which is an SSL VPN protocol and can be used from mobile devices such as Android, iOS 11.0+, Windows, Linux, and macOS 10.13+. Azure Virtual WAN P2S VPN connections can also be performed via IKEv2 connections.

For more information on Azure Virtual WAN, please refer to https://docs.microsoft.com/en-us/azure/virtual-wan/virtual-wan-about.

Implementing Network Security Groups

Network security groups (NSG) contain rules that allow or deny traffic to or from a resource within it. The resource NSGs can have the entire subnet or an individual VM as the resource it is controlling access to. A VM could end up having multiple NSG policies being applied to it at the subnet and VM level individually, making troubleshooting difficult sometimes. Diagnostic tools in Network Watcher (discussed next) make this easier to troubleshoot and administer.

The NSG is an SDN-based firewall solution that is configured, as shown in Figure 9-15. It is a very similar user experience as a firewall configuration. NSG policy is allowing inbound RDP traffic from any source to any source.

If you click NSG RDP Rule, a blade appears. It allows administrators to configure this specific NSG rule such that we can specify an IP address. Figure 9-16 illustrates how we can perform port address translation (PAT) on the inbound traffic to obscure the port we’re exposing for RDP.

Implementing Security and Monitoring for networks

Azure Network Watcher provides a series of tools to surface monitoring metrics on a variety of Azure IaaS networking resources such as virtual machines, virtual networks, application gateways, and load balancers. Azure Network Watcher will not work on Microsoft Azure PaaS resources. Azure Network Watcher is enabled automatically on the creation of a virtual network and allows administrators to diagnose problems on virtual machines, virtual networks, network security groups, and other resources by evaluating traffic flows, latencies, routing, traffic filtering, and other problems on these resources via diagnostic logging. These diagnostic logs are consumable in Azure Monitor and Microsoft Power BI. For more information on Azure Application Gateway (AppGW) and network security groups, please refer to https://docs.microsoft.com/en-us/azure/azure-monitor/insights/azure-networking-analytics .

Network Watcher

Azure Network Watcher allows administrators to view resource-to-network relationships and network-to-network relationships; record the minimum, average, and maximum latencies over time; diagnose routing problems between VMs or virtual networks with next hop packet captures; test connectivity between network resources; troubleshoot and identify broken connections; and analyze network traffic logs from any network resource.

It allows administrators to enable diagnostic logging and IP flow, and verify packet capture and security group membership for Azure networking resources. This allows the manipulation of the resource’s location, configuration, and sizing to evaluate those effects on performance. Since all the data from Azure Network Watcher is surfaced in Azure Monitor, alerts can be generated when specific counters or resources exceed a defined threshold. Figure 9-17 shows a simple RHES Linux VM running in Azure on a VNet protected by an NSG.

NSG flow logs allow administrators to view NetFlow, or ingress/egress, data per rule passing through the NSG. NSG flow logs can be enabled and configured per NSG, and each can have its data stored in its own Log Analytics workspace, or they can be combined. This is very useful in satisfying security requirements and regulatory compliance. NSG flow logs record five-tuple information about the flow (source/destination IP, source/destination port, and protocol) and if the traffic was allowed or denied. Administrators can choose to enable v2 of the NSG flow logs, which record bytes and packet data.

Network Performance Monitor

Network Performance Monitor (NPM) has been rolled into Network Watcher as a subcomponent. This allows administrators to have a single pane of glass, or single blade in the Azure portal, for all their network monitoring and administration. NPM is a hybrid monitoring solution that incorporates legacy stateful monitoring tactics with more dynamic approaches to monitoring endpoints and applications. Figure 9-18 illustrates the NPM dashboard.

NPM, which has its network link performance-monitoring dashboard illustrated in Figure 9-19, can detect traffic blackholing, routing errors, and issues that conventional network monitoring methods aren’t able to detect. NPM has three primary capabilities.

• Performance Monitor: Monitors across clouds and on-premises locations.

• Service Connectivity Monitor: Emulates a synthetic transaction from a user to a service, as illustrated in Figure 9-20. This is useful for identifying performance problems on applications as well as the end-user experience since you’re testing from their endpoint, not within the same datacenter or network.

• ExpressRoute Monitor: End-to-end connectivity and performance between your on-premises locations and Azure. Figure 9-21 illustrates the global connectivity from the East US region.

NPM takes all the network monitoring capability built into Microsoft System Center Operations Manager and adds to it by leveraging cloud scale and analytics.

NPM performs its work via the same agent used for Azure Monitor, the Microsoft Management Agent (MMA). MMA can be downloaded from your Azure portal in the Log Analytics workspace blade.

Once configured, by following the instructions at https://docs.microsoft.com/en-us/azure/azure-monitor/platform/agent-windows, administrators can start monitoring their endpoints, services, applications, and network devices without the heavy infrastructure requirement typically found in most solutions today. Figure 9-22 illustrates how NPM can monitor the individual hops, latency between hops, and routes on a given workload.

Summary

We hope that you’ve seen that we can only scratch the surface of what’s available in Microsoft Azure’s networking portfolio. This should give you confidence that there is much more to the networking capabilities than we can cover in a single chapter. An entire book could be devoted to this topic. Microsoft has developed the second-largest network in the world, which shows in the solutions available to the customer—many of which are PaaS services that would have cost tens of thousands of dollars and countless hours to set up, maintain, and upgrade. To reiterate, all of this can be done as infrastructure as code and deployed via templates.