Azure has multiple methods in which administrators can connect to their on-premises environment and extend their datacenter into the cloud. Different workloads, security considerations, and routing requirements of the various use cases may warrant a combination of a few different types of connections.
Azure virtual network
Azure Load Balancer
Azure DNS
Azure ExpressRoute
Azure Traffic Manager
Azure VPN gateway
Azure Application Gateway
Internet Connectivity
Internet-based workloads in Azure work well for many use cases. When you don’t need the workload to traverse a private or internal network, if the target user population of the workload is coming from the Internet, if the user population is a different organization, making the workload Internet-facing can have many advantages, including security. If you are making a workload external facing to “untrusted” systems and users, meaning they aren’t managed by your organization, not coupling the Azure environment to your on-premises environment provides a security boundary. If a bad actor or malware were able to breach your workload, then the workload is all they would have access to, not on-premises.
Azure VPN
VPN connectivity is frequently used when dedicated private circuits don’t exist between locations that want to share IT services, such as Branch Offices or datacenters. VPN tunnels can either be initiated from the individual user’s device from anywhere in the world, such as a coffee shop or house, known as Point-to-Site (P2S) VPN, or from an on-premises location to your on-premises datacenter. This is commonly referred to as a site-to-site (S2S) VPN tunnel, frequently used for “branch office” or low-cost office to office network connectivity. The key difference between P2S and S2S VPN tunnels is P2S VPN tunnels connect one resource to a remote datacenter or network, whereas S2S VPN tunnels connect many systems and users to a remote datacenter or network.
P2S and S2S VPN connections are designed to run over the Internet but can run on top of a private network connection. Both use the industry-standard protocols Internet Protocol Security (IPsec) and Internet Key Exchange (IKE). These protocols ensure the security of the connection. If the connection is broken or suspected of being tampered with, the connection drops as a security precaution. The connection then needs to be reestablished to continue communication to the remote endpoints or services. A P2S VPN connection usually requires the user to reinitiate the connection. You are likely familiar with this from having experienced this behavior on unreliable networks at hotels or cafés or having to use your smartphone as an Internet hotspot. S2S VPN connections usually have their tunnel reestablished automatically as soon as the first packets attempt to traverse the tunnel.
A virtual network gateway is made up of VMs that are deployed to a specific subnet you create called the gateway subnet . The deployment of these VMs is the reason why standing up a VPN connection to Azure can take up to 45 minutes. The VMs provisioned when you create the virtual network gateway contain routing tables and run specific gateway services, which cannot be configured by administrators. Like all Azure services, if high availability is desired, administrators can deploy VPN gateways to Azure Availability Zones. This creates multiple gateways in an Azure region, which are physically and logically separated.
Azure VPN gateways come in several sizes, with varying performance metrics such as the number of concurrent connections, the connection speeds, and whether Zone Redundancy is an option for high availability. Azure VPN gateways range in speed from 100 Mbps to 10 Gbps. When Azure VPN gateways are used from on-premise to Azure, regardless of whether it is P2S or S2S, the Internet carrier is primarily responsible for the network latency and throughput; the ISP is usually the bottleneck.
In addition to the ISP being the bottleneck, assuming you’ve chosen a fast Azure VPN gateway, the ISP’s Internet connection has no performance service-level agreement (SLA). Because the Internet has so many parties that could cause problems on an ISP’s network (and because they’re outside the ISP’s control), the negative performance and lack of an SLA is a major reason why VPN-over-Internet is not used when private dedicated circuits like ExpressRoute are available. For a detailed list of all Azure VPN gateways and architectural illustrations, please refer to https://docs.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-about-vpngateways.
ExpressRoute
Azure ExpressRoute is a private, dedicated, high-bandwidth, low-latency, SLA’d connection into the Microsoft Azure global network. ExpressRoute allows connectivity not only to Azure resources, but also Office 365 and Dynamics 365 at speeds from 50 Mbps to 100 Gbps. ExpressRoute uses private peering for connectivity to an Azure virtual network, and Microsoft peering for connectivity to PaaS and SaaS workloads, such as Office 365 and Dynamics 365. Peering is a connection between two different networks over the Internet for the sake of exchanging data between them. Peering allows customers to have more control over the routing, performance, capacity, and redundancy. Therefore, the peering process is key to Azure services’ connectivity when SLA high bandwidth or low latency is desired.
- Azure IP Ranges and Service Tags – Public Cloud
- Azure IP Ranges and Service Tags – US Government Cloud
- Office 365 URLs and IP address ranges
- Microsoft Dynamics CRM Online IP Address Ranges
Each of the ExpressRoute routers can have ExpressRoute Private and Microsoft peering enabled for the various, IaaS, PaaS, and SaaS workloads, as shown in Figure 9-5. These IP ranges listed previously are added to the routers on the customer side of the connections illustrated in Figure 9-5. These routers are set up in an Active/Passive architecture. If they are utilized in Active/Active, the SLA is not supported.
In Figure 9-6, Microsoft peering from on-premises connects on the outside of the Internet-facing firewall, using a public IP address to peer with Microsoft’s PaaS and SaaS services, which also use public IPs because they are Internet-facing. The private peering uses private IPs, as the name depicts, and comes from inside the environment to the IaaS VNet(s). Each of these peering types can be run in parallel, with multiple instances of each, or by themselves.
ExpressRoute locations are frequently referred to as peering locations, meet-me locations, or points-of-presence (PoP). These locations are managed by several Microsoft ExpressRoute partners, known as connectivity providers . AT&T, CenturyLink, Orange, and Tata Communications are a few of the connectivity providers.
When you use an ISP not listed as a connectivity provider on the “Azure ExpressRoute partners and peering locations” page listed previously, customers can still connect to Azure through exchange providers. Exchange providers provide ExpressRoute at layer 2, or the data link layer, on the OSI model. We’ll review Azure layer 2 ExpressRoute connections later in this chapter.
If ExpressRoute connectivity providers or exchanges aren’t available in your geography, you may also connect to Azure through satellite operators. This should only be considered for Azure connectivity if lower latency options are not available. Satellite connectivity for ExpressRoute makes Azure one hop away but with a latency subject to the many hundreds of miles each way to communicate with the satellite. ExpressRoute over satellite may be the only option in locations like at sea or very remote locations of the globe.
ExpressRoute ports can be ordered in both a metered plan and an unlimited data plan. Both plans’ costs include high-availability dual ports and are billed by the port speed monthly. ExpressRoute charges cited online and through billing reflect two ports on two routers. Both ExpressRoute data plans included unlimited ingress traffic, but only the unlimited data plan includes unlimited outbound traffic. The cost of egress traffic on the Metered Data plan varies by location. For example, North America is $0.025 per GB. A detailed breakout of ExpressRoute pricing is at https://azure.microsoft.com/en-us/pricing/details/expressroute/. ExpressRoute ports can be upgraded or downgraded at any time, but only upgrading the circuit can be done with no service interruption at this time.
Decreasing an ExpressRoute port speed requires deleting it and recreating a lower speed port. This causes service interruption.
Layer 2 ExpressRoute
The previous graphic illustrates the topology of the various pieces of a layer 2 ExpressRoute connection. Figure 9-8 does not depict the physical architecture for clarity reasons. For ExpressRoute connections to have an SLA, they must have two circuits with redundant routers on each end of the connection.
ExpressRoute has two connectivity models: layer 2 and layer 3. Layer 2 connectivity models are best used when a customer wants more granular control over the routing to and from the cloud or when they don’t have a layer 3 network already in place. Layer 2 ExpressRoute connections require the customer to provide the networking equipment, and the expertise to configure that equipment.
Layer 3 ExpressRoute
Layer 3 ExpressRoute connections are usually part of a telco providers’ Multiprotocol Label Switching (MPLS) network provided to the customer. The connections to Azure via ExpressRoute are merely ports on the mesh.
Think of an MPLS network as a spider web. Each connection on the spider web is a location on the WAN. If a single connection on the spider web is broken, that represents that location being offline, but no other location is affected. The other advantage of MPLS networks is the ability to create virtual direct connections across the WAN from one location to another, making them appear as a single hop, which you should notice by now, is what ExpressRoute does to Azure.
Layer 3 networks are usually managed by the telco, and the telco provides both the equipment and expertise for the configuration. Both the layer 2 and layer 3 ExpressRoute models require the use of two connections, between two routers on both the customer’s and Microsoft’s network edge, for a total of four identically configured routers, for Microsoft to guarantee the SLA. Point-to-point Ethernet connections are available to both layer 2 and layer 3 ExpressRoute customers, depending upon geography and the telco being used.
ExpressRoute Premium
Increased routing table limit from 4000 routes to 10,000 routes for private peering
Increased number of VNets and ExpressRoute Global Reach connections that can be enabled on an ExpressRoute circuit
Connectivity to Office 365
Global connectivity over the Microsoft core network
For more information on the thresholds of Azure services such as ExpressRoute, please refer to the “Azure subscription and service limits, quotas, and constraints” page at https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits. This is a page worth saving since the various service thresholds continually improve. For more documentation on ExpressRoute Premium, please refer to https://docs.microsoft.com/en-us/azure/expressroute/expressroute-faqs#expressroute-premium.
ExpressRoute Direct
ExpressRoute Direct is a special, turbo-charged version of ExpressRoute with ultra-fast port speeds. ExpressRoute Direct is only available from very select ExpressRoute locations, which is at https://docs.microsoft.com/en-us/azure/expressroute/expressroute-locations. The ports offered at these locations are dual 10 Gbps or 100 Gbps into the Microsoft global network. The ExpressRoute Locations page referenced here provides the most up to date list of which network Connectivity Provides support ExpressRoute Direct. ExpressRoute Direct may have the Premium add-on applied to it, and all other metrics are the same as standard ExpressRoute regarding egress and redundancy. The billing of ExpressRoute Direct is different than standard ExpressRoute ports since the billing starts 45 days after the port is requested, or once it is first used, whichever comes first. ExpressRoute Direct is provisioned through specific service providers at the defined locations, which enables fast onboarding. For a detailed list of the ExpressRoute Direct requirements and deployment workflow, please refer to https://docs.microsoft.com/en-us/azure/expressroute/expressroute-erdirect-about.
ExpressRoute Global Reach
Note how in the previous figure, even the Internet is accessed via the connection to Microsoft Azure. This is not required for ExpressRoute Global Reach but was illustrated as an example of the networking possibilities when using Microsoft Azure. This Internet model could be applied to any of the ExpressRoute architectures if desired.
Implementing ExpressRoute
- 1.
In the Azure portal, navigate to ExpressRoute Circuits and click Add.
- 2.
Name the circuit.
Note Naming conventions should make it easy to understand which circuit to pick when selecting from several of them.
- 3.
Choose the peering location.
- 4.
Select the bandwidth that you plan to use. Remember that you can always upgrade with no service interruption, so start conservatively.
- 5.
Select whether ExpressRoute Premium is required.
- 6.
Select whether you want the unlimited egress traffic data plan.
- 7.
Select the subscription.
- 8.
Select the resource group.
- 9.
Select the location in which you will be peering the Azure ExpressRoute circuit.
- 10.
Click Create.
The Subscription ID is needed for several things. This may be an individual subscription that your organization has set up for networking, to make it easy to track the charges or you may need to provide it to Microsoft for various support requests, such as enabling Microsoft peering for Office 365. The subscription can be scoped in Azure Cost Management and Billing. The service key is the GUID that must be provided to the connectivity provider to establish the connection. The connectivity provider may need other details such as VPN ID, VLAN ID, ASN, routing configuration if the provider manages, and so forth. In Figure 9-12, none of the peerings are provisioned. This is where administrators can check the status of their circuit as they work with their connectivity providers establishing the ExpressRoute connection. The ExpressRoute circuit status in the portal is the Microsoft side of the ExpressRoute connection.
Azure Virtual WAN
Azure Virtual WAN is a new Microsoft-managed cloud networking service that allows customers to connect multiple on-premises locations to Azure and network each of the on-premises locations together over the Microsoft Azure backbone in an automated and optimized fashion. Think of Virtual WAN as a competitor solution to an MPLS WAN, with the key differentiator being you still need the Telco for the last mile. Azure Virtual WAN allows Microsoft customers to use Microsoft as their WAN telco to the cloud and across their own WAN. Azure Virtual WAN implements a classic hub-and-spoke topology, where Azure is the hub, and each customer location is the end of a spoke.
On-premises locations
Remote users
VNets
Internet
ExpressRoute
S2S VPN
P2S VPN
VNet to VNet VPN
Branch to VNet
Branch to branch
Remote user to VNet
Remote user to branch
VNet to VNet
Branch to hub to hub to branch
Branch to hub to hub to VNet
VNet to hub to hub to VNet
Azure Virtual WAN P2S VPN connections can be performed via OpenVPN, which is an SSL VPN protocol and can be used from mobile devices such as Android, iOS 11.0+, Windows, Linux, and macOS 10.13+. Azure Virtual WAN P2S VPN connections can also be performed via IKEv2 connections.
For more information on Azure Virtual WAN, please refer to https://docs.microsoft.com/en-us/azure/virtual-wan/virtual-wan-about.
Implementing Network Security Groups
Network security groups (NSG) contain rules that allow or deny traffic to or from a resource within it. The resource NSGs can have the entire subnet or an individual VM as the resource it is controlling access to. A VM could end up having multiple NSG policies being applied to it at the subnet and VM level individually, making troubleshooting difficult sometimes. Diagnostic tools in Network Watcher (discussed next) make this easier to troubleshoot and administer.
Implementing Security and Monitoring for networks
Azure Network Watcher provides a series of tools to surface monitoring metrics on a variety of Azure IaaS networking resources such as virtual machines, virtual networks, application gateways, and load balancers. Azure Network Watcher will not work on Microsoft Azure PaaS resources. Azure Network Watcher is enabled automatically on the creation of a virtual network and allows administrators to diagnose problems on virtual machines, virtual networks, network security groups, and other resources by evaluating traffic flows, latencies, routing, traffic filtering, and other problems on these resources via diagnostic logging. These diagnostic logs are consumable in Azure Monitor and Microsoft Power BI. For more information on Azure Application Gateway (AppGW) and network security groups, please refer to https://docs.microsoft.com/en-us/azure/azure-monitor/insights/azure-networking-analytics .
Network Watcher
Azure Network Watcher allows administrators to view resource-to-network relationships and network-to-network relationships; record the minimum, average, and maximum latencies over time; diagnose routing problems between VMs or virtual networks with next hop packet captures; test connectivity between network resources; troubleshoot and identify broken connections; and analyze network traffic logs from any network resource.
NSG flow logs allow administrators to view NetFlow, or ingress/egress, data per rule passing through the NSG. NSG flow logs can be enabled and configured per NSG, and each can have its data stored in its own Log Analytics workspace, or they can be combined. This is very useful in satisfying security requirements and regulatory compliance. NSG flow logs record five-tuple information about the flow (source/destination IP, source/destination port, and protocol) and if the traffic was allowed or denied. Administrators can choose to enable v2 of the NSG flow logs, which record bytes and packet data.
Network Performance Monitor
Performance Monitor: Monitors across clouds and on-premises locations.
Service Connectivity Monitor: Emulates a synthetic transaction from a user to a service, as illustrated in Figure 9-20. This is useful for identifying performance problems on applications as well as the end-user experience since you’re testing from their endpoint, not within the same datacenter or network.
ExpressRoute Monitor: End-to-end connectivity and performance between your on-premises locations and Azure. Figure 9-21 illustrates the global connectivity from the East US region.
NPM performs its work via the same agent used for Azure Monitor, the Microsoft Management Agent (MMA). MMA can be downloaded from your Azure portal in the Log Analytics workspace blade.
Summary
We hope that you’ve seen that we can only scratch the surface of what’s available in Microsoft Azure’s networking portfolio. This should give you confidence that there is much more to the networking capabilities than we can cover in a single chapter. An entire book could be devoted to this topic. Microsoft has developed the second-largest network in the world, which shows in the solutions available to the customer—many of which are PaaS services that would have cost tens of thousands of dollars and countless hours to set up, maintain, and upgrade. To reiterate, all of this can be done as infrastructure as code and deployed via templates.