4. Managing Broadband Networks

In this chapter:

• Introduction

• The Public Network

• Core Networks—Between Cities and Continents

• Bandwidth Capabilities in Carrier Networks

• Optical Transport Networks—Carrying Multiple Types of Traffic

• Transporting Movies and TV in the Core

• Middle-Mile Networks

• Last-Mile Access Networks

• Access Networks in Cable Operators’ Networks

• Using Cable Modem Termination Systems for IP Traffic

• Telecommunications Services in National Emergencies

• Signaling

• Appendix

Introduction

Growing dependence on cloud computing, social networking, and in particular video streaming has caused data traffic to increase more rapidly than ever before. According to Cisco, busy hour data traffic increased 32 percent from 2015 to 2016, and 51 percent from 2016 to 2017. This increase in traffic volume presents enormous challenges to carriers. Adding capacity to networks does not solve issues of managing the added video on networks. In addition to adding capacity, telephone and cable TV operators are implementing software to:

• Automatically reroute traffic around congested or failed routes

• Prioritize content on networks

• Streamline the process of implementing new revenue, producing services for residential and enterprise subscribers.

• Automatically reroute traffic around congested or failed routes

• Quickly and cost effectively deploy and back up network gear

• Quickly add new subscriber and enterprise features by enabling subscribers to activate them

• Manage routing to minimize traffic delays

• Provide 99 percent network uptime

Software Defined Networking (SDN) is being implemented in providers’ networks to accomplish the above goals via policies defined in SDN software. SDN control software enables automatic network functions, such as rerouting traffic around network outages. SDN is installed on computers in network operations centers where technicians can react to major outages.

Network Function Virtualization (NFV) is a technology that eliminates the need to install network functions on proprietary hardware. Instead, multiple network functions, including routing and switching, are abstracted in software and installed on commodity servers. The power of NFV is that these pieces of abstracted software can be inexpensively duplicated and backed up in spare servers. Importantly, NFV in networks provides the ability for network functions to be upgraded and equipment installed at a faster pace with lower costs.

To accommodate the growing percentage of data traffic, central offices are undergoing a transition that enables them to better manage the data in their networks from centralized locations. Central Office Re-architected as a Data Center (CORD) specifies re-engineering central offices to be more like enterprise data centers. CORD is used by telephone companies to manage their shared, centralized applications and functions in a similar way to data centers in enterprises.

Similarly to carriers’ broadband networks, submarine networks that connect continents using fiber-optic cabling must manage growing amounts of data. According to consulting firm TeleGeography, traffic on undersea cables increased 45 percent in 2016. Reflecting the change in international traffic patterns, traffic from carriers no longer represents the major source of traffic on submarine cables. Rather, traffic from content and cloud providers including Facebook, Microsoft, and Google now generate the largest percentage of traffic on submarine networks.

The desire to ensure adequate capacity on submarine cables has led some organizations to take partial ownership of submarine cable routes. Facebook and Google together own one third of the capacity of a cable route between Hong Kong and California. Microsoft and Amazon also own parts of other submarine cables. Modifications to the electronics connected to fiber-optic cabling on submarine cables are done to increase capacity on submarine cables.

Telephone companies transport traffic on core networks between large cities. They additionally carry traffic on middle-mile networks that connect rural areas to on-ramps to the Internet, and on last-mile networks to individual subscribers and enterprises. Upgrading and installing last-mile, edge networks is challenging. Last-mile networks require the most complex cabling, and entail the highest costs. This is because of the large number of individual connections to homes, enterprises, and multi-dwelling buildings.

Keeping networks available and operational during national emergencies is a goal of all countries worldwide. Monitoring and managing networks for sustainability and the ability to communicate during national emergencies is a major challenge. Alternate routes, software management of networks, and continuous monitoring are critical in ensuring that vital networks are available when businesses, governments, and individuals need them in emergencies and for conducting day-to-day business. In ever-increasing numbers, people depend on broadband networks to stay in touch, access applications, get news, and participate in electronic communications during day-to-day activities.

The Public Network

All networks including mobile, Wi-Fi and LAN networks have access (edge), metropolitan (backhaul), and backbone sections. Rural areas generally also have middle-mile networks. The structure of public networks can be broken into five categories. The list that follows provides a brief description of each.

1. Last-mile (also referred to as edge, first-mile, or access networks).

2. Metropolitan area networks (MANs) within cities.

3. Middle-mile or backhaul, located between small cities and long-haul networks, connect rural areas to long-haul (core) networks.

4. Regional networks connect cities that range from 200 to 400 miles apart. This includes Boston to New York City, and New York City to Washington DC. These links carry traffic in large cities to cable TV and telephone company switches or routers where traffic is aggregated and connected to backbone or middle-mile networks.

5. Core networks connect cities and continents. Core networks are also referred to as the backbone section of networks.

Sections of Carriers’ Broadband Networks

Core Networks—Between Cities and Continents

Core networks, largely based on IP, transmit the highest concentration of traffic in public networks. They transmit packetized voice (VoIP), shrinking amounts of non-VoIP voice traffic, data, and video on high-capacity fiber-optic cabling. Traffic in the core is transmitted across countries, between countries and continents, and under oceans.

At multiple points along the route, the traffic is:

• Dropped off at wiring centers and Points of Presence (POPs) closer to urban and suburban areas.

• Handed off to other long-haul carriers to be transmitted on other routes.

• Transmitted to Metropolitan Area Networks (MANs) located in cities.

• Sent to residential and business customers over last-mile networks by either the long-haul carrier, the local landline telephone company, or a cellular telephone company.

Carriers also transmit data and VoIP traffic between enterprise sites on long-haul networks. Special arrangements are made with these customers for optional services such as security and priority treatment for voice and video conferencing. These networks are considered part of a carrier’s private data networks. They are separate from the Internet and offer guaranteed speeds and low latency (unnoticeable levels of delay or no delay) on an end-to-end basis. For more on this, see Chapter 5, “Broadband Network Services.”

Software Defined Networks to Manage Traffic Surges

Telephone companies are increasingly upgrading sections of their networks, often in anticipation of upgrading fully to SDN to manage the annual doubling of traffic on networks. SDN, as the name implies, controls networks via policies created in software to manage traffic. The growing amounts of traffic in carriers’ networks is due mainly to data the increasing amounts of traffic to and from the cloud, and from streaming video from Netflix, Amazon, and others.

Movies and TV shows that formerly were exclusively available on cable TV networks are now streamed 24 hours a day from sources such as Amazon, Netflix, and Hulu. This causes unpredictable spikes in traffic. Increasingly, cable TV and traditional telephone company customers want anytime-anywhere availability of movies and TV shows. They stream content to smartphones, tablet computers, and Internet-connected televisions at any hour of the day or night.

Streaming video needs to be treated differently from e-mail and text messages. Video in the form of movies and conference calls needs to be prioritized so that delays don’t cause breaks and pauses in the audio or video. In order to dynamically treat these massive amounts of traffic, traffic flows are prioritized in real time by type of content and often by application as well. For example, e-mail usually has a lower priority than real time streaming video because short delays on e-mail and text messages aren’t noticeable. The software control in SDN is able to manage these peaks and prioritize particular types of traffic.

Software Defined Networks have a centralized controller that specifies the treatment of applications and handling of traffic. This is accomplished via policies sent to network nodes in real time, directing the use of particular routes, and priorities for traffic flows. These policies are transmitted on the control plane. The controller sends out policies in real time to network nodes (switches and routers) that transmit the actual data. For example, if a route is out of service, the controller will transmit updated policies to nodes so that all traffic is rerouted to a functioning path in the network.

The following are examples of control plane policies:

• Redirecting traffic during peaks.

• Opening bandwidth when it is needed during an event such as a presidential inauguration. For example, carriers commonly rate-limit traffic by running a 400Gbps pipe at 10Gbps. When required, the bandwidth can be easily increased to 400Gbps.

• Reconfiguring an optical multiplexer on the fly, automatically, or manually, via remotely entered computer commands.

• Selling web portal access to IT managers so that they can add more bandwidth for a special event without waiting for the carrier to provision more service.

• Decreasing delays and increasing revenue from new services more quickly by speeding up installations such as cable modems and high-speed network access for commercial customers. This means that an installer can make one visit to an enterprise to install gear, and then power up the service remotely through software. Previously, multiple onsite trips were required for setup.

Network Function Virtualization—Architecture

Network Function Virtualization (NFV) is an architecture in which network functions such as storage, routers, and switches are represented in software. These functions no longer need to be installed on dedicated, vendor-specific hardware. It’s similar to server virtualization where multiple applications are installed as virtual machines on servers. However, instead of applications being virtualized, with NFV hardware devices (network nodes) are represented as software in servers. NFV specifies the way that these software functions fit together.

Virtual Network Function—Transforming Hardware Nodes into Software Functions

Virtual network function (VNF) refers to each piece of software in virtualized servers that represents a hardware function. These functions, some of which are listed below, can work together, are easily duplicated, and are generally installed on commodity hardware such as X86 and newer type servers with additional processing power. See Figure 4-1 for an example of VNFs in a single server. The equipment in which multiple VNFs are installed is referred to as a White Box.

The following are examples of Virtual Network Functions, which can be virtualized as software within servers and White Boxes:

• Routers that route traffic between networks

• Switches that transmit traffic within a network

• Load balancers that balance traffic between multiple broadband circuits so that no single circuit (path) is unduly congested

• Controllers that manage applications and send commands to devices

• Gateways that translate between networks that use different protocols

• Network Address Translation (NAT), which translates between external IP addresses and internal IP addresses for internal devices

• Security software in appliances (appliances are hardware dedicated to a single function)

• Firewalls that screen incoming traffic for viruses and other types of attacks

The key advantages of installing network functions in software are flexibility, agility, and speed of implementation. Functions can be deployed and duplicated easily and quickly using computer commands in software. Instead of programming, purchasing, and managing distinct pieces of hardware, control software can be used to add network functions to the parts of networks where they are needed.

Because these critical functions are all in one server, if the server malfunctions, all of these functions are lost. For greater reliability and sustainability (the ability to be operational during malfunctions), telephone companies often deploy duplicate White Boxes. If one fails the other is able to take over.

Policies defined within Software Defined Networks indicate how to route, prioritize, balance traffic loads, and program firewalls. The policies that control networks are programmed and defined by IT staff at computer screens. They are then invoked automatically as required.

Bringing Up Network Functions—Open Source MANO

Open Source Management and Organization (MANO) is an ETSI ISG standard. ETSI ISG is the European Telecommunications Standards Institute Industry Specification Group. Open source MANO refers to the ability to manage and bring up network functions uniformly by carriers. The goal is to enable networks to interoperate with each other and for telephone companies to be able to purchase commodity hardware. The standard defines operations in wired telephone company networks and cable TV as well as 5G mobile networks. ETSI’s Industry Specification Group’s goal is to prevent chaos and disorganization when deploying NFV software within SDNs.

Sixty of the largest telephone companies worldwide are members of the MANO (Management and Organization) specification group that defined the MANO standard. The standards are meant to be interoperable with SDN controllers, VFN architecture, and VNF software.

Just as in large data centers where “virtual machine sprawl” can result in unnecessary duplication, organizations can lose control of the location and number of virtualized network functions without standards on how to manage and deploy these functions. MANO is designed to operate in cloud settings as well as onsite data centers. The ETSI NFV MANO standard includes:

• Interfaces to Operations Support System (OSS) to manage installation of carriers’ services, and changes

• Ways that carriers’ MANO customers are able to remotely program changes to their services. This is particularly relevant to enterprises that want control of their network services

• Interfaces to Billing Support Systems (BSS) to manage telephone companies’ billing applications

• The ability to manage elements on Amazon’s cloud services including EC2 (elastic cloud) by use of a plug-in. A plug-in is a small program that adds capabilities to a parent (larger) program. Plug-ins are also used in browsers

• Control and management of virtualized computing, storage and other network resources

• The ability to use containers. See Chapter 2, “Data Centers and LANs, Storage, and IP Private Branch Exchanges,” for containers

• Consistent implementation of open standards across the industry

• Troubleshooting capabilities within networks

Central Office Re-Architected as a Data Center—Streamlined Central Offices

CORD is a set of specifications that lays out a way to design and equip a telephone company’s central offices as data centers. The purpose is to implement central offices to be compatible with SDN and NFV so that new services and applications can be delivered more quickly. It specifies open source software and commodity hardware to be used in central offices. ON.Lab, a non-profit industry association based in Berkeley, California developed CORD, in conjunction with large telephone and cable TV companies, and industry manufacturers. Their goal is to define standardized open source communications network standards for central offices.

In contrast to CORD’s open source software and commodity hardware, most current and older central offices are made up of large, costly proprietary switches and specialized equipment. These switches are similar to telephone switches used by enterprises. However, they have greater capacity and are “hardened” for greater reliability and sustainability to withstand natural disasters such as hurricanes and tornadoes. Moreover, traditional central offices operate on Direct Current (DC) power. CORD uses Alternate Current (AC) power. Equipping and upgrading traditional central offices is costly and often ties telephone companies into purchasing costly equipment from a single manufacturer in each central office so that all the equipment is compatible and able to interoperate together.

CORD is architected for mobile as well as fixed line networks. There are also versions for telephone and cable TVs’ edge networks, which are connected to enterprise and consumer customers. In contrast to today’s central offices, Central Office Re-architected as a Data center makes use of Network Function Virtualization (NFV) to create software copies of network equipment such as servers, switches, and hardware that support and manage access to fiber cabling, cable modems, and mobile networks.

CORD is meant to emulate cloud services that depend on virtualization, can scale up or down easily, and whose features are managed directly by customers. The software in CORD-based central offices can be located in the cloud and managed by manufacturers, or remotely by telephone companies. They may alternately be managed and located at carriers’ own data centers.

The goal, in addition to saving money on central offices, is to simplify and importantly, speed up the availability of new applications for enterprise, mobile, and residential landline telephone companies’ customers. This is a potential source of new revenue for carriers and may lower telephone companies’ total cost of ownership (TCO).

The Central Office Re-architected as a Data center standard includes capabilities that enable enterprise customers to access and download new services and/or change existing features remotely via centrally located consoles. According to the March 22, 2017, LightReading article “CenturyLink Delivers DSL Using CORD Platform,” CenturyLink is supplying DSL to enterprise and residential customers from its CORD central offices. In another article published by Fierce Telecom on February 9, 2017, Glen Post, CEO and Chairman of CenturyLink, was quoted as following:

We plan to have 100% of those (POPs) virtualized by the end of 2019.

Enterprise customers have the ability to manage services such as DSL at centrally located consoles from headquarters and branch offices. The modifications and additions can be performed without carriers dispatching technicians to customer sites. This saves telephone companies the expense of “truck rolls,” dispatching technicians to customers’ locations.

The Pace of Implementation of SDN, NFV, and CORD

Software Defined Networks (SDN), and Network Function virtualization (NFV), are technologies that rely on software to manage functions that previously relied on hardware such as routing and firewalls. Central Office Re-architected as a Data Center (CORD) enables telephone companies and enterprises to more efficiently manage the increasing amounts of data.

One strategy that telephone companies use to implement Central Office Re-architected as a Data Center (CORD) is to add CORD data center elements within their traditional central office. In this way, the functions in traditional central offices can be gradually transitioned to CORD equipment and virtualized servers.

Telephone companies are additionally merging mobile central office functions (the core in mobile networks) and their landline central office functions in CORD data centers. During the transitions clashes between the two types of staff may occur. However, the end result is lower costs to operate networks.

Staff that monitored network conditions from traditional central offices needed to monitor large telephone networks. In CORD, central offices cloud development (DevOps) skills are required to manage new central offices that are essentially large data centers.

Submarine Network Systems

Submarine cables connect all of the earth’s continents to each other. They connect Asia to the United States, and Africa and Europe to the Americas. Underwater cables are placed directly on the ocean floor. They are connected to land at drop-off points, also known as landing points, on each continent. Figure 4-3 depicts an example of a submarine cable system route across the Pacific Ocean.

At landing points, a packet switch or a router delivers traffic to land-based co-location sites where many carriers have a presence. One of these co-location sites is on Eighth Avenue in New York City. At the co-location facility, multiple carriers interconnect their networks and carry one another’s traffic to other cities.

Advances in Submarine Cabling Technology

Advances in submarine fiber-optic cable networks took a leap forward between 2012 and 2015. This is when coherent fiber-optic technology enabled already installed fiber-optic cables to be upgraded to carry additional traffic. Coherent optical electronics connected to fiber-optic cables enabled more channels of traffic to be carried with tighter spacing between each channel of data. Without this innovation, the current undersea cables would be out of capacity, and additional cables would have to be laid. According to Telegeography, a consulting firm owned by Primerica and located in Washington DC, by 2020 additional cables will need to be built because of anticipated traffic growth.

Power Limitations

Power must be supplied at the terrestrial end of each fiber bundle. In contrast to terrestrial fiber where power can be added along the route, it’s not possible to supply power at various points in each undersea cable bundle.

Currently available power technology at each end of a bundle of fiber cables supports only six fiber pairs. This limits the number of fiber pairs in each route. Various organizations are looking at ways to increase power capabilities so that additional strands of fiber can be supported in each bundle.

Traffic Usage—Cloud and Content Providers

Content and cloud providers use the lion’s share of capacity on undersea cables. These hyper-scaled operators are Google, Facebook, Amazon, and Microsoft. Additionally, in recent years there is added traffic between Africa and Europe, particularly between the east coast of Africa and Europe. This has resulted in additional undersea capacity between Europe and Africa.

The above content provider and cloud providers’ primary motivation in taking ownership stakes in cable routes is to be assured of adequate capacity. Prior to the enormous growth in cloud and search traffic, carriers such as AT&T and Verizon used the most capacity in submarine cable systems.

Cable Cuts in Submarine Networks

Cable cuts, when they occur, are major problems, disrupting communications between entire countries. Fishing trawlers are responsible for most accidental cable cuts. Due to the specialized equipment needed to repair cables that lie miles deep in oceans, it often takes weeks to repair a cable. See Figure 4-4 for a list of time to repair cable cuts. Repairing undersea cable cuts requires the costly deployment of submersible robots tethered to and controlled by motherships. Large trawling fishing nets are weighted and often snag cabling as they are being dragged across ocean floors. The second most frequent cause of outages results from ship anchors dragged across the ocean floor.

The following are the reasons for delays in repairing cables:

• There are a limited number of ships equipped to repair submarine cable cuts.

• Nations have concerns that ships in their waters could eavesdrop on communications from ships repairing cables.

• It can take weeks to be granted permission for these ships to be allowed to repair cuts in other countries’ waters.

The non-profit International Cable Protection Committee’s (ISCPC) goal is to prevent submarine cable cuts. To reach that goal it communicates the criticality of undersea cables internationally, distributes cable maps to marine organizations, and lobbies internationally to reduce nations’ reluctance to allow ships to repair damaged cables.

Bandwidth Capabilities in Carrier Networks

Carriers mainly use high-capacity 400 Gigabit Ethernet and Terabit (1,000 Gigabit) services to transport IP data, Ethernet, and VoIP traffic on long-haul networks. They additionally carry the decreasing amounts of legacy traffic from analog circuit-switched voice on these networks.

Carrier Gigabit Ethernet

Gigabit Ethernet is used in both Local Area Networks (LANs) and Wide Area Networks (WANs). It operates over fiber-optic cabling. When it is used in telecommunications and cable TV networks, it is sometimes referred to as Carrier Gigabit Ethernet. There are six capacities of Gigabit Ethernet (also referred to as GigE): 1, 2.5, 10, 100, 400, and terabit. These are used in core, Metropolitan Area Networks (MANs), and access networks. Carrier Gigabit Ethernet is used for VoIP and all IP-packetized traffic. This includes video, data, and graphics traffic that is bundled in packets.

A timing source is included in GigE to make it suitable for switched voice, because it does not natively have timing sources. A timing source provides fixed, guaranteed capacity that circuit-switched voice requires. Without it, circuit-switched voice sounds choppy because of brief, intermittent delays, which are otherwise imperceptible in IP data. An International Telecommunications Union (ITU) 2010 standard called Synchronous Ethernet was developed for chips with timing sources. Timing sources are now standard in most Synchronous GigE switches.

The Drive for Higher-Capacity Carrier Gigabit Ethernet

Large providers such as AT&T and Verizon support 400Gbps capacity in their backbones. The impetus behind 400Gbps Ethernet is the increasing amount of high-definition streaming TV, personal videos uploaded to social networks, network-based storage, cloud based data centers, and mobile broadband backhaul traffic. Backhaul networks “backhaul” traffic from cellular antennas to mobile companies’ data centers. Gigabit Ethernet was expressly developed for IP traffic and is less costly to purchase, maintain, and install than SONET.

The wide availability of Gigabit Ethernet switches from many manufacturers has caused Ethernet switch prices to drop. Moreover, because of its standardization by the Institute of Electrical and Electronics Engineers (IEEE), vendors produce compatible Ethernet gear so that providers are not locked into a single vendor’s equipment.

Gigabit Ethernet and Ethernet are often used in conjunction with Passive Optical Networks (PONs), discussed later in this chapter. PONs are a lower-cost method of extending fiber to premises and neighborhoods because they enable single strands of fiber to be shared between multiple homes and small businesses.

Circuit-Switched Voice—Unsuitable for Packetized Traffic

After 2012, carriers began carrying most traffic over Gigabit Ethernet both because of its suitability for data and because of decreases in the amount of analog voice and TDM traffic. Prior to 2012, SONET, a North American standard for multiplexing streams of traffic onto fiber-optic cabling was the predominant protocol for transporting traffic at Optical Carrier (OC) speeds. SONET was developed to aggregate (multiplex) and carry TDM (Time Division Multiplexed) and circuit-switched voice traffic from multiple sources. TDM multiplexing saves capacity for voice and data in predictable time slots. However, its top speed of 40Gbps is inadequate for today’s long haul traffic. See Table 4-2 in the “Appendix” section at the end of this chapter for SONET speeds.

Using Ring Topology—Greater Reliability and Cost

Carrier Gigabit Ethernet can operate as a straight point-to-point line between sites, in the more fail-safe ring, or in mesh topologies.

When fiber in a point-to-point arrangement is cut, service is lost. Reliability on fiber is critical because each failure affects potentially hundreds or thousands of customers, particularly if a failure occurs in long-haul networks.

When a medium such as copper carries a conversation or data stream from one telephone subscriber or computer to another, a failure only impacts one customer. Because of the large volumes transmitted by fiber, failures in these networks can put hundreds of businesses, police stations, or hospitals out of service. For this reason, the majority of carriers deploy bidirectional ring topology in long-haul and Metropolitan Area Networks (MANs), where each fiber ring, multiplexer, and power supply is duplicated.

Ring topology is costly because the fiber and the multiplexers are all duplicated, even though this combined capacity is not used on a day-to-day basis when there are no failures. The spare fiber in ring topology is known as the protect ring. It reroutes traffic in the other direction, as is illustrated in Figure 4-5. Read the section “Mesh Configuration Backups,” later in this chapter, to learn about new, lower-cost mesh technologies that are being deployed.

Optical Transport Networks—Carrying Multiple Types of Traffic

Optical Transport Network (OTN) is an International Telecommunications Union (ITU) G.709 standard for transmitting, monitoring, and managing traffic on fiber-optic cabling. OTN is an OSI (Open Systems Interconnections) Layer 1 service. As described in Chapter 1, “Computing and Enabling Technologies,” Layer 1 is the physical and wireless medium over which traffic travels. OTN is used for asynchronous protocols such as Gigabit Ethernet as well as synchronous protocols such as Synchronous Optical Network (SONET). In asynchronous protocols, traffic is sent at irregular intervals, not at specific intervals. This contrasts with synchronous protocols where bits are sent at evenly spaced, regular intervals according to an internal timing source. The OTN standard provides a framework for transmitting both types of traffic.

In a telephone interview with Dave Parks, former director, Segment Marketing at Ciena Corporation, Parks stated:

OTN can be thought of as the freight containers that carry Ethernet, IP, and SONET optical traffic together.

The OTN standard was created for the efficient operation of providers’ multi-protocol metropolitan and global optical networks using interoperable equipment. The standard provides a framework for programming, monitoring, and transmitting the growing amounts of IP packet, Gigabit Ethernet, video, and MPLS (Multi-Protocol Label Service) traffic while preserving its capability to carry legacy traffic on optical networks. All of the services—1 and 10 Gigabit Ethernet, SONET, and video can be on the same line card and are software programmable.

It does so by specifying the encapsulation of legacy SONET/Synchronous Digital Hierarchy (SDH) traffic into OTN frames and its overhead information for addressing and billing on fiber-optic cabling. OTN, which can scale to 400Gbps and Terabit capacities, overcomes SONET’s 40 Gigabit per second capacity. See Table 4-2 in the “Appendix” for information on SONET speeds.

OTN equipment has optional modules for connecting to Dense Wavelength Division Multiplexing (DWDM) as well as Reconfigurable Optical Add/Drop Multiplexing (ROADM) so that individual wavelengths can be seamlessly added and dropped off at, for example, carriers’ POPs. Both SONET/SDH and IP traffic can be carried on the same wavelength. (DWDM gear splits each pair of fiber cabling into multiple paths called wavelengths.) Manufacturers that offer OTN gear include Calix, Ciena, Cisco, Fujitsu, Infinera, and Nokia.

Today’s networks are required to carry a mix of the more efficient gigabit Ethernet as well as traffic using the older SONET and MPLS protocols. Optical Transport Network is an important protocol for transporting both digital Internet Protocol (IP) signals as well older synchronous traffic in a single stream. Gigabit Ethernet supports the massive increases in traffic due mainly to traffic generated by enterprises and consumers accessing the cloud, by consumers streaming TV and movies from Netflix, Amazon, and their competitors, and from applications such as online gaming.

The newer, more efficient Gigabit Ethernet is able to carry many times more bits per second than SONET and MPLS. Gigabit Ethernet is available at data rates of 10, 40, 100, and 400 bits per second. While the amount of traffic generated using Gigabit Ethernet is increasing rapidly, there is still traffic in the SONET and MPLS format. However, the amount of traffic generated in particular by SONET gear is decreasing rapidly. At some point OTN use will not be needed as SONET and MPLS traffic drastically decreases.

Optical Transport Networks and SDN and NFV

Software Defined Networks and Network Function Virtualization traffic are carried over fiber equipped with OTN functionality. SDN and NFV are higher-layer protocols than OTN. SDN’s control plane prioritizes and specifies routes on which traffic is to be carried. NFV, as stated above, virtualizes network hardware. SDN and NFV are functionally different than OTN, which manages how traffic is “packaged” or “wrapped” and carried on optical fiber.

Examples of control plane technology functions are available above in the section on SDN.

Mesh Configuration Backups

Mesh backup technology is enabled by signals generated from Control Plane Technology equipment such as is in WA SDNs. The service utilizes a logical rather than a physical design. The physical topology is not designed as an actual mesh, wherein each point on a network is connected to every other node. Rather, as demonstrated in Figure 4-6 above, traffic is backed up in a logical manner with more than one choice of alternate routes. In this way, each carrier and each route do not require duplicated fiber, multiplexing gear, and transmission equipment.

Mesh technology assumes adequate capacity on backup routes. If this capacity is not sufficient, delays will occur. It’s a trade-off between flexibility in having multiple backups, idle capacity when the network is operational, cost savings, and possible congestion during emergencies.

ROADMs Interoperability Plus Adding and Dropping Off Traffic

Reconfigurable Optical Add/Drop Multiplexers (ROADMs) enable carriers to:

• Add traffic

• Separate out traffic

• Drop off traffic

ROADM equipment eliminates the extra cost and maintenance for conversion equipment. Conversion equipment converts signals to electrical signals and back to light signals. Prior to the availability of ROADMs, light signals had to be converted to electrical signals before they could be added or dropped off and routed elsewhere.

ROADMs were first deployed in core networks so that, for example, some of the traffic between cities such as New York and Chicago could be more easily routed to Los Angeles. As traffic within metropolitan areas increased, ROADMs began to be used in these areas as well. Dense Wavelength Division Multiplexers (DWDMs) and OTN equipment can be equipped with ROADM cards so fewer pieces of gear need to be maintained.

ROADMs are additionally used to connect long-haul carriers to carriers in metropolitan areas. For example, ROADMs enable AT&T to drop local traffic off to local providers such as Verizon or Comcast. To transmit traffic between different telephone companies, often technicians from each company call each other to make sure their ROADMs can interoperate. The technicians often determine a way to connect via compatible Application Programming Interfaces (APIs). This causes delays in setting up connections.

The Open ROADM Multi-Source Agreement (MSA), which is in tests and limited availability at providers and manufacturers, is a solution to ROADM interoperability. With ROADM MSA, carriers can add and drop single wavelengths to ROADMs manufactured by other vendors. See Figure 4-6 for an example of an open source ROADM. AT&T announced they were able to add and drop single 100 Gigabit wavelengths between Ciena and Fujitsu ROADMs. Legacy ROADMs may not be upgradeable due to older, fixed optical parts that are not programmable for compatibility to the Open ROADM MSA.

Transporting Movies and TV in the Core

Both telephone service providers and cable TV companies have infrastructure in their core networks to receive and transmit entertainment content to customers. They each do this from centralized headends.

Using Headends to Receive and Transmit Programming

The term “headend” refers to the site where providers transmit content that they receive from satellites. A group of satellite dishes that receives content is referred to as a satellite farm. Central office switches and Voice over Internet Protocol (VoIP) equipment to support telephone services are also located at headends. Network Operations Centers (NOCs) capable of monitoring and making programming changes to the network as well as operations support systems can be located here as well. These centers typically serve 7 to 10 towns. Multiple hubs that distribute content are connected to the headend, as shown in Figure 4-7.

Hub Sites

Hubs, also referred to as distribution hubs, are small buildings that are located closer to customers than headends. Local programming or frequently downloaded content might be located at a hub, which serves between 10,000 and 50,000 residences. In addition, cable modem termination systems, which manage traffic to traditional cable TV customers’ modems and to the headend, are located at hubs. For more information, see the section “Using Cable Modem Termination Systems for IP Traffic” later in this chapter. A hub might serve a metropolitan area. Large towns would have two hubs.

Linking Hubs and Headends via Metropolitan Area Networks

Metropolitan Area Network (MANs) are also referred to as transport networks, or regional transport networks. They link hubs to headends. MANs typically operate over shorter distances than core, backbone networks. They carry a mix of voice, data, and video on demand, as well as television signals. Cable TV Metropolitan networks are made up of hybrid fiber/coaxial cabling with fiber for the core that transmits movies to neighborhoods, and parts of their networks in cities far from the headend. These cities have nodes with wiring center (transmission) equipment.

Older MANs transmit traffic to headends’ SONET rings. Cable operators and traditional telephone companies are transitioning parts of their metropolitan networks to all-IP, Gigabit Ethernet. It is complex and costly to upgrade all of a carrier’s MANs simultaneously. However, carriers have the option of gradually upgrading a few networks at a time. These upgrades don’t involve the immense upgrades as those in core networks. MANs are equipped with either redundant fiber rings or the simpler Optical Transport Network (OTN) option as backup technologies in the event of a fiber cut or equipment failure.

Middle-Mile Networks

Middle-mile networks are the sections of networks between the access provider and the connections that carry traffic to the Internet and to national and international core networks. Figure 4-8 shows an example of connections on middle-mile networks. Middle-mile networks can be the source of congestion on both wireless and wireline networks because they transport increasing amounts of Internet, smartphone, and video traffic. This is straining existing infrastructure.

Middle-mile networks include the fiber or wireless connections between switches in small cities and the carrier’s equipment that transmits traffic to a major carrier’s Point of Presence (POP). Within middle-mile networks, the POP serves as the location where a provider’s switches and connections for multiple carriers connect to the Internet or to a national or undersea cable. The core switches of large providers are also located at POPs.

Independently operated data centers where data is managed for multiple enterprises are another example of sites that require middle-mile connections. These data centers generate large amounts of traffic for which they require high-speed connections on middle-mile networks to backbone POPs.

Large carriers such as Verizon and AT&T in the United States carry the majority of middle-mile traffic. Level 3 Communications (now part of CenturyLink) is an alternative option in some areas. Level 3 offers to connect rural areas to POPs using spare capacity in its regional and long-haul networks. In the newer networks of these providers, hubs with Gigabit Ethernet switches are able to add and drop off traffic. Connections to Gigabit Ethernet are less costly than those to older SONET facilities. However, they require costly fiber cabling or wireless infrastructure from the rural provider’s first-mile network to the middle-mile network switch. Because of the low population densities in rural areas that do not generate adequate revenue, this is not always financially feasible.

Last-Mile Access Networks

Last-mile networks are the portions of a carrier’s network that connect subscribers directly to the provider’s equipment. The large number of connections usually means that the last-mile networks are the last portions of a carrier’s network that benefit from upgrades. The terms “access networks,” “last-mile networks,” and “first-mile networks” are used interchangeably.

Adding Capacity to Access Networks

Upgrading access networks involves numerous devices because service to each telephone company and cable TV customer as well as equipment in neighborhood locations must often be changed. This is a challenge for both traditional telephone companies and cable TV Multiple System Operators (MSOs). However, cable TV providers have newer voice infrastructure than traditional telephone companies because the cable TV providers only started delivering these services around 2004. Thus, their voice switches don’t require upgrades at this time. However, upgrading cabling and electronics connected to them for higher capacity is costly and complex.

One driving force behind upgraded access networks is the need to support more channels of high-definition television (HDTV) and video on demand (VOD), so that individuals are able to select from a menu of movies and premium television. In addition, customers are requesting HDTV, which requires more bandwidth and complex electronics than standard definition.

Data traffic is also driving networks to upgrade compression on set-top boxes. Consumers are additionally consuming more bandwidth by streaming video from Netflix and Amazon, and uploading and downloading both short and long form videos to and from YouTube, Instagram and Facebook.

Improved compression and other techniques such as rate shaping provide the capability to carry additional video and broadband traffic. Rate shaping adds capacity by using a form of over-subscription. It’s analogous to pouring one-and-a-quarter quarts of water into a pail that holds only a single quart. Rate shaping equipment discards bits in real time so that adding 8 megabits of traffic to a 6-megabit channel does not impact quality. This assumes that not everyone will be watching TV at the same time.

Legacy Circuit-Switching Service

Switched services operate on landline and cellular networks that use older mobile protocols. When a person makes a call, the network sets up a path between the caller and the dialed party. Importantly, the path is available exclusively for the duration of the call. The path is not shared. Natural pauses in conversation and data transmission are not used for other voice or data calls. Capacity is reserved in the network for the entire duration of the transmission. This exclusivity causes wasteful utilization of network capacity.

Circuit switching is being gradually phased out in public networks. Carriers increasingly add IP equipment when replacing fully depreciated, older voice switches or when building new infrastructure.

Last-mile access networks use a mix of primarily IP and some circuit switching to carry voice. Traditional telephone companies and cable TV providers are in the process of transitioning to IP in their last-mile networks. When customers select their telephone provider for Internet access and TV services, the provider offers to move the subscriber’s voice service to the same broadband infrastructure as that used for data and TV. Most of them provide VoIP over the broadband.

Transitioning Customers to Voice over Internet Protocol and Fiber

Data and video are the largest source of traffic on landline networks, far exceeding the shrinking amounts of voice. Many providers now transmit voice in packets, the same format as data, as a way to use bandwidth efficiently. Providers are additionally adding capacity to last-mile, edge networks by laying fiber in neighborhoods closer to homes or by connecting fiber directly to individual homes.

Digital Subscriber Line Access Multiplexers

Digital Subscriber Line (DSL) is a service that sends data over the same twisted pair cabling used for voice. DSL is gradually being replaced by fiber to premises, but is still used by residential subscribers and small businesses that don’t have fiber. Telephone companies that have not invested in fiber to the premises (FTTP) offer DSL service for Internet access and other data services. It is also used where customers don’t have access to cable modem service. (See below for a description of cable TV networks.)

Using Small DSLAMs and Mini-RAMs for Internet Access with Digital Subscriber Line Service

DSL Access Multiplexers (DSLAMs) aggregate traffic from multiple customers’ DSL modems and translate electrical signals on copper cabling to light signals before sending traffic to the central office. From the central office Internet Service Providers (ISPs) transmit subscribers’ traffic to the Internet or data networks over fiber-optic cabling. DSLAMs are located in a carrier’s central office or in remote terminals, which are cabinets placed between the central office and the subscriber. The connection between the DSLAM and an ISP is a potential site for network congestion and delays, if capacity is insufficient between the DSLAM and the central office.

When telecoms move fiber closer to customers in neighborhoods, they often use mini Remote Access Multiplexers (mini-RAMs), which are about the size of pizza boxes, to convert light signals carried on fiber to electrical signals and vice versa. Installing fiber closer to homes results in shorter copper lengths. Shorter copper runs have fewer impairments because electrical signals fade less as they travel over shorter distances. Impairments found on longer copper runs farther from central offices cause DSL speeds to decrease dramatically and often prevent DSL from being viable at even slower speeds. Traffic from mini-RAMs is aggregated in DSLAMs, from where it is transmitted over fiber to central offices and the Internet.

Mini-RAMs can be located on utility poles or in stand-alone boxes on the ground and serve 10 to 24 customers (see Figure 4-9). Local power is not required because it is fed to mini-RAMs through the copper telephone lines either on the pole or underground.

The Case for Fiber to the Home—More Capacity and Less Maintenance

Cable TV providers and telephone companies in the United States and other countries sell “triple play” services: voice, TV and movies, and Internet access to enterprises, hotels, restaurants, and municipalities as well as residential customers. One way they provide capacity needed to handle all of these services is to add fiber closer to homes and businesses or directly to premises. When carriers transition to fiber to homes, voice signals are sent in digital, voice over Internet (VoIP) packets, rather than as electrical signals on copper cabling. Providers that transition to fiber to homes or even fiber close to homes in neighborhoods, gain these advantages:

• Fiber cabling has more capacity than copper to support the continuing growth in broadband traffic and cloud and streaming media.

• Operating costs are lower because it is more efficient for providers to maintain just one type of cabling and equipment than two types of cabling.

• Central offices only need to be equipped with equipment that supports a single type of cabling rather than both fiber and copper.

• Less network equipment is required because signals on fiber travel further than those on copper cabling before deteriorating. When signals fade, equipment is required to boost signals.

Battery Backup for Homes Connected to Fiber

The Federal Communications Commission (FCC) requires providers that transition to fiber from copper cabling in neighborhoods to offer in-home battery backup gear. This requirement is meant to ensure that residents such as the disabled or elderly are able to reach 911, police and fire departments, and healthcare facilities during power outages. An example of battery backup equipment for residential customers is containers with 12 D-Cell alkaline batteries. The battery backup devices are connected to optical network terminals. The backup generally provides eight hours of talk time during power outages. The customer is responsible for changing batteries about once a year when they wear down and lose charging ability.

The Economics of Fiber to Large Apartment Buildings and Enterprises

In contrast to the last-mile networks in residential areas, many of which are made up of twisted-pair copper or hybrid fiber/coaxial, cable TV providers and traditional telephone companies lay fiber to their large business customers. This is because the expense of supplying fiber cable to office and apartment buildings with multiple tenants can be spread across many customers. In addition, fiber is required to handle the large amounts of data traffic that enterprise customers generate.

Passive Optical Networks

Telephone companies and cable operators install Passive Optical Networks (PONs) to lower the cost of deploying fiber-optic cabling directly to residential and small business customers. Cost savings are achieved by sharing the capacity of single strands of fiber among multiple homes and small businesses.

Sharing Fiber Capacity—PON Architecture

PON technologies are a less costly way to deploy fiber because the devices, splitters, in the neighborhoods are small and passive. They don’t require electricity and can be mounted directly on utility poles or in existing equipment cabinets. Fewer cable runs are required from the central office to each neighborhood because the splitter divides the capacity on a single fiber strand among multiple subscribers.

PONs are used extensively in mobile networks as well where they transmit signals between antennas and carriers’ data centers. Data centers in mobile networks are where mobile providers manage their wireless networks, collect usage data for billing, and offer special applications such as voice mail and text messaging.

With PON technology, only one pair of fiber is brought to the neighborhood but multiple pairs are split out from PON interfaces to homes. Splitters divide the capacity of the fiber among up to 32 homes and small businesses. Figure 4-10 shows one possible scenario. Splitters are passive in that they don’t require electricity and they are small; they’re about the size of a smartphone. These are key factors because space in the network is at a premium and electrical costs are soaring. PON equipment at the central office, inside customers’ premises, and in cabinets in neighborhoods do require electricity.

PON Standards—Gigabit Ethernet

In contrast to residential PON services, which have higher capacity for downloads, PON services for business customers have synchronous capacity; equal bandwidth to and from their devices. Business and commercial organizations upload and download applications and data to and from the cloud and the Internet in equal amounts. However, 10Gb XG PONs support 10 Gigabit service downstream and 2.5 Gigabit service upstream. Current standards provide the option of symmetric services directly to business customers as well as the option to virtualize PON gear in CORD-equipped central offices.

See Table 4-1 for ITU-T (International Telecommunications Union Terminals for Telematic Services) PON standards.

Table 4-1 PON Standards

In the newest PON standard, Next Generation Passive Optical Network 2 (NG PON2), wavelengths can be turned on remotely and the lasers connected to the fiber can be modified so that the provider is not required to pay for a “truck roll” to send out a technician to make a change at the splitter. According to PON provider Adtran, the ability to add capacity by programming future-proofs NG PON2 against expected increases in traffic, particularly video streaming and cloud computing. As residents’ and businesses’ broadband use increases, new hardware is not required. Additional capacity for individual wavelengths can be added via software.

PON service was initially introduced in North America in 2002. Currently, Adtran, Nokia (through its purchase of Alcatel Lucent), Tellabs, and the Chinese company Huawei manufacture PON components available in North America. Older PON standards are listed in Table 4-3 in the “Appendix.”

PON Components—OLTs and ONTs

OLTs (Optical Line Terminals) are switches with ports on each card. They are located in central offices where they are connected to network nodes in the outside network of cable TV and telephone companies. See Figure 4-11 for an example of an OLT. OLTs place optical signals on the correct fiber optic cable. Each OLT card has from four to eight ports and each port is connected to a specific fiber strand.

OLTs for NG PON2s can also be installed in network operators’ cloud-based CORD central offices where they are virtual network functions (VNFs) in a White Box on X86 Intel commodity servers. Representing NG PON2s optical line terminals in software makes them less costly to install than separate pieces of hardware.

Optical Network Terminals (ONTs) translate between signals on fiber cabling and those on copper cabling as well as between optical signals and wireless signals. ONTs are located wherever fiber optic cabling is connected to other types of cabling and wireless medium where signals need to be translated. This makes them compatible with all types of media.

The following are examples of locations with ONTs:

• Homes

• Enterprises distribution frames where copper cabling is terminated

• Apartment buildings

• Wi-Fi modems if all signals within a building are transmitted on Wi-Fi

• Hospitals

• Cellular backhaul where signals are “backhauled” from cellular data centers to networks

• Wholesale cellular backhaul providers who resell backhaul capacity to facilities-based cellular companies

• Coaxial cabling in cable TV networks where fiber is installed in residential neighborhoods closer to homes

• Nodes in neighborhoods where fiber is terminated closer to, but not directly connected to a residence or small business

Direct Fiber to Enterprises and Multi-Tenant Buildings

Direct fiber is often installed at enterprises and multi-tenant buildings. Because it is directly connected from the central office, PON equipment is not needed. In apartment buildings, data-grade copper carries signals to each apartment unit from the ONT. The ONT converts light signals to electrical signals in the building’s wiring center. The data grade twisted-pair copper cabling carries Internet and other broadband signals to each apartment.

Using PONs to Deliver Fiber to the Home in Cities and Sections of Metropolitan Areas

PONs to the home are widely deployed in Asian countries such as Japan and South Korea, and in China, where much of the infrastructure was upgraded or built from scratch. Dense populations are a factor in making it feasible to lay fiber to homes in these countries. The lack of density in the United States compared to other countries makes it costly and challenging to build fiber to homes. Fiber to the home is capable of reaching up to 32 homes from a single splitter. Figure 4-12 presents a list of countries with the most fiber to the home (FTTH).

Verizon currently has the largest installed base of PON connected fiber. AT&T, CenturyLink, and most cable Multiple System Operators (MSOs) bring fiber deep into neighborhoods and then use twisted-pair or coaxial cabling to serve customers. Some providers bring fiber all the way to curbside pedestals in much of their territory. These implementations serve only 8 to 12 customers per PON device. PON service closer to customers is more expensive per subscriber to deploy but provides higher-speed service because fewer customers share the fiber.

PONs are also used to bring fiber all the way to premises in new “greenfield” developments (new industrial or residential developments). In addition, they are the basis for wireline infrastructure in developing countries. In addition to providing large amounts of capacity, fiber is less costly to maintain than copper cabling. Less equipment is needed to boost signals that fade, there are fewer repair issues, and capacity can be upgraded by changing equipment connected to fiber. Entire new fiber strands are not usually required.

Access Networks in Cable Operators’ Networks

Access networks in cable operators’ and traditional telephone companies’ networks use a different type of customer modem. However, both telephone companies and cable Multiple System Operators (MSOs) face the challenge of upgrading access networks to increase capacity. Access networks are those portions of a network between a provider’s equipment in neighborhoods and that of the customer. Per Neilson, cable and satellite TV reception is available to 82 percent of the population in the United States.

Using Cable Modems to Access the Internet

The basic functions of cable modems are to convert digital signals from computers or data networks to those compatible with coaxial cabling and to convert radio frequency (RF) from cable TV networks to digital signals suitable for Internet access and fiber optical cabling. In cable TV, electromagnetic waves carried on coaxial cabling are referred to as RF signals. The cable TV access network is a hybrid fiber/coax (HFC) network.

The Cable Modem “Handshake”

Cable modems perform a “handshake,” an exchange of signals before data is transmitted to and from the Cable Modem Termination Systems (CMTSs) located at the cable operator site. (See the following section for information regarding CMTS equipment.) Complex signaling, use of frequencies, the speed at which to transmit, and authentication are agreed on between the two devices. At startup, the network checks the user’s login before allowing access to the Web or certain web sites.

Using Set-Top Boxes to Interface to Cable TV

Cable TV set-top boxes are interfaces between televisions, satellite TV, subscription television, and cable TV networks for access to television and other services. At the most basic level, they are tuners that filter out all of the channels except the one selected by the viewer. Because each tuner filters only one channel at a time, set-top boxes used for more than one television simultaneously will require multiple tuners. Personal digital video recorders such as TiVo also have two tuners so that one tuner filters a channel that is being recorded and the other tuner filters the channel currently being viewed. Other tuners in end users’ televisions filter signals for interactive services such as programming guides, VOD, and pay-per-view.

Cable operators remotely administer filters and traps in set-top boxes to allow subscribers access to basic cable TV or premium channels. Set-top boxes also have advanced security functions and contain links to billing systems. The security function scrambles and unscrambles TV signals based on the information provided by the billing system as to which channels the subscriber is permitted to receive. Security is higher and TV theft is lower on digital service because of improved scrambling (encryption).

Set-top boxes are becoming more sophisticated. Comcast’s X-1 set-top box supports speech recognition on remotes, and access to services including Netflix, Pandora music, photos, and weather. The X-1 is required for the MPEG-4 compression that Comcast is upgrading to on their cable networks. MPEG-4 compresses video signals so that they require less capacity on networks. Software in set-top boxes decompresses the compressed signals so that video resolution is acceptable. See Chapter 1 for information on compression.

The High Capital Cost of Set-Top Box Upgrades

On a cost-per-subscriber basis, when cable TV networks are upgraded, the new set-top boxes required cost more than any other part of the network. This is because upgrades to the headend or core network are amortized over the entire network. By contrast, each set-top box is amortized from payments received from the individual customer. New set-top boxes add up to an enormous amount of money for cable providers with millions of customers. In addition, set-top box upgrades require more administrative time for programming the set-top boxes and answering customer inquiries.

These costs are a major factor in many providers’ decision to gradually implement upgrades in their coverage areas. This spreads out the costs over a few years rather than during a single year.

Soft Set-Top Boxes

One way that cable TV providers are saving money is by eliminating set-top boxes on certain Internet-connected televisions. On Samsung, for example, Comcast offers an app with full access to Comcast’s program options. Comcast’s app is also available on Roku devices. RCN, a regional cable TV provider, offers its program guide on other set-top boxes as well.

Using Cable Modem Termination Systems for IP Traffic

Cable Modem Termination Systems (CMTSs) are located at the cable or distribution center (the hub). They modulate and demodulate (hence, the term “modem”) digital voice and data signals and place them on cable infrastructure. Modulation is the technique of making digital signals suitable for radio frequencies (electromagnetic waves) that carry signals on cabling infrastructure. CMTSs demodulate signals received from customers to make them suitable for transmission on a cable company’s fiber-optic rings, which connect smaller hubs to headend facilities.

CMTSs monitor the level of traffic at each fiber node so that cable providers are aware when congestion occurs and nodes need to be added to serve fewer homes per node. Moreover, they are responsible for encryption to ensure privacy, security, and conditional access. Conditional access is the determination of whether a customer is entitled to certain features. CMTS devices have built-in routers that send traffic to different destinations such as the Internet, long-distance providers, or the cable MSO’s VoIP equipment.

CMTSs are similar in function to Digital Subscriber Line Access Multiplexers (DSLAMs) that are used in traditional telephone company networks in that both devices translate between modems at customers’ premises and equipment in backbone networks. They both also aggregate traffic from multiple subscribers into a single stream for transmission in the backbone. (See the section “Digital Subscriber Line Access Multiplexers,” earlier in this chapter, for a discussion on DSLAMs.)

Supporting More Video via Set-Top Boxes

New modulation technologies enable digital cable to increase capacity two to three times from the former 10 to 12 channels of standard-definition video and two to four channels of HDTV video on each 6MHz channel. Standard-definition TV has lower resolution than high-definition television. Moreover, digital cable results in improved resolution because there is less interference from noise, which creates snow and shadows that appear on the television screen.

Consumers can opt for set-top boxes with hard drives capable of storing content and playing it back at their convenience. Digital cable is the basis for VOD, which allows subscribers to order movies for an extra fee. VOD movies are available for a limited time period of a day or two. Nearly 100 percent of all cable TV subscribers in the United States are located in areas where they can receive digital cable. Pay-TV via cable and satellite are available to 82 percent of households of the population in the United States.

Cable Modem Standards Transition to Higher Speeds

CableLabs, the research and development arm of the North and South American cable TV industry, sets cable TV modem standards. CableLab’s standards are intended to provide a technology “road map” for cable MSOs to move toward implementing higher-capacity IP networks used for Internet access. These standards are known as the Data Over Cable System Interface Specifications (DOCSIS). The European Telecommunications Standards Institute (ETSI) and the International Telecommunications Union (ITU) have approved the standards listed in Table 4-4 in the “Appendix” section of this chapter. However, the international versions of DOCSIS vary a little from that used in the United States. The DOCSIS standard in Europe and parts of Latin America is referred to as EuroDOCSIS.

Transitioning from Asymmetric to Symmetric Channels

Until 2016, MSO offerings for residential-consumer Internet access were primarily asymmetric, with higher speeds in the downstream channel from the cable provider to the subscriber, and lower speeds upstream from the subscriber to the provider. Splitting the frequencies into different ranges enables the same coaxial cables to be used for both sending and receiving signals. Unlike fiber cabling, for which separate strands of fiber are used for sending and receiving, a separate coaxial cable does not need to be installed for the reverse, upstream channel. With the advent of new cable modem standards (see DOCSIS below), both upstream and downstream capacities are the same. They are symmetric. Upstream and downstream bits use the entire range of frequencies in the cable.

Full Duplex DOCSIS 3.1—Symmetric Speeds

The DOCSIS 3.1 standard specifies support for 10Gbps per 6MHz (6 megahertz) channel. Users on the same node share this capacity. DOCSIS 3.1 is currently the fastest, most advanced cable modem standard. It is compatible with the previous DOCSIS 3.0 that specified bonding to increase capacity on broadband.

Bonding enables support for higher data rates of 173Mbps to 343Mbps downstream to homes, and 123Mbps upstream to the cable operator.

Fiber-optic cabling supports more capacity than current hybrid fiber/coaxial cabling. For example, Verizon’s fiber to the premises (FTTP) cabling enables it to use its entire 860MHz for TV, with additional gigabits available for voice and Internet access. To add capacity on coaxial cable, operators are upgrading their networks using advanced QAM multiplexing techniques.

Because of its support for Quality of Service (QoS), DOCSIS 3.0 supported applications such as voice and video, which are sensitive to delays. An upgrade to DOCSIS 3.0 required a new DOCSIS 3.0–compatible cable modem at the subscriber location and at the CMTS. This made the upgrade a costly endeavor.

Competition from carriers that offer fiber to the premises (FTTP) is one factor that spurred implementation of DOCSIS 3.0 and full duplex, sending and receiving on the same cable or channel. DOCSIS 3.1’s higher-capacity offerings have capacities from 1 to 10Gbps to support broadband to residential and business customers. MSOs deploy direct fiber to enterprise customers. Unlike offering to residential subscribers, cable TV direct fiber connections to enterprises are not shared with other customers that subscribe to broadband. They are connected directly to cable operators’ transport networks.

Carrying More Data—Quadrature Amplitude Modulation (QAM)

QAM is a modulation technology specified in DOCSIS standards that increase broadband capacity on hybrid fiber cable (HFC) networks. Modulation standards define how signals are placed on networks and how they are transmitted. Full Duplex 4096 QAM enables cable TV networks to carry two to three times as many high definition (HD) movies as the earlier 256 QAM. However, movies must be transmitted using the IP (Internet Protocol). DOCSIS 3.1 specifies 4096 QAM to deliver 10 Gigabits of symmetric capacity shared by nodes within neighborhoods. Currently most cable operators don’t transmit movies using the IP (Internet Protocol).

QAM 4096 is more sensitive to noise and impairments that occur over long distances on coaxial cabling. This is one reason cable TV providers are expanding their fiber footprint closer into neighborhoods. Fiber closer to homes results in shortened coaxial cable lengths. Shorter coaxial cabling runs have less impairment from electrical noise than longer coaxial cable runs.

There is also a DOCSIS 3.1 specification for half duplex, asymmetric service. Asymmetric DOCSIS 3.1 specifies 10 Gigabits downstream and 1 Gigabit upstream from customers to the provider’s headend. In these instances, DOCSIS 3.0 spectrum is used for the upstream bits toward the cable TV network and DOCSIS 3.1 spectrum for downstream spectrum to subscribers.

Upgrading to 10GHz adds capacity for Internet browsing as well as movies sent using IP. Unlike bonding, it does not decrease the amount of capacity for video. However, it is costly. Optical transceivers and receivers as well as amplifiers and set-top boxes need to be replaced or upgraded, depending on the age of the existing cabling and electronics. Optical transceivers and receivers convert signals on optical cabling to those compatible with copper cabling, and vice versa. Amplifiers strengthen signals that naturally weaken after traveling over distances, particularly on copper-based coaxial.

Competition from Streaming TV and Traditional Telephone Companies

Customers that “cut the cord” or that never subscribed to cable TV or satellite TV can opt for over the air service television broadcasts beamed from towers. National broadcasters (ABC, NBC, CBS, Fox, and Spanish channels) and their local affiliates broadcast their programs over the air to customers’ antennas or from cable TV providers that now offer streaming. Subscribers without cable TV service receive programs over the air via an antenna wired to their TVs. The antenna can be installed on their roof or the outside of their home. Subscribers close enough to broadcasting towers are able to use indoor antennas to receive over the air TV broadcasts.

The escalating subscription fees combined with the many interruptions for commercials on cable TV has motivated many subscribers to “cut the cord” and rely only on over the top (OTT) streaming media from Netflix, Amazon, Hulu, and others. According to Nielson, by the first quarter of 2017, 123.6 million people in the United States subscribed to OTT streaming TV from Netflix, Amazon, Hulu, and others. Others subscribe to fewer cable TV stations. They “slim down” their cable TV bundle by subscribing to fewer cable TV stations. These slimmed down cable TV packages are referred to as skinny bundles. Many subscribers pay for an OTT subscription in addition to their slimmed down or full cable TV subscriptions.

Traditional telephone companies including AT&T and CenturyLink are encroaching on cable MSOs by offering subscriptions to their movie and TV programming, IP voice telephony, and broadband over their copper or fiber infrastructure. Verizon currently has the most territory in the United States where they offer service bundles over fiber. They carry IP voice, television, and Internet broadband transmissions over their fiber to homes. AT&T and most other telephone companies compete with MSOs by offering triple play services: voice, data, and TV programs. They build out fiber to the closest neighborhood node (equipment cabinet), and use copper for the last few hundred feet in areas where they don’t provide fiber directly to homes.

Satellite TV

Satellite TV operators offer bundles with both streaming TV and national broadcast stations. Dish and AT&T’s DirecTV are the only satellite TV providers in the United States. Satellite TV is particularly popular in rural areas where fiber to homes is not available. Satellite TV is important in rural areas in the United States, particularly where cable TV is not available because low population density makes it unprofitable to lay cabling directly to homes.

Telecommunications Services in National Emergencies

During natural disasters, extreme weather conditions, and even armed attacks, the government, military, first responders (hospitals, police, 911 call centers, and firefighters), and ordinary citizens depend on communications to coordinate rescue operations and to stay in contact with family members. Even in the absence of a disaster, the ability to call police, fire departments, and hospitals is critical so that people can report emergencies and assistance can be dispatched as quickly as possible. For these reasons, governments worldwide regulate network reliability and sustainability. Equipment and network sustainability refers to the capability to operate continuously during adverse conditions.

In the United States, the Federal Communications Commission (FCC) requires outage reports from the following providers:

• Wireline

• Wireless

• Paging

• Cable TV

• Satellite TV

• Signaling systems

They must further report disruptions that affect the following:

• E-911 facilities affecting 900,000 user-minutes or 30-minute outages in tandem 911 offices connected to long distance networks and other 911 facilities

• Major military installations

• Key government facilities

• Nuclear power plants

• Certain airports

In addition, fiber-optic outages of lines that carry traffic between backbone network switches must be reported. Reports on outages are reviewed quarterly by the Network Reliability Steering Committee (NRSC), which is made up of representatives from nationwide wireline and wireless providers. The committee looks for patterns by type of failure so that they can determine ways to prevent them.

Planning to Insure Reliability and Sustainability

Carrier networks are now largely IP-based, converged networks, wherein a single network infrastructure carries data as well as telephone traffic. Thus, outages cause disruptions to businesses, governments, and consumers. In addition, traffic volume increases significantly in emergencies.

During widespread emergencies, mobile carriers free up capacity for the additional volume required by assigning less frequency to each call. Thus, more calls can be carried, although at lower sound quality. They can also quickly set up portable cell sites and additional generators at towers or mobile switches that lose power. This enables mobile providers to restore service to these areas. In addition, if one part of the country is hit by a natural disaster, other carriers will send staff members and equipment to assist in restoring service.

Carriers plan for emergencies in various ways. For example, they build backup wireless routes between critical switches and public safety answering points (PSAPs) for E-911 calls. They also look at ways to increase security against hackers and terrorists at peering sites where carriers exchange IP traffic. In addition, providers consider methods to avoid user errors. User errors occur most often during network upgrades and programming changes.

Internet Security and Sustainability

Internet security is a global issue that cannot be isolated within a single country. However, the Federal Communications Commission (FCC) currently has no authority to require that Internet providers report network outages and details about cyber attacks. In its 2009 National Broadband Plan, the FCC recommended looking at ways to ensure sustainable communications over the Internet. It also stated the desirability of expanding international participation and outreach to manage cyber security information. Part and parcel of that, the FCC would like a vehicle for collecting more detailed information about cyber security. The report also expressed concern about how dependent first responders are on Internet communications. The FCC further declared that it is considering asking Congress for the authority to require reporting on these issues.

Because the Internet is classified as an information service rather than as a telecommunications service, the FCC has limited authority in regulating it. An information service is defined in the Telecommunications Act of 1996 as a service that adds value to a basic transmission path through functions such as processing information. Telecommunications is defined as the basic transmission (delivery) of voice and data. It’s essentially the path over which voice is carried as well as the equipment used to send and receive voice and data traffic.

Because of its critical role, the FCC attempted to alter the way the Internet is regulated so that the agency has more oversight of cyber security and sustainability. It attempted to have the Internet classified as a telecommunications entity rather than a value-added service. However, with the changeover of the FCC when Donald Trump was elected president, the FCC has dropped its efforts to classify the transmission component of Internet services as telecommunications. The effort to regulate the Internet did not include monitoring content on web sites or content transmitted to web sites.

Signaling

Signaling is the process of sending control information over landline and mobile networks to monitor, control, route, and set up sessions between devices. These sessions include video and audio conference calls, data sessions, video calls, and mobile and landline telephone calls. Signaling is also used to set up instant messaging and chat sessions. Signaling is used within public landline and, mobile networks, and the Internet as well as for intercarrier connections and billing.

An Overview of Signaling

Signaling is used to process every call on the Public Switched Telephone Network (PSTN), the Internet, and the public cellular network. When a caller dials a number, he or she can hear progress tones such as dial tone, ringing, busy signals, or reorder tones. These are all signaling tones. In addition to tones, callers might hear digital messages informing them that the number they dialed is not in service or has been changed.

The PSTN (Public Switched Telephone Network) uses Signaling System 7 (SS7); IP networks mainly use variations of Session Initiation Protocol (SIP) as the common platforms for call setup, teardown and activation, and control of advanced features. SIP was originally designed for the Internet, but is now used in private data networks and data networks that carry IP voice as well. A form of SIP is also used for text messages.

Signaling is the basis of interconnection between mobile, global wireless, and multiple providers’ networks. When AT&T controlled most of the public network in the United States, it had the necessary control of the public network that enabled it to set a standard (SS7) that was followed across the country and was later adopted, with variations, by the international community. SIP is additionally used in IP PBXs to carry calling identification, and to set up and tear down video and audio conferences.

SIP is an Internet Engineering Task Force (IETF)–approved standard. In both SIP and SS7 protocols, signaling messages are carried separately from user content. In addition, they both provide common functions.

Incompatibilities between Different SIP Implementations

SIP is an extremely flexible protocol that offers different ways to implement many IP applications. These differing methods are referred to as Requests for Comments (RFCs). RFCs were originally used only to solicit comments when standards are being developed. Once approved by standards bodies, the final versions of standards are then published as RFCs.

SIP has many RFCs, which has been a complicating factor when carriers connect with one another at peering sites and when carriers purchase new IP equipment from manufacturers that use different SIP implementations from the carrier’s. In these cases, manufacturers modify their equipment to match the RFC implemented by the carrier for its network. The SIP Forum, a non-profit organization that advances interoperability and hosts interoperability testing, has simplified internetworking between carriers as well as between carriers and enterprises. The forum recommends specific profiles (RFCs) to use when connecting enterprise SIP trunks for VoIP service. Most carriers now use these and other specified profiles.

Interconnecting Carriers and Providing Secure Space for Equipment in Co-Location Facilities

Co-location facilities are sites where network service providers house their switches and routers and connect to one another’s networks. Co-location facilities are also referred to as carrier hotels.

There are up to 200 carriers located at the very largest co-location sites. Co-location facilities are connected to the Internet, public and private data networks, and the PSTN. Because so many network providers depend on co-location facilities, security and sustainability in the face of natural and human-made disasters are critical. For these reasons, co-location facilities are located in secure buildings with few windows and no company logos to identify them. They are also monitored by security cameras and heavily guarded by security firms. They often additionally duplicate their service in a completely separate facility connected by fiber cabling to their main location.

Carriers rent space in co-location facilities because no provider has outside cabling everywhere. In addition, certain providers have no cabling facilities. Instead, they own switches, which they connect to other carriers’ networks. In both of these circumstances, rather than construct their own buildings to house their switches, carriers lease space in carrier hotels. They place their equipment in cages or lease space in equipment cabinets in the carrier hotel. Locked wire cages surround the equipment and access to the equipment is available only to employees of the company that owns the equipment. Carriers also remotely access their servers to monitor and program their equipment. For even more reliability, some co-location facilities place redundant equipment at an additional co-location site.

Leasing space in carrier hotels saves network providers the expense of establishing and maintaining the following:

• Physical security against break-ins

• Access to large amounts of power

• Access to backup power

• Backup generators

• Dual air conditioning systems

• Uninterrupted power supplies

• Backup fiber cabling to the facility

• Fire detection and fire suppression equipment

• Alarms to fire and police departments

• Staff members to plan and maintain the facilities

• Construction of earthquake-resistant facilities

Carriers such as CenturyLink, AT&T, and Verizon have carrier-owned co-location facilities. These are generally in the same building as the carrier’s Point of Presence (POP). Carriers offer a bundle of services at these facilities, including connections to routes connected to the co-location facility and connections to the carrier’s POP, which is often located in the same building as the co-location services. AT&T and Verizon buy services from each other’s co-location sites to reach customers whose locations are outside of areas in which they own access networks.

Another type of co-location facility is a carrier-neutral facility. Neutral co-location facilities are owned by non-carrier organizations, including Equinix, Markley Group, and Telehouse. Neutral refers to the fact that carriers that use these facilities can send traffic over routes offered by any carrier connected to the same facility. A carrier at a neutral site can lease routes from any other provider at the site, such as Windstream or CenturyLink. Carriers locate gateways, switches, and routers in these facilities to exchange traffic with other providers and to manage their own traffic.

Equinix provides data center services as well as co-location services. It is the largest provider of co-location and data center services worldwide.

Connecting Smaller Providers and Competitors to Customers

Telephone companies and Multiple System Operators (MSOs) rent various portions of their networks to smaller providers and to competitors that might have some of their own switches and/or long-haul networks, but no last-mile connections to customers. They offer access to their networks on a wholesale basis. For example, they offer connections between their POPs and their local central offices. These connections are considered transport networks. Smaller providers use these lines to transport traffic between their own switch (which might be located at the POP) and the local telephone company’s switch. They also lease last-mile cabling infrastructure to reach each of their customers.

Both large and small providers make these arrangements with one another and with large enterprises. For example, Verizon sells services to large enterprises that have offices throughout the United States and around the world. Often, these enterprises purchase nationwide networking services for all of their locations. Verizon rents transport services from providers such as AT&T in cities where it does not own the local cabling infrastructure. Providers make these types of arrangements with carriers in other countries as well. AT&T, in turn, leases these services for its own large customers’ remote offices that are located in areas where AT&T does not have its own last-mile network.

Appendix

Table 4-2 SONET: A North American Standard for Multiplexing Streams of Traffic onto Fiber-Optic Cabling and Transporting It at Optical Carrier (OC) Speeds*

*SONET was developed to aggregate (multiplex) and carry TDM (Time Division Multiplexed) and circuit-switched voice traffic from multiple sources. The international version of SONET is Synchronous Digital Hierarchy (SDH). SONET SDH carries traffic at Synchronous Transfer Mode (STM) rates.

Table 4-3 Older PON Standards

*Optical carrier

Table 4-4 DOCSIS Standards