Author Note
This appendix contains an entire chapter that was published as a chapter in one of the past editions of this book or a related book. The author includes this appendix with the current edition as extra reading for anyone interested in learning more. However, note that the content in this appendix has not been edited since it was published in the earlier edition, so references to exams and exam topics, and to other chapters, will be outdated. This appendix was previously published as Chapter 11 of the book CCNA Routing and Switching ICND2 200-105 Official Cert Guide, published in 2016.
To troubleshoot a possible IPv4 routing protocol problem, first focus on interfaces, and then on neighbors. The routing protocol configuration identifies the interfaces on which the router should use the routing protocol. After identifying those interfaces, a network engineer can look at the neighbors each router finds on each interface, searching for neighbors that should exist but do not.
This chapter focuses on issues related to these two main branches of logic: on which interfaces should a router enable the routing protocol, and which neighbor relationships should each router create. This chapter’s troubleshooting discussions emphasize how to find incorrect configuration problems by using only show and debug commands.
This chapter first briefly introduces a few broad concepts related to troubleshooting problems with routing protocols. The next major section examines problems related to which interfaces on which a router enables the routing protocol, with the final major section focusing of routing protocol neighbor relationships. Note that the entire chapter moves back and forth between discussing both Enhanced Interior Gateway Routing Protocol (EIGRP) and Open Shortest Path First Version 2 (OSPFv2).
Because a routing protocol’s job is to fill a router’s routing table with the currently best routes, it makes sense that troubleshooting potential problems with routing protocols could begin with the IP routing table. Given basic information about an internetwork, including the routers, their IP addresses and masks, and the routing protocol, you could calculate the subnet numbers that should be in the router’s routing table and list the likely next-hop routers for each route. For example, Figure Q-1 shows an internetwork with six subnets. Router R1’s routing table should list all six subnets, with three connected routes, two routes learned from R2 (172.16.4.0/24 and 172.16.5.0/24), and one route learned from R3 (172.16.6.0/24).
So, one possible troubleshooting process is to analyze the internetwork, look at the routing table, and look for missing routes. If one or more expected routes are missing, the next step would be to determine whether that router has learned any routes from the expected next-hop (neighbor) router. The next steps to isolate the problem differ greatly if a router is having problems forming a neighbor relationship with another router, versus having a working neighbor relationship but not being able to learn all routes.
For example, suppose that R1 in Figure Q-1 has learned a route for subnet 172.16.4.0/24 in Figure Q-1 but not for subnet 172.16.5.0/24. In this case, it is clear that R1 has a working neighbor relationship with R2. In these cases, the root cause of this problem might still be related to the routing protocol, or it might not. For example, the problem may be that R2’s lower LAN interface is down. However, if R1 did not have a route for both 172.16.4.0/24 and 172.16.5.0/24, R1’s neighbor relationship with R2 could be the problem.
Troubleshooting routing protocol problems in real internetworks can be very complex—much more complex than even the most difficult CCNA R&S exam questions. Defining a generic troubleshooting process with which to attack both simple and complex routing protocol problems would require a lot of space and be counterproductive for preparing for the CCNA exam. This chapter instead offers a straightforward process for attacking routing protocol problems—specifically, problems similar to the depth and complexity of the CCNA exam.
If an exam question appears to be related to a problem with a routing protocol, you can quickly identify some common configuration errors with the following process—even if the question does not list the configuration. The process has three main tasks:
Step 1. Examine the internetwork design to determine on which interfaces the routing protocol should be enabled and which routers are expected to become neighbors.
Step 2. Verify whether the routing protocol is enabled on each interface (as per Step 1). If it isn’t, determine the root cause and fix the problem.
Step 3. Verify that each router has formed all expected neighbor relationships. If it hasn’t, find the root cause and fix the problem.
For instance, as noted with asterisks in Figure Q-2, each router should enable the routing protocol on each of the interfaces shown in the figure. Also, routing protocol neighbor relationships should form between R1 and R2, and R1 and R3, but not between R2 and R3.
While the concepts outlined in Figure Q-2 should be somewhat obvious by now, this chapter discusses how some of the most common configuration mistakes can impact the interfaces used by a routing protocol and whether a routing protocol creates neighbor relationships.
This section examines the second major troubleshooting step outlined in the previous section of the chapter: how to verify the interfaces on which the routing protocol has been enabled. Both EIGRP and OSPF configuration enable the routing protocol on an interface by using the network router subcommand. For any interfaces matched by the network commands, the routing protocol tries the following two actions:
Attempt to find potential neighbors on the subnet connected to the interface
Advertise the subnet connected to that interface
At the same time, the passive-interface router subcommand can be configured so that the router does not attempt to find neighbors on the interface (the first action just listed), but still advertises the connected subnet (the second action).
Three show commands are all that is needed to know exactly which interfaces have been enabled with EIGRP and which interfaces are passive. In particular, the show ip eigrp interfaces command lists all EIGRP-enabled interfaces that are not passive interfaces. The show ip protocols command essentially lists the contents of the configured network commands for each routing protocol and a separate list of the passive interfaces. Comparing these two commands identifies all EIGRP-enabled interfaces and those that are passive.
For OSPF, the command works slightly differently, with the show ip ospf interface brief command listing all OSPF-enabled interfaces (including passive interfaces). Using this command, along with the list of passive interfaces listed by the show ip protocols command, again identifies all fully enabled OSPF interfaces as well as all passive interfaces.
Table Q-1 summarizes the commands that identify the interfaces on which OSPFv2 and EIGRP are enabled for easier reference.
Table Q-1 Key Commands to Find Routing Protocol-Enabled Interfaces
Command |
Key Information |
Lists Passive Interfaces? |
---|---|---|
show ip eigrp interfaces |
Lists the interfaces on which EIGRP is enabled (based on the network commands), excluding passive interfaces. |
No |
show ip ospf interface brief |
Lists the interfaces on which the OSPFv2 is enabled (based on the network router subcommands or ip ospf interface subcommands), including passive interfaces. |
Yes |
show ip protocols |
Lists the contents of the network configuration commands for each routing process, and lists enabled but passive interfaces. |
Yes |
Note
All the commands in Table Q-1 list the interfaces regardless of interface status, in effect telling you the results of the network and passive-interface configuration commands.
So, for the major troubleshooting step covered in this section, the task is to use the commands in Table Q-1 and analyze the output. First, an EIGRP example will be shown, followed by an OSPF example.
This section shows a few examples of the commands in the context of Figure Q-3, which is used in all the examples in this chapter.
This example includes four routers, with the following scenario in this case:
R1 and R2 are configured correctly on both LAN interfaces.
R3 is mistakenly not enabled with EIGRP on its G0/1 interface.
R4 meant to use a passive-interface G0/1 command because no other routers are off R4’s G0/1 LAN. However, R4 has instead configured a passive-interface G0/0 command.
This example begins by showing the working details between Routers R1 and R2, and then moves on to discuss the issues related to R3 and R4.
Examples Q-1 and Q-2 list configuration and show commands for R1 and R2, respectively. Each lists the related configuration, the show ip eigrp interfaces and show ip protocols command, and the EIGRP-learned routes on each router.
Example Q-1 EIGRP Interfaces Problem: R1 Commands
R1# show running-config ! only pertinent lines shown router eigrp 99 network 10.0.0.0 ! R1# show ip eigrp interfaces EIGRP-IPv4 Interfaces for AS(99) Xmit Queue PeerQ Mean Pacing Time Multicast Pending Interface Peers Un/Reliable Un/Reliable SRTT Un/Reliable Flow Timer Routes Gi0/0 2 0/0 0/0 2 0/0 50 0 Gi0/1 0 0/0 0/0 0 0/0 0 0 R1# show ip protocols *** IP Routing is NSF aware *** Routing Protocol is "eigrp 99" Outgoing update filter list for all interfaces is not set Incoming update filter list for all interfaces is not set Default networks flagged in outgoing updates Default networks accepted from incoming updates EIGRP-IPv4 Protocol for AS(99) Metric weight K1=1, K2=0, K3=1, K4=0, K5=0 NSF-aware route hold timer is 240 Router-ID: 1.1.1.1 Topology : 0 (base) Active Timer: 3 min Distance: internal 90 external 170 Maximum path: 4 Maximum hopcount 100 Maximum metric variance 1 Automatic Summarization: disabled Maximum path: 4 Routing for Networks: 10.0.0.0 Routing Information Sources: Gateway Distance Last Update 10.1.1.2 90 09:55:51 10.1.1.3 90 00:02:00 Distance: internal 90 external 170 R1# show ip route eigrp ! Legend omitted for brevity 10.0.0.0/8 is variably subnetted, 5 subnets, 2 masks D 10.1.22.0/24 [90/30720] via 10.1.1.2, 00:00:40, GigabitEthernet0/0
Example Q-2 EIGRP Interfaces Problem: R2 Commands
R2# show running-config ! only pertinent lines shown router eigrp 99 network 10.1.0.0 0.0.255.255 R2# show ip eigrp interfaces EIGRP-IPv4 Interfaces for AS(99) Xmit Queue PeerQ Mean Pacing Time Multicast Pending Interface Peers Un/Reliable Un/Reliable SRTT Un/Reliable Flow Timer Routes Gi0/0 2 0/0 0/0 1 0/1 50 0 Gi0/1 0 0/0 0/0 0 0/0 0 0 R2# show ip protocols *** IP Routing is NSF aware *** Routing Protocol is "eigrp 99" Outgoing update filter list for all interfaces is not set Incoming update filter list for all interfaces is not set Default networks flagged in outgoing updates Default networks accepted from incoming updates EIGRP-IPv4 Protocol for AS(99) Metric weight K1=1, K2=0, K3=1, K4=0, K5=0 NSF-aware route hold timer is 240 Router-ID: 2.2.2.2 Topology : 0 (base) Active Timer: 3 min Distance: internal 90 external 170 Maximum path: 4 Maximum hopcount 100 Maximum metric variance 1 Automatic Summarization: disabled Maximum path: 4 Routing for Networks: 10.1.0.0/16 Routing Information Sources: Gateway Distance Last Update 10.1.1.3 90 00:02:30 10.1.1.1 90 09:56:20 Distance: internal 90 external 170 R2# show ip route eigrp ! Legend omitted for brevity 10.0.0.0/8 is variably subnetted, 5 subnets, 2 masks D 10.1.11.0/24 [90/30720] via 10.1.1.1, 00:03:25, GigabitEthernet0/0
The show ip eigrp interfaces command output on both R1 and R2 shows how both R1 and R2 have configured EIGRP using process ID 99, and that EIGRP has been enabled on both G0/0 and G0/1 on both these routers. This command lists only interfaces on which EIGRP has been enabled, excluding passive interfaces.
The highlighted parts of the show ip protocols command output on each router are particularly interesting. These sections show the parameters of the configured network commands. The show ip protocols command lists a separate line under the header “Routing for Networks,” one for each configured network command. Example Q-1’s output suggests R1 has a network 10.0.0.0 configuration command (as shown at the beginning of the example), and Example Q-2’s “10.1.0.0/16” suggests R2 has a network 10.1.0.0 0.0.255.255 command.
The next few pages now look at the problems caused by the configuration on Routers R3 and R4.
First, Example Q-2 gives brief insight into the current problem caused by R3. The end of R2’s show ip protocols command (Example Q-2) lists two routing information sources: 10.1.1.1 (R1) and 10.1.1.3 (R3). However, R2 has learned only one EIGRP route (10.1.11.0/24), as shown in the show ip route eigrp command output. When working properly, R2 should learn three EIGRP routes—one for each of the other LAN subnets shown in Figure Q-3.
Example Q-3 shows the root cause on R3. First, R3’s show ip eigrp interfaces command lists G0/0, but not G0/1, so a problem might exist with how EIGRP has been configured on G0/1. The configuration at the top of the example lists the root cause: an incorrect network command, which does not enable EIGRP on R3’s G0/1 interface.
Example Q-3 EIGRP Problems on R3
R3# show running-config ! lines omitted for brevity router eigrp 99 network 10.1.1.3 0.0.0.0 network 10.1.13.3 0.0.0.0 auto-summary R3# show ip eigrp interfaces EIGRP-IPv4 Interfaces for AS(99) Xmit Queue PeerQ Mean Pacing Time Multicast Pending Interface Peers Un/Reliable Un/Reliable SRTT Un/Reliable Flow Timer Routes Gi0/0 2 0/0 0/0 1 0/1 50 0 R3# show ip protocols *** IP Routing is NSF aware *** Routing Protocol is "eigrp 99" Outgoing update filter list for all interfaces is not set Incoming update filter list for all interfaces is not set Default networks flagged in outgoing updates Default networks accepted from incoming updates EIGRP-IPv4 Protocol for AS(99) Metric weight K1=1, K2=0, K3=1, K4=0, K5=0 NSF-aware route hold timer is 240 Router-ID: 3.3.3.3 Topology : 0 (base) Active Timer: 3 min Distance: internal 90 external 170 Maximum path: 4 Maximum hopcount 100 Maximum metric variance 1 Automatic Summarization: disabled Maximum path: 4 Routing for Networks: 10.1.1.3/32 10.1.13.3/32 Routing Information Sources: Gateway Distance Last Update 10.1.1.2 90 00:05:14 10.1.1.1 90 00:05:14 Distance: internal 90 external 170
The root cause of R3’s problem is that R3 has a network 10.1.13.3 0.0.0.0 configuration command, which does not match R3’s 10.1.33.3 G0/1 IP address. If the configuration was not available in the exam question, the show ip protocols command could be used to essentially see the same configuration details. In this case, the show ip protocols command on R3 lists the text “10.1.13.3/32” as a reference to the contents of the incorrect network command’s parameters, with “/32” translating to a wildcard mask of 32 binary 0s, or decimal 0.0.0.0.
R3’s incorrect configuration means that two actions do not happen on R3’s G0/1 interface. First, R3 does not try to find neighbors on its G0/1 interface, which is not a big deal in this case. However, R3 also does not advertise subnet 10.1.33.0/24, the connected subnet off R3’s G0/1 interface.
Moving on to R4’s problem, Example Q-4 shows why R1 and R2 do not learn R4’s 10.1.44.0/24 subnet. In this case, on R4, the engineer could have correctly used a passive-interface gigabitethernet0/1 router subcommand because no other routers should exist off R4’s G0/1 interface. However, the engineer mistakenly made R4’s G0/0 interface passive.
Example Q-4 EIGRP Problems on R4
R4# show running-config ! lines omitted for brevity router eigrp 99 passive-interface GigabitEthernet0/0 network 10.0.0.0 auto-summary R4# show ip eigrp interfaces EIGRP-IPv4 Interfaces for AS(99) Xmit Queue PeerQ Mean Pacing Time Multicast Pending Interface Peers Un/Reliable Un/Reliable SRTT Un/Reliable Flow Timer Routes Gi0/1 0 0/0 0/0 0 0/1 0 0 R4# show ip protocols | begin Routing for Networks Routing for Networks: 10.0.0.0 Passive Interface(s): GigabitEthernet0/0 Routing Information Sources: Gateway Distance Last Update Distance: internal 90 external 170
Note
The last command on the example, show ip protocols | begin Routing for Networks, lists the command output, but starting with the line with the literal case-sensitive string Routing for Networks. You can use this feature with any output from a command when you prefer to view only later lines of the command’s output.
To find this mistake without the configuration, Example Q-4 lists two useful commands. R4’s show ip eigrp interfaces command omits the (G0/0) passive interface, which means that R4 will not attempt to find EIGRP neighbors off that interface. Also, the highlighted part of R4’s show ip protocols command output lists G0/0 as a passive interface, which again means that R4 does not even attempt to become neighbors with others off its G0/0 interface.
OSPF has the same basic requirements as EIGRP for interfaces, with a few exceptions. First, EIGRP routers need to use the same autonomous system number (ASN) as their neighboring routers, as configured in the router eigrp asn global configuration command. OSPF routers can use any process ID on the router ospf process-id command, with no need to match their neighbors. Second, OSPF requires that the interfaces connected to the same subnet be assigned to the same OSPF area, whereas EIGRP has no concept of areas.
Example Q-5 shows a mostly working OSPF internetwork, again based on Figure Q-3. The problem in this case relates to the area design, as shown in Figure Q-4, the revised version of Figure Q-3. All subnets should be placed into area 0. However, the engineer made a configuration mistake on R2, putting both its interfaces into area 1. As a result, R2’s G0/0 interface breaks the OSPF design rule of being in the same subnet as R1, R3, and R4, but not being in the same OSPF area.
Example Q-5 begins to break down the problem by looking at the status of OSPF on the router interfaces of R1 and R2, using the show ip ospf interface brief command.
Example Q-5 show ip interface brief on R1 and R2
R1> show ip ospf interface brief Interface PID Area IP Address/Mask Cost State Nbrs F/C Gi0/1 1 0 10.1.11.1/24 1 DR 0/0 Gi0/0 1 0 10.1.1.1/24 1 DROTH 2/2
! The following command is from R2 R2> show ip ospf interface brief Interface PID Area IP Address/Mask Cost State Nbrs F/C Gi0/1 2 1 10.1.22.2/24 1 WAIT 0/0 Gi0/0 2 1 10.1.1.2/24 1 WAIT 0/0
From a general perspective, the show ip ospf interface brief command lists output similar to the show ip eigrp interface command, with one line for each enabled interface. The show ip ospf interface command, not shown in the example, lists detailed OSPF information for each interface.
Specific to this problem, the output in Example Q-5 shows that R1 and R2 both have OSPF enabled on both LAN interfaces. However, this command also lists the area number for each interface, with R2 having both LAN interfaces in area 1. Also, these commands repeat the IP address and mask of the interfaces, so together, you can see that R1’s 10.1.1.1/24 address is in the same subnet as R2’s 10.1.1.2/24 address, putting these two routers in the same subnet but in different OSPF areas.
Example Q-6 shows another way to look at the problem, with the show ip protocols commands on both R1 and R2. Because this command lists the OSPF network commands in shorthand form, it can point toward a possible configuration error, even if the configuration is not available.
Example Q-6 Finding OSPF Configuration Errors with show ip protocols R1 and R2
R1> show ip protocols *** IP Routing is NSF aware *** Routing Protocol is "ospf 1" Outgoing update filter list for all interfaces is not set Incoming update filter list for all interfaces is not set Router ID 1.1.1.1 Number of areas in this router is 1. 1 normal 0 stub 0 nssa Maximum path: 4 Routing for Networks: 10.0.0.0 0.255.255.255 area 0 Routing Information Sources: Gateway Distance Last Update 2.2.2.2 110 00:14:32 3.3.3.3 110 00:14:32 10.1.44.4 110 00:14:42 Distance: (default is 110) R1> show ip route ospf ! Legend omitted for brevity 10.0.0.0/8 is variably subnetted, 6 subnets, 2 masks O 10.1.33.0/24 [110/2] via 10.1.1.3, 00:15:32, GigabitEthernet0/0 O 10.1.44.0/24 [110/2] via 10.1.1.4, 00:15:42, GigabitEthernet0/0
! Now moving to Router R2 R2> show ip protocols *** IP Routing is NSF aware *** Routing Protocol is "ospf 2" Outgoing update filter list for all interfaces is not set Incoming update filter list for all interfaces is not set Router ID 2.2.2.2 Number of areas in this router is 1. 1 normal 0 stub 0 nssa Maximum path: 4 Routing for Networks: 10.0.0.0 0.255.255.255 area 1 Routing Information Sources: Gateway Distance Last Update Distance: (default is 110) R2> Nov 15 12:16:39.377: %OSPF-4-ERRRCV: Received invalid packet: mismatched area ID, from backbone area must be virtual-link but not found from 10.1.1.1, GigabitEthernet0/0
Interestingly, a closer look at R2’s show ip protocols command output, particularly the highlighted portion, points out the configuration error. As usual, the section with the heading “Routing for Networks:” points to a shorthand version of the configuration. In this case, the highlighted phrase “10.0.0.0 0.255.255.255 area 1” is actually the exact syntax of the one network command on Router R2, minus the word network, or network 10.0.0.0 0.255.255.255 area 1. Because Figure Q-4 shows the design should put all interfaces in area 0, reconfiguring this command to instead be network 10.0.0.0 0.255.255.255 area 0 would solve this particular problem.
The end of the example also shows an unsolicited log message generated by Router R2, notifying the console user that this router has received a Hello from a router in a different area.
As you check the interfaces, you could also check several other details. It makes sense to go ahead and check the interface IP addresses, masks, and interface status values by using the show interfaces and show ip interface brief commands. In particular, it is helpful to note which interfaces are up/up, because a router will send no packets (including routing protocol packets) out interfaces that are not in an up/up state.
This final major section of the chapter examines the large number of facts that each router must check with each potential neighbor before the two routers become neighbors.
At a very basic level, routing protocols can easily create neighbor relationships using a Hello protocol. First, the routing protocol must be enabled on an interface. In addition, the interface may not be configured as a passive interface, because that stops the routing protocol from sending the Hello messages.
Beyond this basic process, the routing protocols actually check several other parameters to find out whether the routers should become neighbors. Both OSPF and EIGRP use Hello messages, and these messages each list information used to perform some basic verification checks. For example, as just shown in earlier Example Q-5, an OSPF router should not become neighbors with another router in another area because all routers on a common subnet should be in the same OSPF area by design.
After an EIGRP or OSPF router hears a Hello from a new neighbor, the routing protocol examines the information in the Hello, and compares that information with the local router’s own settings. If the settings match, great. If not, the routers do not become neighbors. Because there is no formal term for all these items that a routing protocol considers, this book just calls them neighbor requirements.
Table Q-2 lists the neighbor requirements for both EIGRP and OSPF. Following the table, the next few pages examine some of these settings for both EIGRP and OSPF, again using examples based on Figure Q-3.
Note
Even though it is important to study and remember the items in this table, when reading this chapter the first time, just keep reading. When later reviewing the chapter or part, make sure you remember the details in the table.
Table Q-2 Neighbor Requirements for EIGRP and OSPF
Requirement |
EIGRP |
OSPF |
---|---|---|
Interfaces must be in an up/up state. |
Yes |
Yes |
Interfaces must be in the same subnet. |
Yes |
Yes |
Access control lists (ACL) must not filter routing protocol messages. |
Yes |
Yes |
Must pass routing protocol neighbor authentication (if configured). |
Yes |
Yes |
Must use the same ASN/PID on the router configuration command. |
Yes |
No |
Hello and hold/dead timers must match. |
No |
Yes |
Router IDs (RID) must be unique. |
No1 |
Yes |
K-values must match. |
Yes |
N/A |
Must be in the same area. |
N/A |
Yes |
1 Having duplicate EIGRP RIDs does not prevent routers from becoming neighbors, but it can cause problems when external EIGRP routes are added to the routing table.
Unlike most of the neighbor requirements listed in Table Q-2, the first three requirements have very little to do with the routing protocols themselves. The two routers must be able to send packets to each other over the physical network to which they are both connected. To do that, the router interfaces must be up/up, and they must be in the same subnet. In addition, the routers must not be using an ACL that filters the routing protocol traffic.
For instance, OSPF sends many messages to the well-known multicast IP addresses 224.0.0.5 and 224.0.0.6, whereas EIGRP uses 224.0.0.10. An ACL command like access-list 101 deny ip any host 224.0.0.10, in an inbound ACL on a router interface, would filter incoming EIGRP packets. Or, an ACL command like access-list 102 deny ospf any any could filter all OSPF traffic. Even more difficult to notice is an ACL that has lots of permit commands that match different TCP and UDP port numbers, but does not match the routing protocol explicitly, so the routing protocol packets match the implicit deny any at the end of the ACL. So, take extra care to watch for ACLs, especially when it seems like all the routing protocol configuration looks good.
In practice, before examining the rest of the details of why two routers do not become neighbors, confirm that the two routers can ping each other on the local subnet. If the ping fails, investigate all the Layer 1, 2, and 3 issues that could prevent the ping from working (such as an interface not being up/up).
Now, on to the specific discussions about EIGRP and OSPF. Because the details differ slightly between the two routing protocols, this section first examines EIGRP, followed by OSPF.
Note
This section assumes that the routing protocol has actually been enabled on each required interface, as covered earlier in this chapter in the “Interfaces Enabled with a Routing Protocol” section.
Any two EIGRP routers that connect to the same data link, and whose interfaces have been enabled for EIGRP and are not passive, will at least consider becoming neighbors. To quickly and definitively know which potential neighbors have passed all the neighbor requirements for EIGRP, just look at the output of the show ip eigrp neighbors command. This command lists only neighbors that have passed all the neighbor verification checks.
Example Q-7 shows an example of the show ip eigrp neighbors command, with the four routers from Figure Q-3 again. In this case, all the routers have been configured correctly, so each has a neighbor relationship with the other three routers on the same LAN subnet.
Example Q-7 R1 show ip eigrp neighbors Command with All Problems Fixed
R1# show ip eigrp neighbors EIGRP-IPv4 Neighbors for AS(99) H Address Interface Hold Uptime SRTT RTO Q Seq (sec) (ms) Cnt Num 1 10.1.1.3 Gi0/0 13 00:00:20 1 100 0 31 2 10.1.1.4 Gi0/0 13 00:00:43 80 480 0 10 0 10.1.1.2 Gi0/0 13 00:13:52 1 100 0 20
If the show ip eigrp neighbors command does not list one or more expected neighbors, the first problem isolation step should be to find out if the two routers can ping each other’s IP addresses on the same subnet. If that works, start looking at the list of neighbor verification checks, as relisted for EIGRP here in Table Q-3. Table Q-3 summarizes the EIGRP neighbor requirements, while noting the best commands with which to determine which requirement is the root cause of the problem.
Table Q-3 EIGRP Neighbor Requirements and the Best show/debug Commands
Requirement |
Best Commands to Isolate the Problem |
---|---|
Must be in the same subnet. |
show interfaces, show ip interface |
Must use the same ASN on the router configuration command. |
show ip eigrp interfaces, show ip protocols |
Must pass EIGRP neighbor authentication. |
debug eigrp packets |
K-values must match. |
show ip protocols |
Of the four rows of requirements listed in Table Q-3, the first two have already been discussed in this chapter, and do not need further discussion.
For EIGRP authentication (the third item in the table), EIGRP supports the capability for routers to trust routers as EIGRP neighbors only if the routers share the same security key (password); if that check fails, the neighbor relationship fails. By default, routers do not attempt EIGRP authentication, which allows the routers to form EIGRP neighbor relationships. If one router uses authentication, and the other does not, they will not become neighbors. If both use authentication, they must use the same authentication key to become neighbors.
The last item in the table, EIGRP K-values, refers to the EIGRP metric components and the metric calculation. These K-values are variables that basically enable or disable the use of the different components in the EIGRP composite metric. Cisco recommends leaving these values at their default settings, using only bandwidth and delay in the metric calculation. The K-value settings must match before two routers will become neighbors; you can check the K-values on both routers with the show ip protocols command.
Example Q-8 shows three problems that can cause EIGRP routers to fail to become neighbors. This example uses the usual design for this chapter, as repeated in Figure Q-5. The figure shows the same routers, and same interfaces, but with the following problems:
R2 has been configured with IP address 10.1.2.2/24 in a different subnet than R1, R3, and R4.
R3 has been configured to use ASN 199 with the router eigrp 199 command instead of ASN 99, as used on the other three routers.
R4 has been configured to use message digest 5 (MD5) authentication, whereas the other routers use no authentication.
R1 can actually detect two of the problems using local commands and messages, as shown in Example Q-8. R1 generates an unsolicited log message for the mismatched subnet problem, and a debug command on R1 can reveal the authentication failure. The example shows some running commentary inside the example.
Example Q-8 Common Problems Preventing the Formation of EIGRP Neighbors (R1)
! First, R1 has no neighbor relationships yet. R1 uses ASN (process) 99. R1# show ip eigrp neighbors EIGRP-IPv4 Neighbors for AS(99) R1# ! Next, R1 generates a log message, which shows up at the console, stating ! that the router with IP address 10.1.2.2 is not on the same subnet as R1. ! *Nov 15 16:19:14.740: %DUAL-6-NBRINFO: EIGRP-IPv4 99: Neighbor 10.1.2.2 (GigabitEthernet0/0) is blocked: not on common subnet (10.1.1.1/24) ! Next, R1 enables a debug that shows messages for each packet received from R4, ! which uses the wrong password (authentication key string) ! R1# debug eigrp packets EIGRP Packets debugging is on (UPDATE, REQUEST, QUERY, REPLY, HELLO, IPXSAP, PROBE, ACK, STUB, SIAQUERY, SIAREPLY) R1# *Nov 15 16:20:30.865: EIGRP: Gi0/0: ignored packet from 10.1.1.4, opcode = 5 (authentication off or key-chain missing)
Example Q-8 shows some evidence of the mismatched subnet with R2, and the invalid authentication problem with R4. Even without knowing the details, it is easy to imagine that if one router’s EIGRP process uses authentication with a defined password, and the other does not, that authentication will fail. The result? Neighbor relationships do not form.
Example Q-8 shows details about two of the problems, but not any details about the incorrect ASN configured on R3. Example Q-9 shows those details by listing excerpts from two show commands on R3, both of which identify the ASN configured on that router. By using these same commands on all the routers, you could note that R1, R2, and R4 use ASN 99, whereas R3 uses 199, as shown in Example Q-9.
Example Q-9 Displaying the Incorrect ASN (199) on R3
R3# show ip protocols Routing Protocol is "eigrp 199" ! ! The first line of output from show ip eigrp interfaces lists ASN 199 ! R3# show ip eigrp interfaces EIGRP-IPv4 Interfaces for AS(199) Xmit Queue Mean Pacing Time Multicast Pending Interface Peers Un/Reliable SRTT Un/Reliable Flow Timer Routes Gi0/0 0 0/0 0 0/1 0 0 Gi0/1 0 0/0 0 0/1 0 0
Similar to EIGRP, a router’s show ip ospf neighbor command lists all the neighboring routers that have met all the requirements to become an OSPF neighbor as listed in Table Q-2. So, the first step in troubleshooting OSPF neighbors is to look at the list of neighbors.
Example Q-10 lists the output of a show ip ospf neighbor command on Router R2, from Figure Q-4. All four routers sit on the same LAN subnet, in area 0, with correct configurations, so all four routers form a valid OSPF neighbor relationship.
Example Q-10 Normal Working show ip ospf neighbors Command on Router R2
R2# show ip ospf neighbor Neighbor ID Pri State Dead Time Address Interface 1.1.1.1 1 FULL/BDR 00:00:37 10.1.1.1 GigabitEthernet0/0 3.3.3.3 1 2WAY/DROTHER 00:00:37 10.1.1.3 GigabitEthernet0/0 4.4.4.4 1 FULL/DR 00:00:31 10.1.1.4 GigabitEthernet0/0
First, note that the neighbor IDs, listed in the first column, identify neighbors by their router ID (RID). For this example network, all four routers use an easily guessed RID. Further to the right, the Address column lists the interface IP address used by that neighbor on the common subnet.
A brief review of OSPF neighbor states can help you understand a few of the subtleties of the output in the example. A router’s listed status for each of its OSPF neighbors—the neighbor’s state—should settle into either a 2-way or full state under normal operation. For neighbors that do not need to directly exchange their databases, typically two non-designated router (DR) routers on a LAN, the routers should settle into a 2-way neighbor state. In most cases, two neighboring routers need to directly exchange their full link-state databases (LSDB) with each other. As soon as that process has been completed, the two routers settle into a full neighbor state.
In Example Q-10, Router R4 is the DR, and R1 is the backup DR (BDR), so R2 and R3 (as non-DRs) do not need to directly exchange routes. Therefore, R2’s neighbor state for R3 (RID 3.3.3.3) in Example Q-10 is listed as 2-way.
Note
Notably, OSPF neighbors do not have to use the same process ID on the router ospf process-id command to become neighbors. In Example Q-10, all four routers use different PIDs.
If the show ip ospf neighbor command does not list one or more expected neighbors, you should confirm, even before moving on to look at OSPF neighbor requirements, that the two routers can ping each other on the local subnet. But if the two neighboring routers can ping each other, and the two routers still do not become OSPF neighbors, the next step is to examine each of the OSPF neighbor requirements. Table Q-4 summarizes the requirements, listing the most useful commands with which to find the answers.
Table Q-4 OSPF Neighbor Requirements and the Best show/debug Commands
Requirement |
Best show Command |
Best debug Command |
---|---|---|
Must be in the same subnet. |
show interfaces |
debug ip ospf hello |
Hello and dead timers must match. |
show ip ospf interface |
debug ip ospf hello |
Must be in the same area. |
show ip ospf interface brief |
debug ip ospf adj |
RIDs must be unique. |
show ip ospf |
(N/A; log messages identify this problem) |
Must pass any neighbor authentication. |
show ip ospf interface |
debug ip ospf adj |
This topic looks at a couple of OSPF neighbor problems using the usual four-router network from Figure Q-4, with all interfaces in area 0. However, the following problems have been introduced into the design:
R2 has been configured with both LAN interfaces in area 1, whereas the other three routers’ G0/0 interfaces are assigned to area 0.
R3 is using the same RID (1.1.1.1) as R1.
R4 has been configured with a Hello/dead timer of 5/20 on its G0/0 interface, instead of the 10/40 used (by default) on R1, R2, and R3.
Figure Q-6 shows these same problems for reference.
Earlier in this chapter, the “OSPF Interface Troubleshooting” section showed how to use the show ip ospf interface command to list the area numbers and find OSPF area mismatches. This next topic shows how to see that same issue using the debug ip ospf adj command, as shown in Example Q-11. This command lists messages related to OSPF neighbor adjacency events, and shows messages that identify the area mismatch (with R2).
Example Q-11 Finding Mismatched Area Problem with R1 debug
R1# debug ip ospf adj OSPF adjacency events debugging is on R1# *Nov 15 13:42:02.288: OSPF-1 ADJ Gi0/0: Rcv pkt from 10.1.1.2, area 0.0.0.0, mismatched area 0.0.0.1 in the header R1# R1# undebug all All possible debugging has been turned off
As noted in Table Q-4, the debug ip ospf adj command helps troubleshoot mismatched OSPF area problems. The first part of the highlighted message in the example lists shorthand about a received packet (“Rcv pkt”) from 10.1.1.2, which is R2’s IP address. The rest of the message mentions R1’s area (0.0.0.0), and the area claimed by the other router (0.0.0.1). (Note that the message lists the 32-bit area number as a dotted-decimal number.)
This particular example focuses on the symptom (that a neighbor relationship does not start), and the debug messages that identify the problem (mismatched areas). However, finding the configuration error may take some work, because the problem could be more complex than just having the wrong area number configured on a command.
One harder-to-notice configuration error happens when the configuration has multiple network commands, with different area numbers, that all happen to match one interface’s IP address. IOS stores the OSPF network commands to the configuration in the same order they are configured (which is the same order listed in the output of show running-config). IOS processes the commands in sequence, so that the first network command that matches a particular interface is used to set the OSPF area number.
For instance, imagine a router with interface G0/1 configured with IP address 1.1.1.1. The OSPF configuration lists the following two network commands, in that order. Both would match the interface IP address of 1.1.1.1, so IOS uses the first command, which lists area 1. IOS would not use the second command, even though it uses a wildcard mask that is more specific.
network 1.0.0.0 0.255.255.255 area 1
network 1.1.1.1 0.0.0.0 area 0
Another tricky configuration error that can result in an area mismatch occurs when configuring both the network OSPF subcommand and the ip ospf interface subcommand on the same router. IOS supports using both on the same router at the same time. However, IOS does not prevent a case in which a network command attempts to enable OSPF in one area, and the ip ospf interface subcommand attempts to enable OSPF in a different area. When that happens, IOS uses the area number defined in the ip ospf interface subcommand.
For instance, with the two network commands just listed, if the ip ospf 1 area 5 command was configured on that router’s interface, that interface would be in area 5; IOS would prefer that setting over any OSPF network command.
Note
Using both network router subcommands and ip ospf interface subcommands allows an easier migration from the older to newer style OSPF configuration. However, most enterprises today would use either network commands or ip ospf commands in one router.
Next, Example Q-12 shows R1 and R3 both trying to use RID 1.1.1.1. Interestingly, both routers automatically generate a log message for the duplicate OSPF RID problem between R1 and R3; the end of Example Q-12 shows one such message. For the exams, just use the show ip ospf commands on both R3 and R1 to easily list the RID on each router, noting that they both use the same value.
Example Q-12 Comparing OSPF Router IDs on R1 and R3
! Next, on R3: R3 lists the RID of 1.1.1.1
!
R3# show ip ospf
Routing Process "ospf 3" with ID 1.1.1.1
Start time: 00:00:37.136, Time elapsed: 02:20:37.200
! lines omitted for brevity
! Back to R1: R1 also uses RID 1.1.1.1 R1# show ip ospf Routing Process "ospf 1" with ID 1.1.1.1 Start time: 00:01:51.864, Time elapsed: 12:13:50.904 Supports only single TOS(TOS0) routes Supports opaque LSA Supports Link-local Signaling (LLS) Supports area transit capability Supports NSSA (compatible with RFC 3101) Event-log enabled, Maximum number of events: 1000, Mode: cyclic Router is not originating router-LSAs with maximum metric Initial SPF schedule delay 5000 msecs Minimum hold time between two consecutive SPFs 10000 msecs Maximum wait time between two consecutive SPFs 10000 msecs Incremental-SPF disabled Minimum LSA interval 5 secs Minimum LSA arrival 1000 msecs LSA group pacing timer 240 secs Interface flood pacing timer 33 msecs Retransmission pacing timer 66 msecs Number of external LSA 0. Checksum Sum 0x000000 Number of opaque AS LSA 0. Checksum Sum 0x000000 Number of DCbitless external and opaque AS LSA 0 Number of DoNotAge external and opaque AS LSA 0 Number of areas in this router is 1. 1 normal 0 stub 0 nssa Number of areas transit capable is 0 External flood list length 0 IETF NSF helper support enabled Cisco NSF helper support enabled Reference bandwidth unit is 100 mbps Area BACKBONE(0) (Inactive) Number of interfaces in this area is 3 Area has no authentication SPF algorithm last executed 00:52:42.956 ago SPF algorithm executed 9 times Area ranges are Number of LSA 1. Checksum Sum 0x00C728 Number of opaque link LSA 0. Checksum Sum 0x000000 Number of DCbitless LSA 0 Number of indication LSA 0 Number of DoNotAge LSA 0 Flood list length 0 *May 29 00:01:25.679: %OSPF-4-DUP_RTRID_NBR: OSPF detected duplicate router-id 1.1.1.1 from 10.1.1.3 on interface GigabitEthernet0/0
First, focus on the problem: the duplicate RIDs. The first line of the show ip ospf command on the two routers quickly shows the duplicate use of 1.1.1.1. To solve the problem, assuming R1 should use 1.1.1.1 and R3 should use another RID (maybe 3.3.3.3), change the RID on R3, and restart the OSPF process. To do so, use the router-id 3.3.3.3 OSPF subcommand and use the EXEC mode command clear ip ospf process.
Also, take a moment to read over the log message generated on each router when a duplicate RID exists.
Finally, note that the show ip ospf commands in Example Q-12 also show a common false positive for a root cause of OSPF neighbor problems. OSPF PIDs—the number of the router ospf command—do not have to match. Note that in Example Q-12 that same first line of output shows that R3 uses the router ospf 3 command, per the phrase “Process ospf 3,” whereas R1 uses the router ospf 1 command, as noted with the phrase “Process ospf 1.” These mismatched numbers are not a problem.
Finally, consider the problem created on R4, with the configuration of a different Hello timer and dead timer as compared with the default settings on R1, R2, and R3. Whereas EIGRP allows neighbors to use a different Hello timer, OSPF does not, so this mismatch prevents R4 from becoming neighbors with any of the other three OSPF routers.
Example Q-13 shows the easiest way to find the mismatch, using the show ip ospf interface command on both R1 and R4. This command lists the Hello and dead timers for each interface, as highlighted in the example. Note that R1 uses 10 and 40 (Hello and dead), whereas R4 uses 5 and 20.
Example Q-13 Finding Mismatched Hello/Dead Timers
R1# show ip ospf interface G0/0
GigabitEthernet0/0 is up, line protocol is up
Internet Address 10.1.1.1/24, Area 0, Attached via Network Statement
Process ID 1, Router ID 1.1.1.1, Network Type BROADCAST, Cost: 1
Topology-MTID Cost Disabled Shutdown Topology Name
0 1 no no Base
Transmit Delay is 1 sec, State DR, Priority 1
Designated Router (ID) 1.1.1.1, Interface address 10.1.1.1
No backup designated router on this network
Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5
! lines omitted for brevity
! Moving on to R4 next
!
R4# show ip ospf interface Gi0/0
GigabitEthernet0/0 is up, line protocol is up
Internet Address 10.1.1.4/24, Area 0, Attached via Network Statement
Process ID 4, Router ID 10.1.44.4, Network Type BROADCAST, Cost: 1
Topology-MTID Cost Disabled Shutdown Topology Name
0 1 no no Base
Transmit Delay is 1 sec, State DR, Priority 1
Designated Router (ID) 10.1.44.4, Interface address 10.1.1.4
No backup designated router on this network
Timer intervals configured, Hello 5, Dead 20, Wait 20, Retransmit 5
! lines omitted for brevity
The debug ip ospf hello command can also uncover this problem because it lists a message for each Hello that reveals the Hello/dead timer mismatch, as shown in Example Q-14.
Example Q-14 Finding Mismatched Hello/Dead Timers with debug
R1# debug ip ospf hello OSPF hello events debugging is on R1# *Nov 15 14:05:10.616: OSPF-1 HELLO Gi0/0: Rcv hello from 10.1.44.4 area 0 10.1.1.4 *Nov 15 14:05:10.616: OSPF-1 HELLO Gi0/0: Mismatched hello parameters from 10.1.1.4 *Nov 15 14:05:10.616: OSPF-1 HELLO Gi0/0: Dead R 20 C 40, Hello R 5 C 10 Mask R 255.255.255.0 C 255.255.255.0
Although debug messages can be a little difficult to understand, a few comments make the meaning of these messages much clearer. The highlighted message uses a C to mean “configured value”—in other words, the value on the local router, or R1 in this case. The R in the message means “received value,” or the value listed in the received Hello. In this case
“Dead R 20 C 40” means that R1 received a Hello with a dead timer set to 20, while R1’s configured value is set to 40.
“Hello R 5 C 10” means that R1 received a Hello with the Hello timer set to 5, while R1’s configured value is set to 10.
Note that any IP subnet mismatch problems could also be found with this same debug, based on the received and configured subnet masks.
This last short discussion in this chapter looks at these two additional topics: shutting down the routing protocol process and the interface maximum transmission unit (MTU) size.
Cisco uses the IOS shutdown command in several contexts. You can use the shutdown command in interface configuration mode to disable the interface so that it no longer sends and receives packets. Cisco IOS switches allow the shutdown command in VLAN configuration mode, causing the switch to stop forwarding frames in that VLAN. In both cases, the shutdown command does not remove any configuration; it simply causes IOS to stop a particular function. Then, the no shutdown command in the same command mode re-enables that function.
IOS allows both the OSPFv2 and EIGRP routing protocol processes to be disabled and enabled with the shutdown and no shutdown commands, respectively, in routing protocol configuration mode. When a routing protocol process is shut down, IOS
Brings down any existing neighbor relationships
Does not form new neighbor relationships
Quits sending Hello messages
Does not remove routing protocol configuration
Basically, shutting down the routing protocol process gives the network engineer a way to stop using the routing protocol on that router, without having to remove all the configuration.
From a troubleshooting perspective, on the exam, what would you expect to see if a small design was configured perfectly, except that one router’s OSPF process was shut down? First, the router with the shutdown routing protocol process would not have any OSPF neighbors, and other routers would not list that router as a neighbor. But because the OSPF shutdown subcommand does not remove any configuration, the show ip ospf interfaces command still shows evidence that OSPF is configured on the interfaces.
Example Q-15 shows an example on Router R5, as shown in Figure Q-7. R5 is a different router than the one used in earlier examples, but it begins the example with two OSPF neighbors, R2 and R3, with router IDs 2.2.2.2 and 3.3.3.3. The example shows the OSPF process being shut down, the neighbors failing, and those two key OSPF show commands: show ip ospf neighbor and show ip ospf interface brief.
Example Q-15 Shutting Down an OSPF Process, and the Resulting Neighbor States
R5# show ip ospf neighbor Neighbor ID Pri State Dead Time Address Interface 2.2.2.2 1 FULL/DR 00:00:35 10.1.12.2 GigabitEthernet0/1 3.3.3.3 1 FULL/DR 00:00:33 10.1.13.3 GigabitEthernet0/2 R5# configure terminal Enter configuration commands, one per line. End with CNTL/Z. R5(config)# router ospf 1 R5(config-router)# shutdown R5(config-router)# ^Z R5# *Mar 23 12:43:30.634: %OSPF-5-ADJCHG: Process 1, Nbr 2.2.2.2 on GigabitEthernet0/1 from FULL to DOWN, Neighbor Down: Interface down or detached *Mar 23 12:43:30.635: %OSPF-5-ADJCHG: Process 1, Nbr 3.3.3.3 on GigabitEthernet0/2 from FULL to DOWN, Neighbor Down: Interface down or detached R5# R5# show ip ospf neighbor R5# R5# show ip ospf interface brief Interface PID Area IP Address/Mask Cost State Nbrs F/C Gi0/1 1 0 10.1.12.1/24 1 DOWN 0/0 Gi0/2 1 0 10.1.13.1/24 1 DOWN 0/0
The two show commands point out a couple of particularly important facts. First, before the shutdown, the show ip ospf neighbor command lists two neighbors. After the shutdown, the same command lists no neighbors at all. Second, the show ip ospf interface brief command does list the interfaces on which OSPF is enabled, on the local router’s own IP addresses. However, it lists a state of DOWN, which is a reference to the neighbor’s state.
The MTU size defines a per-interface setting used by the router for its Layer 3 forwarding logic, defining the largest network layer packet that the router will forward out each interface. For instance, the IPv4 MTU size of an interface defines the maximum size IPv4 packet that the router can forward out an interface.
Routers often use a default MTU size of 1500 bytes, with the ability to set the value as well. The ip mtu size interface subcommand defines the IPv4 MTU setting, and the ipv6 mtu size command sets the equivalent for IPv6 packets.
In an odd twist, two OSPFv2 routers can actually become OSPF neighbors, and reach 2-way state, even if they happen to use different IPv4 MTU settings on their interfaces. However, they fail to exchange their LSDBs. Eventually, after trying and failing to exchange their LSDBs, the neighbor relationship also fails.
The concepts behind what happens with an MTU mismatch work the same with both OSPFv2 and OSPFv3.
Tables Q-5, Q-6, and Q-7 list configuration, verification, and debug commands used in this chapter. As an easy review exercise, cover the left column in a table, read the right column, and try to recall the command without looking. Then repeat the exercise, covering the right column, and try to recall what the command does.
Table Q-5 Appendix Q Configuration Command Reference
Command |
Description |
---|---|
ip hello-interval eigrp as-number timer-value |
Interface subcommand that sets the EIGRP Hello interval for that EIGRP process |
ip hold-time eigrp as-number seconds |
Interface subcommand that sets the EIGRP hold time for the interface |
ip ospf hello-interval seconds |
Interface subcommand that sets the interval for periodic Hellos |
ip ospf dead-interval number |
Interface subcommand that sets the OSPF dead timer |
passive-interface type number |
Router subcommand, for both OSPF and EIGRP that tells the routing protocol to stop sending Hellos and stop trying to discover neighbors on that interface |
Table Q-6 Appendix Q show Command Reference
Command |
Description |
---|---|
show ip protocols |
Shows routing protocol parameters and current timer values, including an effective copy of the routing protocols’ network commands and a list of passive interfaces |
show ip eigrp interfaces |
Lists the interfaces on which EIGRP has been enabled for each EIGRP process, except passive interfaces |
show ip route eigrp |
Lists only EIGRP-learned routes from the routing table |
show ip eigrp neighbors |
Lists EIGRP neighbors and status |
show ip ospf interface brief |
Lists the interfaces on which the OSPF protocol is enabled (based on the network commands), including passive interfaces |
show ip ospf interface [type number] |
Lists detailed OSPF settings for all interfaces, or the listed interface, including Hello and dead timers and OSPF area |
show ip route ospf |
Lists routes in the routing table learned by OSPF |
show ip ospf neighbor |
Lists neighbors and current status with neighbors, per interface |
show ip ospf |
Lists a group of messages about the OSPF process itself, listing the OSPF Router ID in the first line |
show interfaces |
Lists a long set of messages, per interface, that lists configuration, state, and counter information |
show interfaces description |
Lists one line of output per interface with brief status information |
Table Q-7 Appendix Q debug Command Reference
Command |
Description |
---|---|
debug eigrp packets |
Lists log messages for EIGRP packets that flow in and out of the router |
debug ip ospf adj |
Issues log messages for adjacency events, meaning events related to routers becoming neighbors |
debug ip ospf events |
Issues log messages for each action taken by OSPF, including the receipt of messages |
debug ip ospf packet |
Issues log messages describing the contents of all OSPF packets |
debug ip ospf hello |
Issues log messages describing Hellos and Hello failures |
undebug all |
EXEC command used to disable all current debugs |