Screen scraping versus API structured output

Imagine a common scenario where we need to log into the network device and make sure all the interfaces on the devices are in an up/up state (both the status and the protocol are showing as up). For the human network engineers getting into a Cisco NX-OS device, it is simple enough to issue the show IP interface brief command in the Terminal to easily tell from the output which interface is up:

    nx-osv-2# show ip int brief
IP Interface Status for VRF "default"(1)
Interface IP Address Interface Status
Lo0 192.168.0.2 protocol-up/link-up/admin-up
Eth2/1 10.0.0.6 protocol-up/link-up/admin-up
nx-osv-2#

The line break, white spaces, and the first line of the column title are easily distinguished from the human eye. In fact, they are there to help us line up, say, the IP addresses of each interface from line one to line two and three. If we were to put ourselves in the computer's position, all these spaces and line breaks only takes us away from the really important output, which is: which interfaces are in the up/up state? To illustrate this point, we can look at the Paramiko output for the same operation:

    >>> new_connection.send('sh ip int briefn')
16
>>> output = new_connection.recv(5000)
>>> print(output)
b'sh ip int briefrrnIP Interface Status for VRF
"default"(1)rnInterface IP Address Interface
StatusrnLo0 192.168.0.2 protocol-up/link-up/admin-up
rnEth2/1 10.0.0.6 protocol-up/link-up/admin-up rnrnx-
osv-2# '
>>>

If we were to parse out that data, here is what I would do in a pseudo-code fashion (simplified representation of the code I would write):

  1. Split each line via the line break.
  2. I may or may not need the first line that contains the executed command of show ip interface brief. For now, I don't think I need it.
  3. Take out everything on the second line up until the VRF, and save it in a variable as we want to know which VRF the output is showing.
  4. For the rest of the lines, because we do not know how many interfaces there are, we will use a regular expression statement to search if the line starts with possible interfaces, such as lo for loopback and Eth for Ethernet interfaces.
  5. We will need to split this line into three sections via space, each consisting of the name of the interface, IP address, and then the interface status.
  6. The interface status will then be split further using the forward slash (/) to give us the protocol, link, and the admin status.

Whew, that is a lot of work just for something that a human being can tell at a glance! You might be able to optimize the code and the number of lines, but in general this is what we need to do when we need to screen scrap something that is somewhat unstructured. There are many downsides to this method, but some of the bigger problems that I can see are listed as follows:

  • Scalability: We spent so much time on painstaking details to parse out the outputs from each command. It is hard to imagine how we can do this for the hundreds of commands that we typically run.
  • Predictability: There is really no guarantee that the output stays the same between different software versions. If the output is changed ever so slightly, it might just render our hard-earned battle of information gathering useless.
  • Vendor and software lock-in: Perhaps the biggest problem is that once we spend all this time parsing the output for this particular vendor and software version, in this case, Cisco NX-OS, we need to repeat this process for the next vendor that we pick. I don't know about you, but if I were to evaluate a new vendor, the new vendor is at a severe on-boarding disadvantage if I have to rewrite all the screen scrap code again.

Let's compare that with an output from an NX-API call for the same show IP interface brief command. We will go over the specifics of getting this output from the device later in this chapter, but what is important here is to compare the following output to the previous screen scraping output:

    {
"ins_api":{
"outputs":{
"output":{
"body":{
"TABLE_intf":[
{
"ROW_intf":{
"admin-state":"up",
"intf-name":"Lo0",
"iod":84,
"ip-disabled":"FALSE",
"link-state":"up",
"prefix":"192.168.0.2",
"proto-state":"up"
}
},
{
"ROW_intf":{
"admin-state":"up",
"intf-name":"Eth2/1",
"iod":36,
"ip-disabled":"FALSE",
"link-state":"up",
"prefix":"10.0.0.6",
"proto-state":"up"
}
}
],
"TABLE_vrf":[
{
"ROW_vrf":{
"vrf-name-out":"default"
}
},
{
"ROW_vrf":{
"vrf-name-out":"default"
}
}
]
},
"code":"200",
"input":"show ip int brief",
"msg":"Success"
}
},
"sid":"eoc",
"type":"cli_show",
"version":"1.2"
}
}

NX-API can return output in XML or JSON, and this is the JSON output that we are looking at. Right away, you can see the output is structured and can be mapped directly to the Python dictionary data structure. There is no parsing required—you can simply pick the key and retrieve the value associated with the key. You can also see from the output that there are various metadata in the output, such as the success or failure of the command. If the command fails, there will be a message telling the sender the reason for the failure. You no longer need to keep track of the command issued, because it is already returned to you in the input field. There is also other useful metadata in the output, such as the NX-API version.

This type of exchange makes life easier for both vendors and operators. On the vendor side, they can easily transfer configuration and state information. They can add extra fields when the need to expose additional data arises using the same data structure. On the operator side, they can easily ingest the information and build their infrastructure around it. It is generally agreed on that automation is much needed and a good thing. The questions are usually centered on the format and structure of the automation. As you will see later in this chapter, there are many competing technologies under the umbrella of API. On the transport side alone, we have REST API, NETCONF, and RESTCONF, among others. Ultimately, the overall market might decide about the final data format in the future. In the meantime, each of us can form our own opinions and help drive the industry forward.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset