Chapter 15
Programmability and Automation
This chapter covers the following topics:
Applications
Introduction to Automation and Programmability
What is automation? This question can be answered differently as it applies to various industries, such as robotics, process management, and information technology (IT). This chapter discusses automation in the specific context of computer networks. To start understanding automation, first consider an example. Assume that the network of an organization consists of 100 nodes, all running the same software and hardware. After months of testing and software validation, the network operations team agrees to upgrade the software to a newer release. The team then spends nearly 2 months working day and night to complete the upgrade. Soon after the upgrade, the network faces a massive outage and the network collapses. The root cause is found to be a software defect. The software also has a configuration command as a workaround that helps in service restoration. Now the network operations team has two options:
Upgrade to a release that has the fix for the defect
Apply the configuration workaround
Either option is time consuming, but the second one, applying a configuration workaround, involves the least amount of time. Applying a workaround on 100 nodes is not an easy task, however. This is where automation comes into play. If the process of provisioning and validating the successful application of the configuration can be automated, the workaround or fix can be quickly deployed on all the nodes in the network, possibly at the same time.
Network automation is simply defined as the process of recording certain steps using a script, a programming language, or an application and then replaying those recorded steps across the network at a designated time or at specific time intervals without human intervention. Network automation not only saves time, but also prevents the possibility of human error after the scripts are validated. The automation and orchestration methodologies leverage the pragmatic interfaces of the devices present in the network infrastructure to help reduce provisioning time and accelerate service delivery.
Programmability, on the other hand, refers to the capability of an entity to be programmed. Programmability enhances the capability of the programmable device to be extended or modified. The acts of programming an entity and interacting with it are different, a distinction that is often misunderstood. An application or script can interact with the network device without extending or modifying any of its existing capabilities. Manageability and programmability are distinct from one another because they serve different purposes:
Manageability: Allows an entity to be managed in terms of how it can be operated, provisioned, administered, and maintained
Programmability: Enables an entity or device to be extended or modified, to add new features and properties that enhance its capability
The following sections discuss in detail the multiple automation and programmability tools available with NX-OS to give network engineers more control and flexibility in performing various actions and running third-party applications.
Introduction to Open NX-OS
NX-OS has always been built on top of a Linux kernel. However, until recently, much of the underlying Linux operating system was not fully exposed to the user. The latest Nexus 9000 and Nexus 3000 series of switches run what Cisco has named Open NX-OS, beginning with NX-OS Release 7.0(3).
One reason Linux has achieved success in all aspects of computing and networking is its flexibility and vast user support community. With Open NX-OS, Linux applications can run on the switch to complement the feature-rich NX-OS operating system without a wrapper library or customization. The major components of Open NX-OS are listed here:
Kernel version 3.4: This is a 64-bit kernel that provides a balance of features and stability.
Kernel stack: The user space Netstack process that previous versions of NX-OS used has been replaced with the kernel stack. This allows the interfaces on the switch to be mapped to the kernel as standard Linux netdevs and namespaces. Interfaces are managed using standard Linux commands such as ifconfig and tcpdump from the Bash shell.
Open package management: Tools such as RPM Package Manager (RPM) and Yellowdog Updater, Modified (YUM) aid in installing or patching software on the switch and provide extensibility.
Container support: Linux containers (LXCs) run directly on the platform and provide access to a Centos 7–based Guest shell. This enables users to customize their switch in a secure and isolated environment.
Open NX-OS provides the foundation for a true DevOps-managed data center switch by providing Linux capabilities such as modularity, fault isolation, resiliency, and much more.
Note
For more details on Open NX-OS architecture, refer to the book Programmability and Automation with Cisco Open NX-OS, at Cisco.com.
Shells and Scripting
A shell is a text-based user interface that enables the user to perform administrative tasks and interact directly with the operating system. The shell is commonly used to perform various administrative tasks, such as executing filesystem operations, starting and stopping processes, creating user accounts, executing scripts, and configuring the system. The concept of a shell dates back to the early days of the UNIX operating system. After years of evolution and refinement, the shell is a key functional component in modern operating systems.
The NX-OS operating system provides a shell more commonly known as the command-line interface (CLI). As the practice of automation through scripting and network management techniques has evolved, the capability to have direct shell access to the underlying Linux operating system of NX-OS has become desirable. The following section provides examples of the NX-OS Bash shell, the Guest shell, and Python capabilities. These powerful tools enable the automation of many operational tasks, reducing the administrative burden.
Bash Shell
Bourne-Again Shell (Bash) is a modern UNIX shell, a successor of the Bourne shell. It provides a rich feature set and built-in capability to interact with the low-level components of the underlying operating system. The Bash shell is currently available on the Nexus 9000, Nexus 3000, and Nexus 3500 series platforms. The Bash shell provides shell access to the underlying Linux operating system, which has additional capabilities that the standard NX-OS CLI does not provide. To enable the Bash shell on Nexus 9000 switches, enable the command feature bash-shell. Then use the command run bash cli to execute any Bash CLI commands. Users can also move into shell mode by using the NX-OS CLI command run bash and then can execute the relevant Bash CLI commands from the Bash shell. Example 15-1 illustrates how to enable the bash-shell feature and use the Bash shell command pwd to display the current working directory. To check whether the bash-shell feature is enabled, use the command show bash-shell. Example 15-1 also demonstrates various basic commands on the Bash shell. The Bash command id -a is used to verify the current user, as well as Group and Group ID information. You can also use echo commands to print various messages based on the script requirements.
Example 15-1 Enabling the bash-shell Feature and Using Bash Commands
N9k-1# show bash-shell
N9k-1# run bash
bash-4.2$ pwd
bash-4.2$ id -a
uid=2002(admin) gid=503(network-admin) groups=503(network-admin)
bash-4.2$
bash-4.2$ echo "First Example on " 'uname -n' " using bash-shell " $BASH_VERSION
Note
It is recommended that you become familiar with the UNIX/Linux bash shell commands for this section.
On NX-OS, only users with the roles network-admin, vdc-admin, and dev-ops can use the Bash shell. Other users are restricted from using Bash unless it is specially allowed in their role. To validate check roles are permitted to use the Bash shell, use the command show role [name role-name]. Example 15-2 displays the permission for the network-admin and dev-ops user roles.
Example 15-2 network-admin and dev-ops User Role Permissions
Role: network-admin
Description: Predefined network admin role has access to all commands
on the switch
-------------------------------------------------------------------
Rule Perm Type Scope Entity
-------------------------------------------------------------------
1 permit read-write
N9k-1# show role name dev-ops
Role: dev-ops
Description: Predefined system role for devops access. This role
cannot be modified.
-------------------------------------------------------------------
Rule Perm Type Scope Entity
-------------------------------------------------------------------
6 permit command conf t ; username *
5 permit command attach module *
4 permit command slot *
3 permit command bcm module *
2 permit command run bash *
1 permit command python *
With the NX-OS bash-shell feature, it becomes possible to create Bash shell scripts consisting of multiple Bash commands that execute in sequence on the underlying Linux operating system. The Bash script is created and saved with the extension .sh. The Bash shell also gives users traceability options, which is useful for debugging purposes while executing a shell script. This is activated by using the option -x along with the #!/bin/bash statement. Example 15-3 illustrates how to create a shell script and verify its execution with script debugging enabled via the -x option.
Example 15-3 Creating and Debugging Bash Shell Scripts
/bootflash/home/admin
bash-4.2$ cat test.sh
#!/bin/bash
echo "Troubleshooting Route Flapping Issue Using Bash Shell"
counter="$(vsh -c "show ip route ospf | grep 00:00:0 | count")"
echo "Printing Counter - " $counter
if [ $counter -gt 0 ]
then
echo "Following Routes Flapped @ " 'date'
vsh -c "show tech ospf >> bootflash:shtechospf"
vsh -c "show tech routing ip unicast >> bootflash:shtechrouting_unicast"
else
echo "No Flapping Routes at this point"
bash-4.2$ /bin/bash -x test.sh
bash-4.2$ vsh -c "clear ip route *"
Clearing ALL routes
bash-4.2$ /bin/bash -x test.sh
N9k-1# dir bootflash:
2677690 Nov 21 20:21:19 2017 shtechrouting_unicast
In addition, the Bash shell is used to install RPM packages on NX-OS. Use the yum command from the Bash shell to perform various RPM-related operations such as install, remove, and delete. Example 15-4 demonstrates how to view the list of all installed packages on the Nexus switch, as well as how to install and remove a package. In this example, the BFD package is installed and removed. Note that when the package is removed, the feature becomes unavailable from the NX-OS CLI; the packages determine which features are made available to NX-OS.
Example 15-4 Installing and Removing RPM Packages from the Bash Shell
base-files.n9000 3.0.14-r74.2 installed
bfd.lib32_n9000 2.0.0-7.0.3.I6.1 installed
bgp.lib32_n9000 2.0.0-7.0.3.I6.1 installed
container-tracker.lib32_n9000 2.0.0-7.0.3.I6.1 installed
core.lib32_n9000 2.0.0-7.0.3.I6.1 installed
eigrp.lib32_n9000 2.0.0-7.0.3.I6.1 installed
eth.lib32_n9000 2.0.0-7.0.3.I6.1 installed
fcoe.lib32_n9000 2.0.0-7.0.3.IFD6.1 installed
isis.lib32_n9000 2.0.0-7.0.3.I6.1 installed
lacp.lib32_n9000 2.0.0-7.0.3.I6.1 installed
linecard2.lib32_n9000 2.0.0-7.0.3.I6.1 installed
lldp.lib32_n9000 2.0.0-7.0.3.I6.1 installed
ntp.lib32_n9000 2.0.0-7.0.3.I6.1 installed
nxos-ssh.lib32_n9000 2.0.0-7.0.3.I6.1 installed
ospf.lib32_n9000 2.0.0-7.0.3.I6.1 installed
perf-cisco.n9000_gdb 3.12-r0 installed
platform.lib32_n9000 2.0.0-7.0.3.I6.1 installed
rip.lib32_n9000 2.0.0-7.0.3.I6.1 installed
shadow-securetty.n9000_gdb 4.1.4.3-r1 installed
snmp.lib32_n9000 2.0.0-7.0.3.I6.1 installed
svi.lib32_n9000 2.0.0-7.0.3.I6.1 installed
sysvinit-inittab.n9000_gdb 2.88dsf-r14 installed
tacacs.lib32_n9000 2.0.0-7.0.3.I6.1 installed
task-nxos-base.n9000_gdb 1.0-r0 installed
telemetry.lib32_n9000 2.2.1-7.0.3.I6.1 installed
tor.lib32_n9000 2.0.0-7.0.3.I6.1 installed
vtp.lib32_n9000 2.0.0-7.0.3.I6.1 installed
bash-4.2$ sudo yum -y install bfd
Loaded plugins: downloadonly, importpubkey, localrpmDB, patchaction, patching,
protect-packages
groups-repo | 1.1 kB 00:00 ...
localdb | 951 B 00:00 ...
patching | 951 B 00:00 ...
thirdparty | 951 B 00:00 ...
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package bfd.lib32_n9000 0:2.0.0-7.0.3.I6.1 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
Installing:
bfd lib32_n9000 2.0.0-7.0.3.I6.1 groups-repo 483 k
Transaction Summary
================================================================================
Install 1 Package
Total download size: 483 k
Installed size: 1.8 M
Downloading Packages:
Running Transaction Check
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : bfd-2.0.0-7.0.3.I6.1.lib32_n9000 1/1
starting pre-install package version mgmt for bfd
pre-install for bfd complete
starting post-install package version mgmt for bfd
post-install for bfd complete
Installed:
bfd.lib32_n9000 0:2.0.0-7.0.3.I6.1
Complete!
N9k-1(config)# feature bfd
Please disable the ICMP / ICMPv6 redirects on all IPv4 and IPv6 interfaces
running BFD sessions using the command below
'no ip redirects '
'no ipv6 redirects '
bash-4.2$ sudo yum -y erase bfd
Loaded plugins: downloadonly, importpubkey, localrpmDB, patchaction, patching,
protect-packages
Setting up Remove Process
Resolving Dependencies
--> Running transaction check
---> Package bfd.lib32_n9000 0:2.0.0-7.0.3.I6.1 will be erased
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Removing:
bfd lib32_n9000 2.0.0-7.0.3.I6.1 @groups-repo 1.8 M
Transaction Summary
================================================================================
Remove 1 Package
Installed size: 1.8 M
Downloading Packages:
Running Transaction Check
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Erasing : bfd-2.0.0-7.0.3.I6.1.lib32_n9000 1/1
starting pre-remove package version mgmt for bfd
pre-remove for bfd complete
Removed:
bfd.lib32_n9000 0:2.0.0-7.0.3.I6.1
Complete!
N9k-1(config)# feature bfd
^
Guest Shell
The network paradigm has moved from hardware, software, and management network elements to extensible network elements. The built-in Python and Bash execution environments enable network operators to execute custom scripts in NX-OS environments using the Cisco-supplied APIs and classes to interact with some of the major NX-OS components. However, in some scenarios, network operators want to integrate third-party applications and host the application on NX-OS. To meet those needs, NX-OS provides a third-party application hosting framework that enables users to host their applications in a dedicated Linux user space environment. Network operators must use the Cisco Application Development Toolkit (ADT) to cross-compile their software and package it with a Linux root file system into a Cisco Open Virtual Appliance (OVA) package. These OVAs are then deployed on the NX-OS network element using the application hosting feature.
NX-OS software introduces the NX-OS Guest shell feature on the Nexus 9000 and Nexus 3000 series switches. The Guest shell is an open source and secure Linux environment for rapid third-party software development and deployment. The guestshell feature leverages the benefits of the Python and Bash execution environments and the NX-OS application hosting framework.
The Guest shell is enabled by default on Nexus 9000 and Nexus 3000. You can explicitly enable or destroy the guestshell feature on NX-OS. Table 15-1 describes the various guest shell commands.
Table 15-1 Guest Shell Feature Commands
Command |
Description |
guestshell enable |
This CLI installs and enables the Guest shell service. When this command is enabled, you can enter the Guest shell using the guestshell command. |
guestshell disable |
This command disables the guest shell service. Access to the Guest shell then is also disabled. |
guestshell destroy |
The CLI deactivates and uninstalls the current guest shell. All system resources associated with the Guest shell return to the system. |
guestshell reboot |
The CLI deactivates and reactivates the current Guest shell. |
guestshell run command-line |
The CLI is used to execute a program inside a Guest shell, return the output, and exit the Guest shell. |
guestshell sync |
The CLI deactivates the current active Guest shell, syncs its root file system contents to the standby RP, and then reactivates the Guest shell on the active RP. |
guestshell upgrade |
The CLI deactivates and performs an upgrade of the current Guest shell using the OVA that is embedded within the booted system image. Upon successful upgrade, the Guest shell is reactivated. |
guestshell resize |
The CLI allows modification to the default or existing parameters related to the Guest shell, such as CPU, memory, and root file system parameters. |
guestshell |
The CLI is used to enter the Guest shell. |
When the Guest shell is up and running, you can check the details using the command show guestshell detail. This command displays the path of the OVA file, the status of the Guest shell service, resource reservations, and the file system information of the Guest shell. Example 15-5 displays the detailed information of the Guest shell on a Nexus 9000 switch.
Example 15-5 Guest Shell Details
Virtual service guestshell+ detail
State : Activated
Package information
Name : guestshell.ova
Path : /isanboot/bin/guestshell.ova
Application
Name : GuestShell
Installed version : 2.1(0.0)
Description : Cisco Systems Guest Shell
Signing
Key type : Cisco release key
Method : SHA-1
Licensing
Name : None
Version : None
Resource reservation
Disk : 250 MB
Memory : 256 MB
CPU : 1% system CPU
Attached devices
Type Name Alias
---------------------------------------------
Disk _rootfs
Disk /cisco/core
Serial/shell
Serial/aux
Serial/Syslog serial2
Serial/Trace serial3
If the Guest shell does not come up, check the log for any error messages using the show logging logfile command. To troubleshoot issues with the Guest shell, use the command show virtual-service [list] to view both the status of the Guest shell and the resources the Guest shell is using. Example 15-6 displays the virtual service list and the resources being utilized by the current Guest shell on the Nexus 9000 switch.
Example 15-6 Virtual Service List and Resource Utilization
Virtual Service List:
Name Status Package Name
-----------------------------------------------------------------------
guestshell+ Activated guestshell.ova
N9k-1# show virtual-service
Virtual Service Global State and Virtualization Limits:
Infrastructure version : 1.9
Total virtual services installed : 1
Total virtual services activated : 1
Machine types supported : LXC
Machine types disabled : KVM
Maximum VCPUs per virtual service : 1
Resource virtualization limits:
Name Quota Committed Available
-----------------------------------------------------------------------
system CPU (%) 20 1 19
memory (MB) 3840 256 3584
bootflash (MB) 8192 250 4031
Note
If you cannot resolve the Guest shell problem, collect the output of show virtual-service tech-support and contact the Cisco Technical Assistance Center (TAC) for further investigation.
Python
With the networking industry’s push toward software-defined networking (SDN), multiple doors have opened for integrating scripting and programming languages with network devices. Python has gained industry-wide acceptance as the programming language of choice. Python is a powerful and easy-to-learn programming language that provides efficient high-level data structures and object-oriented features. These features make it an ideal language for rapid application development on most platforms.
Python integration is available on most Nexus platforms and does not require the installation of any special license. The interactive Python interpreter is invoked from the CLI on Nexus platforms by typing the python command. On Nexus 9000 and Nexus 3000 platforms, Python can also be used through the Guest shell. After executing the python command, the user is placed directly into the Python interpreter. Example 15-7 demonstrates the use of the Python interpreter from both the CLI and the guest shell.
Example 15-7 Python Interpreter from CLI and the Guest Shell
Python 2.7.5 (default, Nov 5 2016, 04:39:52)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print "Hello World...!!!"
[admin@guestshell ~]$ python
Python 2.7.5 (default, Jun 17 2014, 18:11:42)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-16)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print "Hello Again..!!!"
Note
Readers are advised to become familiar with the Python programming language. This chapter does not focus on writing specific Python programs, however; instead, it focuses on how to use Python on Nexus platforms.
In addition to the standard Python libraries, NX-OS provides the Cisco and CLI libraries, which you can import into your Python script to perform Cisco-specific functions on the Nexus switch. The Cisco library provides access to Cisco Nexus components. The CLI library provides the capability to execute commands from the Nexus CLI and return the result. Example 15-8 displays the package contents of both the Cisco and CLI libraries on NX-OS.
Example 15-8 Cisco and CLI Python Libraries on NX-OS
>>> help(cisco)
Help on package cisco:
NAME
cisco
FILE
/usr/lib64/python2.7/site-packages/cisco/__init__.py
PACKAGE CONTENTS
acl
bgp
buffer_depth_monitor
check_port_discards
cisco_secret
dohost
feature
history
interface
ipaddress
key
line_parser
mac_address_table
nxapi
nxcli
ospf
routemap
section_parser
ssh
system
tacacs
transfer
vlan
vrf
>>> help(cli)
Help on module cli:
NAME
cli
FILE
/usr/lib64/python2.7/site-packages/cli.py
FUNCTIONS
Noninteractive Python scripts are created and saved in the bootflash:scripts/ directory and are invoked with the source [script name] command. Another option is to utilize the Guest shell to create and invoke a Python script. The first line of your Python script must include the path to the Python interpreter, which is /usr/bin/env. Example 15-9 provides a sample Python script to configure a loopback interface and also to list all the interfaces in UP state on the Nexus switch. This script is created and invoked from within the Guest shell environment.
Example 15-9 Python Script to Print All Interfaces in UP State
import sys
cli ("conf t ; interface lo5 ; ip add 5.5.5.5/32")
print " ***Configured interface loopback5***"
print " ***Listing All interfaces on the device in UP state*** "
intf_list = json.loads(clid ("show interface brief"))
i = 0
while i < len (intf_list['TABLE_interface']['ROW_interface']):
intf = intf_list['TABLE_interface']['ROW_interface'][i]
i += 1
if intf['state'] == 'up':
print intf['interface']
sys.exit(0)
[admin@guestshell ~]$ python test.py
***Configured interface loopback5***
***Listing All interfaces on the device in UP state***
mgmt0
Ethernet1/4
Ethernet1/5
Ethernet1/13
Ethernet1/14
Ethernet1/15
Ethernet1/16
Ethernet1/19
Ethernet1/32
Ethernet1/37
Ethernet2/1
port-channel10
port-channel101
port-channel600
loopback0
loopback5
loopback100
Vlan100
Vlan200
Vlan300
A Python script can also be invoked from an Embedded Event Manager (EEM) applet as part of the action statement. Because multiple actions can be performed per event, multiple Python scripts can be called at different action steps, providing flexibility in the logic used to build the EEM applet. Example 15-10 illustrates the configuration of an EEM applet that triggers the previous Python script from an action statement. In this example, because the Python script is configured within the Guest shell, the Python script in the EEM script is invoked from the Guest shell. If the Python script is present in the bootflash:source/ directory, the command action number cli source python file-name must be used.
Example 15-10 Python Script Invoked from an EEM Applet
N9k-1(config-applet)# event syslog pattern "IF_.*DOWN:"
N9k-1(config-applet)# action 1 cli guestshell python test.py
N9k-1(config-applet)# exit
NX-SDK
The NX-OS software development kit (NX-SDK) is a C++ plug-in library that allows custom, native applications to access NX-OS functions and infrastructure. Using the NX-SDK, you can create custom CLI commands, syslog messages, event handlers, and error handlers. An example of using this functionality would be creating your custom application to register with the route manager to receive routing updates from the routing information base (RIB) and then taking some action based on the presence of the route. Three primary requirements must be met for using NX-SDK:
Docker
Linux environment (Ubuntu 14.04 or higher, Centos 6.7 or higher)
Cisco SDK (optional)
Note
NX-SDK can also be integrated with Python. Thus, Cisco SDK is not required for Python applications.
The NX-SDK must be installed before it can be used in the development environment. The installation steps follow:
Step 1. Pull a docker image from https://hub.docker.com/r/dockercisco/nxsdk
Step 2. Set the environment variables as follows for a 32-bit environment:
export ENXOS_SDK_ROOT=/enxos-sdk
cd $ENXOS_SDK_Root
source environment-setup-x86-linux
Step 3. Clone NX-SDK toolkit from GitHub.
Explore the API after forking the NX-SDK from GitHub and use it to create custom application packages to be installed on the Nexus switch.
Note
When creating custom applications, refer to the documentation and custom sample application code available as part of the NX-SDK.
Once the applications are built, use the rpm_gen.py Python script to automatically generate the RPM package. The script is present in the /NX-SDK/scripts directory. When the RPM package is built, the RPM package can be copied to the Nexus Switch in the bootflash: directory, where the package is then installed on the Nexus switch for further use. Example 15-11 demonstrates the installation steps for an RPM package on the Nexus 9000 switch. This example demonstrates the sample RPM package named customCliApp that is available as part of the NX-SDK kit. To start a custom application, first enable feature nxsdk. Then add the custom application as a service using the command nxsdk service-name app-name. You can check the status of the application using the command show nxsdk internal service.
Example 15-11 Installing a Custom RPM Package
N9k-1(config)# install add bootflash:customCliApp-1.0-1.0.0.x86_64.rpm
[####################] 100%
Install operation 1 completed successfully at Sun Nov 26 06:12:49 2017
N9k-1(config)# show install inactive
Boot Image:
NXOS Image: bootflash:/nxos.7.0.3.I6.1.bin
Inactive Packages:
Inactive Base Packages:
N9k-1(config)# install activate customCliApp-1.0-1.0.0.x86_64
[####################] 100%
Install operation 2 completed successfully at Sun Nov 26 06:13:40 2017
N9k-1(config)# show install active
Boot Image:
NXOS Image: bootflash:/nxos.7.0.3.I6.1.bin
Active Packages:
N9k-1(config)# feature nxsdk
N9k-1(config)# nxsdk service-name customCliApp
N9k-1# show nxsdk internal service
NXSDK Started/Temp unavailabe/Max services : 1/0/32
NXSDK Default App Path : /isan/bin/nxsdk
NXSDK Supported Versions : 1.0
Service-name Base App Started(PID) Version RPM Package
--------------------------------------------------------------------------------
customCliApp nxsdk_app1 VSH(not running) 1.0 customCliApp-1.0-1.0.0.x86_64
bash-4.2# /isan/bin/nxsdk/customCliApp &
[1] 8887
N9k-1# show nxsdk internal service
NXSDK Started/Temp unavailabe/Max services : 2/0/32
NXSDK Default App Path : /isan/bin/nxsdk
NXSDK Supported Versions : 1.0
Service-name Base App Started(PID) Version RPM Package
--------------------------------------------------------------------------------
customCliApp nxsdk_app1 VSH (8887) 1.0 customCliApp-1.0-1.0.0.x86_64
Note
In Example 15-11, the RPM package is installed using the Virtual shell (VSH). The RPM package can also be installed from the Bash shell.
If any failure or erroneous events occur with the custom application installation, you can check the NX-SDN event history logs using the command show nxsdk internal event-history [events | error]. Example 15-12 displays the event history logs for NX-SDK and highlights the logs that indicate the successful activation and startup of the application customCliApp.
Example 15-12 NX-SDK Event History
06:16:16 nxsdk_mgr : Added confcheck capability for 1.0, en 1, counter: 1
06:16:16 nxsdk_mgr : Adding confcheck capability for 1.0, en 0, counter: 0
06:16:16 nxsdk_mgr : Done: start service customCliApp nxsdk_app1
06:16:15 nxsdk_mgr : Heartbeat sent while start service customCliApp, nxsdk_app1
06:16:14 nxsdk_mgr : Feature nxsdk_app1 not enabled, State: 2, Error: SUCCESS
(0x0), Reason: SUCCESS (0x40aa000a)
06:16:14 nxsdk_mgr : Config service name customCliApp
06:16:14 nxsdk_mgr : App: /isan/bin/nxsdk/customCliApp is linked to libnxsdk
06:16:14 nxsdk_mgr : App: /isan/bin/nxsdk/customCliApp, MaJor Version: 1,
Minor Version: 0
06:14:30 nxsdk_mgr : Received CLIS Done Callback
06:14:30 nxsdk_mgr : Initialized with all core components
06:14:30 nxsdk_mgr : Initialized with sdwrap
06:14:30 nxsdk_mgr : Request all commands from CLIS
06:14:30 nxsdk_mgr : Initialized with CLIS
06:14:30 nxsdk_mgr : All core components for NXSDK_MGR are UP
06:14:30 nxsdk_mgr : Core component "clis(261)" is Up
06:14:30 nxsdk_mgr : Received feature enable message from FM
06:14:30 nxsdk_mgr : Started NXSDK_MGR mts thread, pid 6864
06:14:30 nxsdk_mgr : Started NXSDK_MGR mq mts thread, pid 6865
06:14:30 nxsdk_mgr : NXSDK_MGR Main Done: await join before exiting
06:14:30 nxsdk_mgr : nxsdk_mgr_create_threads: ok
06:14:30 nxsdk_mgr : Query sysmgr for core comp status: clis(261)
06:14:30 nxsdk_mgr : Done with NXSDK_MGR stateless recovery
Note
In addition to the event history logs, you can collect the output of show tech nxsdk if custom applications are failing to install or are not working.
NX-API
NX-OS provides an API known as the NX-API that enables you to interact with the switch using a standard request/response language. The traditional CLI was designed for human-to-switch interaction. Requests are made by typing a CLI command and receiving a response from the switch in the form of output to the client terminal. This response data is unstructured and requires the human operator to evaluate the output line by line to find the interesting piece of information in the output. Operators that use the traditional CLI interface to automate tasks through scripting are forced to follow the same data interpretation method by screen-scraping the output for the interesting data. This is not only inefficient, but also cumbersome because it requires output iteration and specific text matching through regular expressions.
The benefit of using the NX-API is the capability to send requests and receive responses that are optimized for machine-to-machine communication. In other words, when communicating through the NX-API, the request and response are formatted as structured data. The response received from the NX-API is provided as either Extensible Markup Language (XML) or JavaScript Object Notation (JSON). This is much more efficient and less error prone than parsing the entire human-readable CLI output for only a small percentage of interesting data. NX-API is used to obtain output from show commands, as well as to add or remove configuration, thus streamlining and automating operations and management in a large-scale network.
Communication between the client and NX-API running on the switch uses the Transport Control Protocol (TCP) and can be either Hypertext Transfer Protocol (HTTP) or Hypertext Transfer Protocol Secure (HTTPS), depending on the requirements. NX-API uses HTTP basic authentication. Requests must carry the username and password in the HTTP header. After a successful authentication, NX-API provides a session-based cookie using the name nxapi_auth. That session cookie should be included in subsequent NX-API requests. The privilege of the user is checked to confirm that the request is being made by a user with a valid username and password on the switch who also has the proper authorization for the commands being executed through the NX-API.
After successful authentication, you can start sending requests. The NX-API request object is either in JSON-RPC or a Cisco proprietary format. Table 15-2 describes the fields present in the JSON-RPC request object.
Table 15-2 JSON-RPC Request Object Fields
Field |
Description |
jsonrpc |
A string specifying the version of the JSON-RPC protocol. It must be exactly “2.0”. |
method |
A string containing the name of the method to be invoked. NX-API supports either of the following: “cli”: show or configuration commands “cli_ascii”: show or configuration commands, output without formatting |
params |
A structured value that holds the parameter values to be used during the invocation of the method. It must contain the following fields: “cmd”: A CLI command “version”: NX-API request version identifier |
id |
An optional identifier established by the client that must contain a string, a number, or a NULL value. If the user does not specify the id parameter, the server assumes that the request is simply a notification and provides no response. |
Figure 15-1 shows an example JSON-RPC request object used to query the switch for its configured switch name.
Figure 15-1 JSON-RPC Request Object
The second type of request object is the Cisco proprietary format, which is either XML or JSON. Table 15-3 provides a description for the fields used in the Cisco proprietary request object.
Table 15-3 Cisco Proprietary Request Object Fields
Field |
Description |
version |
Indicates the current NX-API version. |
type |
A string containing the type of command to be executed: cli_show: Used for show commands. cli_show_ascii: Used for show commands. Output is unformatted. cli_conf: Used for noninteractive configuration commands. bash: Used to execute noninteractive Bash commands on devices on which the Bash shell has been enabled. |
chunk |
Used to chunk the output of large show commands. A value of 0 indicates no chunk output; a value of 1 indicates that the output can be returned in multiple chunks. |
sid |
Valid when the response message is chunked. To retrieve the next chunk of a message, the user should send a request with the sid set to the sid in the previous response message. |
input |
The input can be one or multiple commands. Multiple commands should be separated with “ ;” (a blank character followed by semicolon). |
ouput_format |
The expected output format of the request message (XML or JSON). |
Figure 15-2 shows an example Cisco proprietary request object in both JSON and XML formats. This request object is used to query the switch for its configured switch name.
Figure 15-2 Cisco Proprietary Request Objects
The request object is sent to the switch on the configured HTTP (TCP port 80) or HTTPS (TCP port 443) port. The received request object is validated by the web server and the appropriate software object is provided with the request. The response object is then sent back from the switch in either JSON-RPC or Cisco proprietary formats to the client. Table 15-4 provides the field descriptions of the JSON-RPC response object.
Table 15-4 JSON-RPC Response Object Fields
Field |
Description |
jsonrpc |
This string specifies the version of the JSON-RPC protocol. It is exactly “2.0”. |
result |
This field is included only for successful requests. The value of this field contains the requested CLI output. |
error |
This field is included only on an errored request. The error object contains the following fields: “code”: An integer error code specified by the JSON-RPC specification “message”: A human-readable string that corresponds to the error code “data”: An optional structure that contains other useful information for the user. |
id |
This field contains the same value as the id field in the corresponding request object. If a problem occurred while parsing the id field in the request, this value is null. |
Figure 15-3 shows an example JSON-RPC response object.
Figure 15-3 JSON-RPC Response Object
Table 15-5 describes the fields included in the Cisco proprietary response object.
Table 15-5 Cisco Proprietary Response Object Fields
Field |
Description |
version |
The current NX-API version. |
type |
The type of command being executed. |
sid |
Session ID of the current response (valid only for chunked output). |
outputs |
Tag enclosing all of the commands’ output. |
output |
Tag enclosing a single command output. If the type is cli_conf or bash, it contains output from all of the commands. |
body |
The body of the response to the command requested. |
code |
The error code of the command execution. Standard HTTP error codes are used. |
msg |
The error message associated with the error code. |
Figure 15-4 shows an example Cisco proprietary response object in both JSON and XML formats.
Figure 15-4 Cisco Proprietary Response Objects
Multiple commands can be sent in a single request. For the JSON-RPC request object, this is done by linking an unlimited number of single JSON-RPC requests into a single JSON-RPC array. For the Cisco proprietary request object, up to 10 semicolon-separated commands can be linked in the input object. With either request object type, if a request fails for any reason, the subsequent requests are not executed.
The NX-API feature must be enabled in the global configuration of the switch using the feature nxapi command, as shown in the output of Example 15-13.
Example 15-13 NX-API Feature Configuration
Enter configuration commands, one per line. End with CNTL/Z.
NX-2(config)# feature nxapi
NX-2# show nxapi
HTTPS Listen on port 443
Note
The default HTTP and HTTPS ports are changed using the nxapi http port and nxapi https port configuration commands.
When the NX-API feature is enabled, you may authenticate and begin sending requests to the appropriate HTTP or HTTPS port. NX-OS also provides a sandbox environment for testing the functions of the API; this is accessed by using a standard web browser and connecting through HTTP to the switch management address.
Troubleshooting problems related to NX-API typically involve the TCP connection used to deliver the request and response messages between the switch and the client. The NX-OS ethanalyzer capture tool is used to troubleshoot connection issues from the client and confirm that the TCP three-way handshake is completed (see Example 15-14).
Example 15-14 Ethanalyzer Capture of the Client Connection
Capturing on mgmt0
192.168.1.50 -> 192.168.1.201 TCP 52018 > https [SYN] Seq=0 Win=65535 Len=0
MSS=1460 WS=5 TSV=568065210 TSER=0
192.168.1.201 -> 192.168.1.50 TCP https > 52018 [SYN, ACK] Seq=0 Ack=1
Win=16768 Len=0 MSS=1460 TSV=264852 TSER=568065210
192.168.1.50 -> 192.168.1.201 TCP 52018 > https [ACK] Seq=1 Ack=1 Win=65535
Len=0 TSV=568065211 TSER=264852
192.168.1.50 -> 192.168.1.201 SSL Client Hello
192.168.1.201 -> 192.168.1.50 TLSv1.2 Server Hello, Certificate, Server Key
Exchange, Server Hello Done
192.168.1.50 -> 192.168.1.201 TCP 52018 > https [ACK] Seq=518 Ack=1294
Win=65535 Len=0 TSV=568065232 TSER=264852
192.168.1.50 -> 192.168.1.201 TLSv1.2 Client Key Exchange, Change Cipher Spec,
Hello Request, Hello Request
192.168.1.201 -> 192.168.1.50 TLSv1.2 Encrypted Handshake Message, Change
Cipher Spec, Encrypted Handshake Message
After confirming that the TCP session from the client is established, additional information about the NX-API communication with the client is found with the show nxapi-server logs command. The server logs in Example 15-15 show the connection attempt, as well as the details of the request that was received. The execution of the CLI command is also shown in the log file, which is helpful in identifying why a particular batch of commands is failing. Finally, the response object sent to the client is also provided.
Example 15-15 NX-API Server Logs
ngx_http_cookie_set:627 2017 November 17 07:18:25.292 : creating cookie
ngx_http_ins_api_post_body_handler:549 2017 November 17 02:18:25.292 : Input
Message {
"ins_api": {
"version": "1.0",
"type": "cli_show",
"chunk": "0",
"sid": "1",
"input": "show switchname",
"output_format": "json"
}
}
parse_user_from_request:41 2017 November 17 02:18:25.292 : cookie had user
‘admin’
parse_user_from_request:55 2017 November 17 02:18:25.292 : auth header had user
‘admin’
pterm_idle_vsh_sweep:667 2017 November 17 02:18:25.292 : pterm_idle_vsh_sweep
pterm_get_vsh:710 2017 November 17 02:18:25.292 : vsh found: child_pid = 10558,
fprd = 0x98d0800, fpwr = 0x98d0968, fd = 14, user = admin, vdc id = 1
pterm_write_to_vsh:446 2017 November 17 02:18:25.292 : In vsh [14] Writing cmd
"show switchname | xml "
pterm_write_to_vsh:522 2017 November 17 02:18:25.302 : Cmd ‘show switchname | xml
‘ returned with ‘0’
pterm_write_to_vsh:627 2017 November 17 02:18:25.302 : Done processing vsh output
(ret=0)
_ins_api_cli_cmd:288 2017 November 17 02:18:25.302 : Incorrect XML data,
replacing special characters
_ins_api_cli_cmd:304 2017 November 17 02:18:25.302 : found ns vdc_mgr and copied
it to blob vdc_mgr len 7
pterm_write_to_vsh:446 2017 November 17 02:18:25.302 : In vsh [14] Writing cmd
"end"
pterm_write_to_vsh:522 2017 November 17 02:18:25.304 : Cmd ‘end’ returned with
‘0’
pterm_write_to_vsh:627 2017 November 17 02:18:25.304 : Done processing vsh output
(ret=0)
ngx_http_ins_api_post_body_handler:675 2017 November 17 02:18:25.304 : Sending
response {
"ins_api": {
"type": "cli_show",
"version": "1.0",
"sid": "eoc",
"outputs": {
"output": {
"input": "show switchname",
"msg": "Success",
"code": "200",
"body": {
"hostname": "NX02"
}
}
}
}
}
Note
Any activity from the NX-API is logged in the switch accounting log just like in the traditional CLI. The username associated with the NX-API is listed in the accounting log as nginx.
In addition to the NX-API server logs, NX-OS has a detailed show tech nxapi command that provides the server logs in addition to the nginx web server logs from the Linux process.
Summary
Automation and programmability are the defining building blocks for the future of networking. Open NX-OS was conceived to meet the future needs of SDN and the desire for users to natively execute third-party applications directly on Nexus switches. Open NX-OS provides the architecture that allows network operators and developers to create and deploy custom applications on their network devices. Integration of the powerful Bash shell and Guest shell has made it easy to create scripts for automating tasks on Nexus switches. This chapter covered in detail how you can leverage the Bash shell and the Guest shell to deploy third-party applications. Integration of Python with NX-OS enables you to create dynamic applications that enhance the functionality and manageability of Nexus switches. In addition to Python support, Cisco provides the NX-SDK, which supports building applications in both the C++ and Python languages and compile them as RPM packages. Finally, this chapter covered NX-API, an API that enables users to interact with the Nexus switch using standard request/response language.
References
Programmability and Automation with Cisco Open NX-OS: https://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/nexus9000/sw/open_nxos/programmability/guide/Programmability_Open_NX-OS.pdf