Chapter 9. Troubleshooting Tools

NetScaler has extensive logging and tracing capabilities and includes some GUI-based tools that help you with issue analysis. To complement these, Citrix also provides external tools to support you with your NetScaler deployments and associated troubleshooting. A discussion of these various tools is the focus of this chapter. We will touch on these utilities in the following order:

  • The nsconmsg utility
  • NetScaler packet tracing (nstrace)
  • The showtechsupport utility
  • NetScaler dashboard and reporting
  • Citrix insight (TAAS)
  • Command Center and Insight Center

The nsconmsg utility

Nsconmsg is a logging process that runs nonstop on the NetScaler. It contains a snapshot of all the available counters taken once every 7 seconds. There are an excess of 10,000 counters each with specific conditions for specific features, which makes this such a formidable troubleshooting tool in identifying if a problem condition has been hit.

The logs produced by this utility are available in /var/nslog. By default, they roll over once every two days or on reaching a size of 300 MB whichever is first. The file newlog will always be latest (current log file). They can go up to 99 files after which they restart from 0.

nsconmsg syntax and options

nsconmsg is used from the shell prompt with the following syntax:

nsconmsg –K <path> –g <counter_name> -d <display_option>

The commonly used options are as follows:

  • -K: This is used to specify the name of the log file; if not used, current values of counters are shown.
  • -g: This is used to grep for the values of specific counters, making nsconmsg more usable. For example, use –g ssl to see all available SSL counters before narrowing down to specific ones.
  • -d: This is used to specify the type of operation. The common ones are:
    • setime: This displays the start and end time of the newslog file.
    • current: This displays the current value of each counter, on a 7-second basis.
    • event: This displays all events during that log period. It is useful for looking for any service or monitor failures and HA-related events.
    • oldconmsg: This is used for load balancing and content switching; it allows you to see a variety of information such as the number of requests received by each service, number of active transaction, and how the surge queue looks at each 7-second interval.
    • stats: This displays the current statistics. It is useful to see whether a particular counter (and thus a specific error condition) has ever been incremented.
    • statswt0: This displays only counters that incremented during the current log.
    • memstats: This displays memory-related statistics.
    • consmsg: This displays console messages.

Here are some useful examples:

  • Use setime to display the time covered by the logs. If you know the time of the issue, this command will help zero in on the newnslog file you need for analysis:
    nsconmsg -K /var/nslog/newnslog.x.tar.gz -d setime
    nsconmsg syntax and options
  • Use current to find out the rate at which a counter is incrementing:
    nsconmsg -K /var/nslog/newnslog -g <counter name> -d current

    In the following example, I am trying to identify whether a rate limit is being hit and at which point packets will be dropped. The pattern I would grep for this would -g _rl_.

    nsconmsg syntax and options

    Tip

    The easiest way to arrive at the counters you need is to Google search for various NetScaler counters. For example, to find the rate limiting counter I used in the screenshot, I looked for all NetScaler NIC-related counters and found the article CTX132772. Similar articles exist for all features. You can then use the –g option to see whether the counter is being hit.

    Some of the common patterns I use are -g ssl_err, -g http_err, -g tcp_err, -g nic_err, and –g csw_err.

  • Use event to look at event-related data:
    nsconmsg -K /var/nslog/newnslog -d event

    This command is very useful to see if and when a particular failure event such as a HA failover (by grepping for the word node) has occurred.

    nsconmsg syntax and options

    Another important use case for event data is checking for services and vServer flaps.

  • Use oldconmsg to obtain traffic management data. Load balancing as well as other feature level info such as for content switching, SSL, or monitors can be extracted using the -d oldconmsg switch by specifying the -s feature-specific switch. For example:
    nsconmsg -K /var/nslog/newnslog -s ConLB=1 -d oldconmsg
nsconmsg -K /var/nslog/newnslog -s ConCSW=1 -d oldconmsg

    The output of these commands were discussed in detail in Chapter 2, Traffic Management Features.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset