In order for end-to-end Kerberos to work, the client needs to be able to reach the KDC. This will not be possible in most environments, either because there are external users or partners in the mix, or simply because security policies require not exposing the KDC directly to all users. This is where Protocol Transition comes into the picture. Let's now turn our attention to the flow for this scenario. For the purpose of this section, I have set up LDAP in the frontend, with Kerberos in the backend.

Here, NetScaler authenticates the users with whatever authentication policy is bound to the AAA vServer but then uses Kerberos in the backend to obtain service tickets for those authenticated users. The following image demonstrates the flow:

The flow goes like this:

Here's a filtered trace snapshot that demonstrates the complete flow:

Note that there are no AS-REQ/AS-REP packets in the NetScaler to KDC exchange. This is because a TGT, once obtained, can be reused for several hours, and such a reuse is happening here:

There are a couple of important points to note here:

Troubleshooting Kerberos

Here are some steps you should consider when troubleshooting Kerberos:

• Ensure port 88 is open to the KDC.
• Ensure the backend servers providing the content are configured to negotiate Kerberos. The following screenshot shows what you might ideally set on IIS on the server under authentication:
  

  Kerberos with fallback to NTLM

• DNS plays a vital role in Kerberos, so ensure that it's properly working by trying to resolve the various entities.
• Check that the User and the resources are in the same Kerberos realm. If different, there are limitations on S4U depending on the AD domain and forest functional levels (2012R2 introduced some new cross-domain S4U functionality), but down-level domains may fail when the User and resource are not in the same domain.
• Ensure that delegation is configured correctly, that is:
  • The Service Account has delegation enabled
  • The Service Account is trusted for delegation
  • If you are performing constrained delegation, verify that the necessary services are in the list
  • For Protocol Transition to work, check Use any authentication protocol

The following is what I have used for the test I performed for my Windows-based Kerberos environment:

Configuration on AD allowing NetScaler system to trust "nskcd" to perform delegation on behalf of users to the HTTP service on the server

Focus on aaad.debug and nskrb.debug, as these are the most useful outputs to look at when troubleshooting Kerberos authentication. The screenshots that follow show a successful protocol transition from LDAP to Kerberos:

1. We start first by looking at the aaad.debug output (cat/tmp/aaad.debug) to see if the frontend authentication has been successful and that protocol transition has happened (S4U) before moving our attention to nskrb.debug (cat/tmp/nskrb.debug).
2. The following screenshot shows LDAP authentication starting for the protocol transition case:
   
3. In the following screenshot we see LDAP authentication succeed:
   
4. At this point, take a look at nskrb.debug. The following screenshot shows NetScaler obtaining TGT for aaauser:
   
5. NetScaler gets the service ticket for aaauser to the service sf.xmx.lab, as shown in the following screenshot:
   
6. Compare and look for any errors reported in this output if you are having a problem. A common one that comes up is KDC_ERR_S_PRINCIPAL_UNKNOWN, which means there is a duplicate SPN and the server is, as a result, unsure which key to use to decrypt the request. There is a list of these errors available at the Microsoft site: https://technet.microsoft.com/en-us/library/bb463166.aspx. You will need the SETSPN tool to troubleshoot these on the server.
   • A further validation of success or failure would be done with the means of counters:
     
   • If you are seeing no action in the aaad.debug file, verify that your authentication policies are configured correctly.
   • If you see entries in aaad.debug but S4U does not happen, verify that the client is not somehow falling back to NTLM by taking a trace and looking at the WWW-Authenticate header.
   • Another way to troubleshoot KCD step by step is to do it manually via the shell, using kgetcred. This will allow you to do a baseline verification by taking any browser or client machine particularities out of the picture. In the following screenshot, the individual steps are highlighted:
     
   • Look for Kerberos logs on the KDC; these can be very helpful too. To do this, head over to event viewer on the Active Directory Server and look for Kerberos logon attempts. Some of the event IDs to look for are listed in the following screenshot:
     

     Logon entries in event viewer

   • Kerberos logon attempts will show up as special logons; expanding one of them will show Kerberos-level details such as the following: