Chapter 19

Troubleshooting

So far you have read about various aspects of Configuration Manager 2012 R2, and in each chapter we have explained different aspects of the product and have assumed that every component will work correctly. However, everyone knows that situations may occur with the product that require troubleshooting, and every IT professional needs to know how to identify the problem and find a possible solution. There are many resources online that can assist you in this process; some of these places are Microsoft TechNet, Bing, and MVP blogs.

In this chapter, we will cover the basics of troubleshooting a ConfigMgr 2012 infrastructure and determining which log file you should look at first when a problem arises.

In this chapter, you will learn to

• Create a basic maintenance plan
• View log files using CMTrace
• Troubleshoot DRS replication
• Master the troubleshooting steps

Creating the Maintenance Plan

The best way to prevent issues from arising in Configuration Manager 2012 is to create and follow a standard maintenance plan. A well-executed maintenance plan allows administrators to be actively aware of their ConfigMgr 2012 hierarchy, offering a better chance of finding possible concerns before they become issues. Proactively watching the environment can reduce the number of major issues and thus keep your environment healthier.

ConfigMgr 2012 has some predefined site maintenance tasks that are enabled by default and some that need to be enabled. To view and modify the site maintenance tasks, take the following steps:

1. Open the ConfigMgr console, and choose the Administration workspace.
2. Expand Overview Site Configuration Sites, and select Site Name.
3. Right-click, browse to Site Maintenance, and click it to open the Site Maintenance window.

   By default, Microsoft has 14 predefined tasks for the CAS server and 37 predefined tasks for primary sites. Secondary sites have only 4 predefined tasks. Some of these predefined tasks are enabled, while others are disabled by default.

4. To enable a task, select its name and click Edit to display the task's Properties dialog box.

   Figure 19.1 shows the Properties dialog box for the Backup Site Server task.

5. Select Enable This Task.

   FIGURE 19.1 Enabling the Backup Site Server task in its Properties dialog box

   

To ensure the proper health of the site, you should enable some of the predefined site maintenance tasks provided by Microsoft, and you'll need to modify the schedule of other tasks. Table 19.1 lists how the tasks should be configured in a typical maintenance plan; this might not be appropriate for all environments, but you can use the table as a guide.

Some tasks do not need to be performed as frequently as others, but it's important to develop a maintenance plan for them to ensure the proper health of the site.

Table 19.2 summarizes what a typical maintenance plan should be for ConfigMgr 2012. Again, these recommendations might not be appropriate for organizations of all sizes; the table is meant to be used only as a guide.

Getting an approved maintenance plan for your hierarchy is your first step to developing a solid ConfigMgr infrastructure. The next step is to automate as many of the tasks as possible so they can be done in the most efficient manner without sacrificing the results. A maintenance plan is only as strong as the people implementing the plan.

Using Troubleshooting Tools

The most important troubleshooting tools are log files and status messages. Getting to know the most common log files for ConfigMgr is the first step in identifying a potential situation on your site servers; you can also use this proactively to monitor your site server activity and to know what is being processed in your environment. That being said, let's take a look at the most common log files.

Log Files

For the majority of troubleshooting, administrators will focus on the log (.log) files on either the client or the server and in some cases both. The client stores the log files in the folder C:Windowsccmlogs, while the site server stores the log files in the folder <installation Directory>Microsoft Configuration ManagerLogs. Table 19.3 lists site server log files, Table 19.4 lists management point log files, and Table 19.5 lists client log files.

The default location for the management point log files is Program FilesSMS_CCMLogs on the management point.

There are also log files outside ConfigMgr 2012 that you will need to be aware of when troubleshooting issues within Configuration Manager; Table 19.6 lists them.

There are many more log files for each ConfigMgr component. To learn more about the rest of the log files you can check out the ConfigMgr 2012 Library at http://technet.microsoft.com/en-us/library/hh427342.aspx. Here you can find details including the log file location and descriptions. Since ConfigMgr has many log files, we point out only the most common ones in this chapter.

The best tool for reading log files we have seen is CMTrace, a log viewer that constantly monitors the opened file for updates. You can find CMTrace in the SMSSETUPTools folder on the ConfigMgr 2012 R2 source media. It provides real-time updates of any log file, allowing administrators to see exactly what is happening on a client or site system. If that isn't enough to win you over, CMTrace includes the capability to highlight and filter features to allow at-a-glance log viewing. Finally, the tool includes an error code dictionary, shown in Figure 19.2, so that you can quickly translate most of Microsoft's decimal error codes into useful information right within the tool. For example, entering 5 in the Error Lookup window returns “Access Denied.” This feature is available by selecting Tools Lookup in the CMTrace utility. CMTrace will highlight errors, as shown in Figure 19.3, so they stand out.

The other tool you need is a reference to all the log files. We covered the log files and their locations earlier in this chapter, and this information is a great reference. So now you are well equipped to troubleshoot a ConfigMgr 2012 R2 site using the log files from the client or site systems, using the status messages in Configuration Manager, or using the web reporting feature of Configuration Manager.

Status Messages

Now that you have seen where the log files reside on both the client and servers, we will cover the troubleshooting components in the Configuration Manager 2012 R2 console. Microsoft has continued to provide status messages within the console.

Status messages reside in one location, which is why they provide some benefit when troubleshooting. They are arranged in one place so you can quickly see the health of the site at a glance. To view the site status, in the ConfigMgr console choose Monitoring Overview System Status Site Status.

The System Status dashboard is a summarized collection of the status of all the reported sites for easy viewing, and it has drill-down capability for each message. The System Status dashboard is organized into two categories, Component Status and Site Status, as shown in Figure 19.4.

To dig deeper into a Component Status message to see details about a problem, follow these steps:

1. In the ConfigMgr console, choose Monitoring Overview System Status Component Status.
2. Select the component you want to investigate. In this example, we will select the component SMS_DATABASE_NOTIFICATION_MONITOR.
3. Right-click the component, and select Show Messages All, as shown in Figure 19.5.

   FIGURE 19.5 The Show Messages menu

   

   This opens the Configuration Manager Status Message Viewer for <Site Code> <Site Name> window, as shown in Figure 19.6.

   Note that you can filter the returned messages by selecting All, Info, Warning, or Error messages.

4. Within the Configuration Manager Status Message Viewer window, double-click any of the messages to get a detailed view of the message, as shown in Figure 19.7.

   FIGURE 19.6 The Configuration Manager Status Message Viewer window

   

   FIGURE 19.7 The Status Message Details dialog box

   

USING WEB REPORTS TO VIEW STATUS MESSAGES

You can view status messages from the Web Reports tool within Configuration Manager 2012 R2. This is beneficial in case you are not able to access the ConfigMgr console or you have limited access to it. In either case, you can browse to your reporting point web console and view the status messages within a browser.

As discussed in Chapter 13, “Reporting,” web reports allow administrators to provide focused views of a ConfigMgr site database via a web browser and can be set up within a dashboard to give managers an at-a-glance overview of the hierarchy. Table 19.7 lists the reports available in the Status Messages category.

As you can see, Microsoft has provided some nice reports out of the box for status messages. Of course, you can create your own report or modify any of the existing reports to fit your needs.

DEPLOYMENT STATUS

Configuration Manager offers deployment status via the Monitoring workspace in the ConfigMgr console; you can monitor the deployment of all software (software updates, compliance settings, applications, task sequences, packages, and programs). To view this section of the console, do the following.

1. In the Configuration Manager console, choose the Monitoring workspace.
2. In Overview, choose Deployments.
3. To review general status information about an application deployment, select a deployment, and then choose the Summary tab of the Selected Deployment window.
4. To review deployment details for a compliance state and the devices in that state, select a deployment, and then, at the bottom of the screen in the Compliance Statistics section, click View Status.
5. To review information about the application's deployment type, select a deployment, and then click the Deployment Types tab of the Selected Deployment window.

Applications in Configuration Manager support state-based monitoring, which allows you to track the last application deployment state for users and devices. These state messages display information about individual devices. As you can see in Table 19.8, you can review the different states within the deployment.

You can view additional information for each compliance state, which includes subcategories within the compliance state and the number of users and devices in this category. For example, the Error compliance state contains three subcategories:

• Error Evaluating Policy
• Content Related Errors
• Installation Errors

When more than one compliance state applies for an application deployment to a user who has more than one device, you will see the aggregate state that represents the lowest compliance. For example, if a user logs into two devices, and the application is successfully installed on one device but fails to install on the second device, the aggregate deployment state of the application for that user is Error.

Use these subcategories to help you to quickly identify any important issues with an application deployment. You can also view additional information about which devices fall into a particular subcategory of a compliance state.

DISTRIBUTION STATUS

The Configuration Manager console provides improved content monitoring, including the status for all package types in relation to the associated distribution points, including the content validation status for the content in the package, the status of content assigned to a specific distribution point group, the state of content assigned to a distribution point, and the status of optional features for each distribution point.

The console provides a Content Status node in the Monitoring workspace (see Table 19.9). Inside this workspace you can review the information of the different packages and how many distribution points have been targeted. This is very similar to the old package status on Configuration Manager 2007. To monitor the content status and view the status, follow these steps:

1. In the Configuration Manager console, choose Monitoring.
2. In the Monitoring workspace, under Overview, expand Distribution Status, and then choose Content Status. The packages are displayed.
3. Select the package for which you want detailed status information.
4. At the bottom of the screen, in the Completion Statistics area, click View Status. Detailed status information for the package is displayed.

DISTRIBUTION POINT GROUP STATUS

This view can be found in the Monitoring workspace in the Configuration Manager console; here you can review information such as the distribution point group name, description, how many distribution points are members of the distribution point group, how many packages have been assigned to the group, distribution point group status, and compliance rate. You can also identify errors for the distribution point group, how many distributions are in progress, and how many have been successfully distributed. To perform this action, take the following steps:

1. In the Configuration Manager console, choose Monitoring.
2. In the Monitoring workspace, under the Overview, expand Distribution Status, and then choose Distribution Point Group Status. The distribution point groups are displayed.
3. Select the distribution point group for which you want detailed status information.
4. In the bottom section, in the Distribution Statistics area, click View Status.

   Detailed status information for the distribution point group is displayed.

DISTRIBUTION POINT CONFIGURATION STATUS

On this node you can review what attributes are enabled for the distribution point, such as PXE, multicast, and content validation, as well as the distribution status for the distribution point. To view this information, perform the following steps.

1. In the Configuration Manager console, choose Monitoring.
2. In the Monitoring workspace, under the Overview, expand Distribution Status, and then choose Distribution Point Configuration Status. The distribution points are displayed.
3. Select the distribution point for which you want distribution point status information.
4. In the results pane, click the Details tab.

   Status information for the distribution point is displayed.

SYSTEM STATUS

The System Status home page will highlight all the site systems within your Configuration Manager infrastructure and show a summary of the systems. Table 19.10 displays the information shown when you browse to the System Status home page by choosing the Monitoring workspace Overview System Status in the Configuration Manager console.

Because of the comparatively slower replication speed of status messages, most ConfigMgr administrators will focus on the log files of the client, the server, and often a combination of both. Because there are so many log files, to keep track of which log file contains the information needed for troubleshooting every ConfigMgr administrator needs two tools:

Troubleshooting Configuration Manager Deployment

If you experience issues deploying ConfigMgr 2012, you should first look into the following areas to see whether the issue is related to permissions, disk space, network connectivity, or timing:

Troubleshooting Configuration Manager Database Replication

Database Replication in Configuration Manager 2012 R2 (shown in Figure 19.8) is based on the Data Replication Service (DRS). This depends on two SQL features: SQL Server Service Broker and Change Tracking; these have nothing to do with transactional replication. This replication is very important in ConfigMgr, and for this reason you need to ensure that this replication is working at all times. SQL Server Service Broker manages internal and external processes that can send and receive guaranteed async messages by using a data manipulation language. Messages can be sent to a queue in the same database as the sender, to another database in the same SQL instance, or to another SQL instance on a remote server. To better understand this concept you can visit the following site: http://technet.microsoft.com/en-us/library/gg712701.aspx#database_replication.

To look at the status of the current database replication, choose the Monitoring workspace Overview Database Replication.

A SQL communication link is a logical entity that is used to reflect the overall status of SQL communication between two sites. The link reflects the overall status of global and site data being replicated between two sites.

Each global and site data replication link will have one of the following states:

Data Types

There are various data types in Configuration Manager. Objects that will be replicated in Configuration Manager are based on these types.

The type of data generated at the Central Administration Site and primary sites and replicated across the hierarchy is called global data. Since this data is globally available, it can be modified or deleted from the Central Administration Site or any primary site, regardless of where it was created, provided proper Role-Based Access Control (RBAC) permissions are configured.

The following elements are part of global data:

• Collections
• Packages
• Programs
• Deployments
• Configuration items
• Software updates
• Task sequences
• OS images (boot images, driver packages, and the like)
• Site control file
• System resource list (site servers)
• Site security objects (roles, scopes, and so on)
• Client authentication
• Client discovery

Global data replication is built on the Service Broker infrastructure provided by SQL Server.

Primary sites generate site data; it is potentially replicated to the Central Administration Site but never replicated between primary sites. You can view this process in Figure 19.9. Since this data is visible only at the Central Administration Site and the primary site where the data originated, it cannot be modified or deleted from other primary sites. If the data needs to be modified, it can be modified only at the originating site.

The following elements are part of site data:

• Collection membership
• Alerts
• Hardware inventory
• Software inventory and metering
• Status messages
• General site data
• Asset Intelligence CAL track data
• Software distribution status details
• Software updates replicated site data
• Software updates non-replicated site data
• Status summary data
• Component and site status summarizers
• Client health data
• Client health history
• Wake On LAN
• Quarantine client restriction history

  FIGURE 19.9 Replication flow

  

DRS Initialization

The process of DRS initialization is as follows:

1. A receiving site sends an init request to the sending site for the required replication group.
2. The sending site uses bcp (Bulk Copy Program) to export all the data from the tables in the replication group. During this phase, you can open the RCMCtrl.log file, as shown in Figure 19.10.
3. Hierarchy replication copies the BCP files and sends a count of each row for each table to the receiving site inbox via the senders. To view the details about this process, see Figure 19.11.

   FIGURE 19.10 RCMCtrl.log viewed in CMTrace.exe

   

   FIGURE 19.11 Replication process

   

By default ConfigMgr starts with the Replication Configuration Data group, which contains the data about other groups to be replicated.

To evaluate this process you can then open the RCMCtrl.log at the primary site or CAS (Figure 19.10).

You can also open SQL Server Management Studio and expand the Configuration Manager database Programmability and locate the spDiagDRS stored procedure. Executing this stored procedure will show the different statuses of the data types and the link status between sites.

Understanding these processes is key to troubleshooting DRS replication in Configuration Manager. Be sure to review this process and learn how the DRS and stored procedures are executed in this process for a successful replication.

Additional Tools

One other set of tools available to you for troubleshooting is the System Center 2012 R2 Configuration Manager Toolkit, available at www.microsoft.com/en-us/download/details.aspx?id=36213. This toolkit contains 15 tools to help you manage and troubleshoot ConfigMgr 2012 R2. We cover the tools next, divided into server-based tools and client-based tools.

Server-Based Tools

DP Job Manager (DPJobManager) helps you troubleshoot and manage ongoing content distribution jobs to ConfigMgr distribution points, as shown in Figure 19.12.

With DP Job Manager open, you enter the name of the primary site server where those distribution points are connected. You can then choose the Overview, Distribution Point Info, Manage Jobs, or Help tabs to get additional details about the queues.

Collection Evaluation Viewer (CEViewer) assists you in troubleshooting collection evaluation–related issues by viewing collection evaluation details, as shown in Figure 19.13.

Once you're connected to the primary site, you will be able to review the current collections. For example, as you can see in Figure 19.13, from the current collections that are performing a full evaluation in this list, you can determine the collection name, site id, run time, last evaluation time, next evaluation time, member changes, last member change time, and percent of members. All this information is useful when troubleshooting collections. Besides the full evaluation, there are other tabs that will help on this troubleshooting. Here are more details about each tab.

• Incremental Evaluation will show you all the collections that have been marked with the Use Incremental Updates For This Collection.
• All Queues gives you a dashboard of what is currently running on all the queues.
• Manual Queue is for collections that have been manually selected for evaluation from the administrative console UI.
• New Queue is for newly created collections.
• Full Queue is for collections due for full evaluation.
• Incremental Queue is for collections with incremental evaluation.

This is a very good tool that will help you address the question of why a collection has not updated.

Content Library Explorer assists you in troubleshooting issues with and viewing the contents of the content library, as shown in Figure 19.14.

Once you open Content Library Explorer, it will ask you to connect to the distribution point by entering its FQDN. Once you're connected to the distribution point, you will be able to browse the content library and select the package ID you are trying to troubleshoot.

You can use the Security Configuration Wizard Template for System Center 2012 R2 ConfigMgr to help secure the site server. The wizard is an attack-surface reduction tool for the Microsoft Windows Server 2008 R2 operating system. Security Configuration Wizard determines the minimum functionality required for a server's role or roles and disables the functionality that is not required. An example of this is shown in Figure 19.15. With this file you can then use the Security Configuration Wizard (SCW) to apply the .xml file to the site server by running the following command line: scwcmd register /kbname:ConfigMgr /kbfile:c:program files (x86)ConfigMgr 2012 toolkitServerToolsConfigMgr2012SCW.xml

Content Library Transfer is a tool that transfers content from one disk drive to another, as you can see in Figure 19.16.

You can imagine a distribution point whose drive is filling up quickly, and you need to move the content from one drive to another but don't want to rebuild the distribution point. Instead, you can move the content from one drive to another by entering the following command: ContentLibraryTransfer.exe -SourceDrive X: -TargetDrive Y. Once the command finishes, the content library will be located on the new drive.

The Content Ownership Tool changes the ownership of orphaned packages (packages without an owner site server), as shown in Figure 19.17.

If you select Only Orphaned Packages, all orphaned packages will be displayed in the list. You can also choose All Packages to see all the package names. Once you find an orphaned package, you can click Change Site Ownership To and specify the new site; then click Apply.

The Role-Based Administration Modeling and Auditing Tool (RBA Viewer) helps administrators model and audit RBA configurations, as shown in Figure 19.18.

RBA Viewer is a great tool to use to model the security role; more importantly, you can audit a user by clicking the green play button at the top of the screen. Then all you need to do is enter the user information and click Check. This will display the current assignments and console view.

The Run Metering Summarization Tool (runmetersumm) runs the metering summarization task to analyze raw metering data, as shown in Figure 19.19.

You can use this tool to update the software metering data. This needs to run at the primary site server where the data is going to be inserted. Once that's completed, you will be able to run the software metering reports with no problem.

Client-Based Tools

Client Troubleshooting Tool (CliSpy) is a tool that helps you troubleshoot issues related to software distribution, inventory, and software metering on ConfigMgr clients, as shown in Figure 19.20.

To use this tool, you must click Tools and connect to the client you are currently troubleshooting. Once you're connected, you will have the option to review each tab and identify the potential problem on the client.

Configuration Manager Trace Log Viewer is a tool used to view log files created by ConfigMgr components and agents. This tool was covered earlier in this chapter and is shown in Figure 19.3; it is the day-to-day tool of every ConfigMgr administrator.

The Deployment Monitoring Tool is a graphical user interface designed to help troubleshoot applications, updates, and baseline deployments on ConfigMgr clients, as shown in Figure 19.21.

Once open, it will connect by default to the current computer. To connect to another computer, you must click Actions and then click Connect To Remote Machine. Once you have chosen the computer to troubleshoot, you can click Client Properties to review the information of the client. You can click Deployments to validate what deployments have been completed on the machine or are optional to the machine. This will also include baselines deployments, as mentioned earlier. You can also click All Updates; this status will display all the current installed or missing.

Policy Spy is a policy viewer that helps you review and troubleshoot the policy system on ConfigMgr clients, as shown in Figure 19.22.

You can use the Power Viewer Tool to view the status of the power-management feature on ConfigMgr clients, as shown in Figure 19.23.

The Send Schedule Tool is used to trigger a schedule on a client or trigger the evaluation of a specified DCM baseline. You can trigger a schedule either locally or remotely, as shown in Figure 19.24.

Wakeup Spy is a tool that provides a view of the power state of ConfigMgr client computers, as shown in Figure 19.25.

The Bottom Line