Chapter 22
Investigating User Issues

  • images Objective 4.3: Given a scenario, analyze and troubleshoot user issues.

images An old troubleshooting technique is to break a problem in half: if the cause is not in the first half, it’s in the second half. Break the second part in half and continue to analyze. In this chapter, we’re splitting user issues into system access and file difficulties. We further divide these problem categories while stepping you through various items to consider as you move toward a solution.

Troubleshooting Access

If a user is having trouble accessing their desired applications, a mixed bag of authentication issues can be the cause. We’ll take a look at local and remote access as well as authentication scenarios.

Local

Local access refers to those users who are using a directly connected interface to the server. These are typically server administrators but may be application users as well. Begin troubleshooting by gathering some basic information:

  • Is this a newly created user account?
  • What is the username being entered for the account?
  • Has the user ever logged into the account?
  • Is the user attempting to log in via the GUI or a text-based (virtual) terminal (for example, tty2)?

Checking a Newly Created User Account

If the account is a newly created account, confirm that it was properly built. New system administrators often create user accounts with the useradd command (see Chapter 10) but forget to add its password with the passwd utility. Use either the grep or getent command to check the /etc/passwd and /etc/shadow file records. An example is shown in Listing 22.1 for a new user account, JKirk, on an Ubuntu Desktop distribution.

Listing 22.1: Viewing a user account records with the getent command

$ sudo getent passwd JKirk
JKirk:x:1002:1002::/home/JKirk:/bin/bash
$
$ sudo getent shadow JKirk
JKirk:!:17806:0:99999:7:::
$

Notice that in the password field for the JKirk shadow record, there is an exclamation mark (!). This indicates a password was not created for the account.

images Make sure your system users know that usernames are case sensitive on Linux. Other operating systems, such as Windows, have usernames that are case insensitive, and this can cause confusion.

Checking Account Accesses

Look at the last time the account was successfully accessed. The lastlog utility searches through the /var/log/lastlog file for users who have logged into the system. But it only maintains the most recent login. The last command searches the /var/log/wtmp for users that have logged in/out and keeps records for the most recent logins and beyond. This file is typically rotated with earlier versions and given a numeric extension. Thus, the next oldest version of wtmp is wtmp.1. The last command’s -f option will help you search through the various files, as shown in Listing 22.2.

Listing 22.2: Viewing successful logins with the last command

$ sudo last -f /var/log/wtmp -f /var/log/wtmp.* | grep JPicard
JPicard  tty3                          Wed Jan  2 13:14 - 13:14  (00:00)
$
$ sudo lastlog -u JPicard
Username         Port     From             Latest
JPicard          tty3                      Wed Jan  2 13:14:11 -0500 2019
$

Notice the last command shows that the JPicard account was logged into on January 2nd at the tty3 terminal and is confirmed via the lastlog command.

You can also employ the lastb command. This command allows you to search for unsuccessful login attempts as shown in Listing 22.3. Notice that the JPicard account had two failed login attempts at the tty4 terminal.

Listing 22.3: Viewing unsuccessful login attempts with the lastb command

$ sudo lastb -f /var/log/btmp -f /var/log/btmp.* | grep JPicard
JPicard  tty4                          Wed Jan  2 13:29 - 13:29  (00:00)
JPicard  tty4                          Wed Jan  2 13:29 - 13:29  (00:00)

Checking GUI Issues

If you find that the account successfully logged into the system in the past and has no recent failed attempts, find out the user’s local access method. If using the GUI, have the user attempt to log into a text-based terminal, such as the tty2 terminal. If the user cannot successfully log into a terminal, then something else is amiss. The next few chapter sections after this one will help.

If the user can successfully log into a text-based terminal but not the GUI, you’ve narrowed down the problem. Determine what services are running. For older SysVinit systems, the runlevel command will show if the graphical environment is current (see Chapter 6). For systemd systems, use the command shown in the snipped example in Listing 22.4.

Listing 22.4: Viewing the current systemd target via the systemctl command

$ sudo systemctl status graphical.target
•graphical.target - Graphical Interface
[…]
   Active: active since Wed […]
[…]

The graphical.target is active, which indicates GUI services are available. Thus, providing GUI services is not the problem. Begin investigating other potential GUI issues starting with the display manager (Chapter 8).

images If your system does not use a graphical environment, check to see if multiple users are allowed to log into the system. For systemd systems, check if the multi-user.target is active. See Chapter 6 for SysVinit systems.

Checking Terminal Issues

If the user typically logs into a text-based terminal and cannot, have them log into a different terminal. If the login is successful, then look at the original terminal’s device file to determine if it is corrupted, using the ls -l command. An example is shown snipped in Listing 22.5.

Listing 22.5: Viewing terminal device files with the ls -l command

$ ls -l /dev/tty?
crw--w---- 1 root  tty 4, 0 Jan  2 09:38 /dev/tty0
crw--w---- 1 gdm   tty 4, 1 Jan  2 09:38 /dev/tty1
[…]
crw--w---- 1 root  tty 4, 9 Jan  2 09:38 /dev/tty9
$

Notice the c at the beginning of each terminal’s device file record. This indicates the device file is a character file. If you see a dash (-) instead, the file is corrupt. Rebuild it using super user privileges and the mknod command.

images If you are attempting to log into the root account or your own system administrator account and have forgotten the password, follow the guidelines in Chapter 20’s “Surviving a Lost root Password” section.

If the user attempts to log into a different text-based terminal and cannot, check to see if getty services are running. These services provide the login prompts for the text-based terminals. An example of checking for getty services on a systemd system is shown snipped in Listing 22.6.

Listing 22.6: Checking for getty services with the systemctl command

$ sudo systemctl status getty.target
•getty.target - Login Prompts
[…]
   Active: active since Wed 2019-01-02 09:38:13 EST; 6h ago
[…]

The getty.target is active, so getty services are available. If a user still cannot log into the system, you’ll need to explore additional issues.

images Check the /etc/security/access.conf file. This file is scanned when a user attempts to log into the system. Its configuration accepts or blocks users/groups from accessing the system. It can also prohibit certain logins to the text-based terminals, as well as logins originating over the network.

Checking Additional Local Issues

Determine if the account is locked. You can employ the passwd -S or the getent command to check this as shown snipped in Listing 22.7.

Listing 22.7: Checking if an account is locked with the passwd and getent commands

$ sudo passwd -S KJaneway
KJaneway L 01/02/2019 0 99999 7 -1
$
$ sudo getent shadow KJaneway
KJaneway:!$6$[…]0:17898:0:99999:7:::
$

The L after the user KJaneway account’s name indicates the account is locked. However, that code is also shown for accounts that have no password set. Thus, the getent command is also employed. The exclamation point (!) at the front of the account password’s field verifies that the account is indeed locked. To unlock the account, if desired, use super user privileges and the usermod -U or the passwd -u command.

images Check the user’s keyboard. Sometimes incorrect keyboard mappings or corrupt hardware can cause wrong characters to be sent to authentication programs.

The account may have expired. Account expiration dates are typically set up for temporary account users, such as contractors or interns. You can view this information using the chage command as shown snipped in Listing 22.8.

Listing 22.8: Checking if an account is expired with the chage command

$ date
Wed Jan  2 16:17:48 EST 2019
$
$ sudo chage -l JArcher
[…]
Account expires             : Jan 01, 2019
[…]

Notice that this account’s expiration date has passed. Therefore the JArcher account is now expired and the user cannot log into it. If this was a mistake or you need to modify it, use super user privileges and the chage -E command to set a new expiration date for the account.

Confirm the user is using the correct password, and check if the account’s password is expired. Employ the chage -l command to view this as well.

Remote

For remote access problems, first check that the system is accessible over the network (network troubleshooting was covered in Chapter 20). If the system is accessible, determine how the user is attempting to access the system.

If the user is employing OpenSSH, first confirm that the OpenSSH server is running on the system and the firewall is properly configured to allow access (firewalls were covered in Chapter 18). Next review the sshd_config configuration file. The AllowUsers and AllowGroups directives restrict access. Ensure that these are correctly set. In addition, verify that there are no specific override settings for this particular user at the file’s bottom. Be sure to review any configuration files on the client-side as well, such as ~/.ssh/config and /etc/ssh/ssh_config (OpenSSH was covered in Chapter 16).

images Have the user tack the -vvv option onto their ssh command. This provides a great deal of verbose information, which may help in the troubleshooting process.

Determine if authentication through OpenSSH is via a username and password or via a token. If it is a username/password, check that the sshd_config directive PasswordAuthentication is set to yes. If all is well with the configuration file, troubleshooting topics in the next section may help.

If the OpenSSH authentication is token based, ensure that the private key was properly copied over to the server from the remote system. Also, confirm that the public key is stored in the ~/.ssh/authorized_keys file on the client side.

images If the user’s X11 GUI is transferred over the network, make sure the ForwardX11 directive is set and that the user is employing the -X option with the ssh command. See Chapter 8 for additional details.

If your users are employing a remote desktop application, such as VNC, Xrdp, NX, or SPICE, review their configurations. More information can be found on these topics in Chapter 8.

Authentication

Layered authentication software could be at the problem’s heart. One place to check is PAM (covered in Chapter 16). Look through the PAM configuration files, such as /etc/pam.d/sshd, to ensure that directives are properly set. Also employ the pam_tally2 or faillock utility to check if the user’s account is locked due to too many failed login attempts.

Does your system employ other authentication products, such as LDAP (covered in Chapter 16) or Kerberos (covered in Chapter 19)? You’ll want to check their configuration files. Also take a look through their log files to ensure that a policy violation has not locked out the user’s account.

Don’t forget to check your Linux kernel security module, such as SELinux or AppArmor (covered in Chapter 15). While the purpose of these modules is to protect the system from attackers, sometimes policy violations can lock out legitimate users. For AppArmor, policy violations are stored in either audit.log (produced by auditd) or messages.log, depending upon its configuration. If using the auditd service, you can search for AppArmor policy violations in the audit.log file using the ausearch command.

For SELinux, check the audit log file using the sealert command as shown in Listing 22.9. Notice that no SELinux policy violations were logged.

Listing 22.9: Checking SELinux policy violations with the sealert command

# sealert -a /var/log/audit/audit.log
100% done
found 0 alerts in /var/log/audit/audit.log
#

images For SELinux, use the id -Z command to view a user’s SELinux context. See Chapter 15 for additional helpful SELinux and AppArmor commands.

For authentication issues, peruse through these various log files as well:

  • /var/log/auth.log (Debian-based distros)
  • /var/log/secure (Red Hat–based distros)

These log files typically contain information such as authentication failures and can provide you with a great deal of information. Use grep to search for a particular username, if desired.

Examining File Obstacles

Various problems may ensue when managing files. Understanding the typical problems will help in your file troubleshooting efforts.

File Permissions

You’re editing a configuration file using the vim editor, but when you try to save the file, you get a “Can’t open file for writing” error message. That’s frustrating. Problems like this are directly related to file permissions.

When encountering these issues, first use the ls -l command to view a file’s permission settings and ownerships (covered in Chapter 15). Note the file’s owner and group. Then determine the permissions of the owner, group, and everyone else (world or other). An example is shown in Listing 22.10.

Listing 22.10: Determining file permissions with the ls -l command

$ ls -l HelloWorld.sh
-rwxr-xr--. 1 root wheel 72 Jan  4 09:29 HelloWorld.sh
$

In this case, the file is owned by root, who has permission to read, write to, and execute the file. The file’s group is wheel. Those members can read and execute the file. Finally, everyone else who is not the file’s owner nor belongs to the file’s group can only read the file.

If a problem occurs trying to access this file, determine the user’s username and group memberships using the id command. Match their identity against the file’s permissions to find the problem. An example is shown in Listing 22.11.

Listing 22.11: Troubleshooting file access with the id command

$ whoami
Christine
$
$ ./HelloWorld.sh
-bash: ./HelloWorld.sh: Permission denied
$
$ id Christine
uid=1001(Christine) gid=1001(Christine) groups=1001(Christine)

Christine cannot execute this file because she is not the file’s owner, nor is she in the file’s group (see Listing 22.10). The third permission set applies to her and only allows her to read the file. To let her run the file, add her to the wheel group, set her as the file’s owner, or modify the file’s world permissions.

Directory Permissions

While the directory permission settings look very similar to file permissions, their effect is different. Table 22.1 shows permission effects on actions (covered in Chapter 15) associated with whom the permission applies (owner, group, or other).

Table 22.1 Directory permission effects

Permission Effect
r Allows user to display directory’s files
w Allows user to create, move (rename), modify attributes, and delete files within the directory
x Allows user to change their present working directory to this directory (via the cd command) as long as this permission is set in all parent directories as well

A few examples will help to understand these permissions. For example, the user Christine can list the /etc directory’s contents, due to the directory’s read permission for other (world), as shown snipped in Listing 22.12.

Listing 22.12: Understanding directory permission effects of the read privilege

$ whoami
Christine
$ ls -ld /etc
drwxr-xr-x. 143 root root 12288 Jan  4 09:22 /etc
$
$ ls /etc
abrt                        gshadow-                  profile
adjtime                     gss                       profile.d
[…]

The next example shows that the parent directory (/home) does allow a user to use cd to change their present working directory to it. However, a subdirectory (Samantha) blocks it via not granting the execute (x) privilege (see Listing 22.13).

Listing 22.13: Understanding directory permission effects of the execute privilege

$ whoami
Christine
$ ls -ld /home/
drwxr-xr-x. 5 root root 52 Nov  8 14:31 /home/
$ cd /home
$ pwd
/home
$
$ ls -ld /home/Samantha
drwx------. 5 Samantha Samantha 128 Nov  8 14:38 /home/Samantha
$
$ cd /home/Samantha
-bash: cd: /home/Samantha: Permission denied
$

 

images If a subdirectory does grant execute permission but one of its parent directories does not, a user will not be able to use cd to get to that subdirectory. All directories in that path must have execute permission set on them for that user.

If you have a directory shared among users and they are able to delete each other’s files but that is not desired, employ the sticky bit on the directory. This permission (covered in Chapter 15) will solve the problem.

File Creation

A user goes to create a file and permission is denied. First check the directory permissions. If all is well there, consider the following items.

Are quotas enforced on this partition? The user may have hit a quota limit. Check the partition’s /etc/fstab record for either usrquota or grpquota to see if quotas are enabled on this filesystem (covered in Chapter 10) or use the repquota utility. If quotas are enabled, check the user’s (or group’s) quota usage via the quota command. If the user has exceeded quota limits and the grace period has passed, turn off quotas via quotaoff, extend the user’s quotas, or have the user delete unneeded files.

Has the disk run out of space? The df command (covered in Chapter 20) can help you quickly determine if the partition on which the user is attempting to create a file has run out of space. If you are out of space, take the appropriate action (also covered in Chapter 20).

Has the partition run out of inodes? Inode exhaustion is an unusual situation, but it does happen. An inode number (covered in Chapter 3) is a file’s associated index number. If a filesystem runs out of inodes, no additional files can be created on it. Check if this is the problem by employing the df -i command, which shows each filesystem’s inode use.

If inodes are exhausted, you cannot extend them after filesystem creation. Instead, look for directories that have a large amount of small unnecessary files, such as a temporary directory, and remove what you can. A good practice is to put applications that create many small files on their own large partitions formatted to provide higher amounts of inode numbers. At filesystem creation, increase inode counts above their defaults via utility options like the mke2fs -i command.

What is the user’s umask setting? The umask setting (covered in Chapter 15) subtracts permissions from the default file and directory permission settings. Thus, when a user is creating files and/or directories, the permissions set on them are affected by this setting. Have the user enter umask at the command line (or check the user’s environment file) to determine its setting. If it is set too high (for example, removing needed write permissions for created subdirectories), you have found the cause of problem.

images If a file cannot be deleted or renamed, check the file’s attributes via the lsattr filename command. If you see an i among the dashes preceding the filename, the file has the immutable bit set. This bit prevents even users with super user privileges from deleting the file. To remove the file’s immutable bit, use super user privileges and issue the chattr -i filename command.

Just as with access issues, check your Linux kernel security module (SELinux or AppArmor) in situations where a user cannot create or delete files. Peruse the appropriate policy violation log files (/var/log/messages or /var/log/audit/audit.log) using the appropriate tools.

images If your system uses SELinux but does not employ auditd, the SELinux policy violation messages are stored in the /var/log/messages file.

After you have found the policy violation concerning the file or directory in question, determine if the file/directory was mislabeled, the policy is incorrect, or possibly the wrong security context was used. Take appropriate actions to remedy the situation.

Exploring Environment and Shell Issues

When dealing with user problems, a potential issue is the user account’s default shell. Check it using the getent command, as shown on an Ubuntu distribution in Listing 22.14.

Listing 22.14: Determining an account’s default shell with the getent command

$ getent passwd BSisko
BSisko:x:1007:1007::/home/BSisko:/bin/sh
$
$ readlink -f /bin/sh
/bin/dash
$

Notice that instead of the /bin/bash shell as the default shell, the /bin/sh shell is used. On this system, the /bin/sh file is linked to the /bin/dash shell. If this is not desired, you can change the user’s default shell via the usermod command (see Chapter 10).

images If a user cannot log into the system, check to see that their default shell is set to /sbin/false, /sbin/nologin, or something similar. These shells are typically for daemons and prevent the daemon from logging into the system.

Incorrect or improperly set environment variables can cause various user difficulties. Review a user’s environment files (covered in Chapter 10), such as the ~/.profile file. While you’re at it, check the system’s global environment variable settings via the set, env, or printenv command.

Ensure that environment variables are exported so they are available in subshells (a subshell may occur when a shell script is executed.) An example of this problem is shown in Listing 22.15.

Listing 22.15: Demonstrating what happens when variables are not exported

$ EDITOR='nano'
$ echo $EDITOR
nano
$
$ bash
$
$ echo $EDITOR
$
$ exit
exit
$

The user sets the environment variable EDITOR so that their default text editor is now nano. However, when a new subshell is created via the bash command, the setting is lost. To keep the setting persistent, it needs to be exported as well as stored in a user’s environment file. An example of exporting the variable is shown in Listing 22.16.

Listing 22.16: Demonstrating the export command

$ export EDITOR='nano'
$
$ echo $EDITOR
nano
$
$ bash
$
$ echo $EDITOR
nano
$

It is also important to make sure you have environment variables in the correct file, global or per user. Also, you must understand what type of login the user conducts in order to set these variables in the right file (see Chapter 10).

Summary

User issue troubleshooting is tricky. Many components are involved in access, authentication, file management, and the user’s shell and environment. Understanding common problems will assist in fixing an issue.

Exam Essentials

Summarize user access problems/solutions. For impeded local access, research corrupt terminal files, improperly configured GUI components, and expired passwords/accounts. Remote access problems are often caused by misconfigured OpenSSH components or remote desktop applications. Other issues can involve layered authentication software such as PAM or a system’s kernel security module, such as SELinux or AppArmor.

Describe various file problems/solutions. File access and management requires understanding of basic file and directory permissions as well as ownership and group membership. Additional system items to review include filesystem quotas, disk space, inode use, and umask settings. Check the kernel security module log files for policy violations as well. If a user cannot delete a file, look for the immutable bit set on the file.

Explain user environment and shell issues. Improperly configured environment variables or ones that are not exported will cause user problems. Examine the various environment files, both global and user, for issues. Difficulties may also arise from the user account’s default shell setting.

Review Questions

  1. Lamar, a contractor, claims he cannot log into his account locally. He was able to do so yesterday. No one else seems to be having problems accessing the system. What should you check first?

    1. Check if GUI services are running via the systemctl command.
    2. Look at the OpenSSH server configuration files.
    3. Determine if his account is expired via the chage command.
    4. See if the account is locked via the faillock utility.
    5. Check for policy violations in the SELinux log files.

  2. Irene normally logs into the system locally via the tty4 terminal but cannot today. She tries her authentication at the tty3 terminal and logs in successfully. What should you check first?

    1. Determine if getty services are running via the systemctl command.
    2. Review access rules in the /etc/security/access.conf file.
    3. See if the account is locked via the passwd -S command.
    4. Use the last command to see when she last logged in.
    5. Check if the tty4 device file is corrupt via the ls -l command.

  3. Vincent is attempting to remotely log into the system using OpenSSH without success. He asks you what he can do to help troubleshoot this problem. What should you recommend first?

    1. Check the /etc/ssh/sshd_config configuration file.
    2. Add the -vvv option on to Vincent’s ssh command.
    3. Add the -X option onto Vincent’s ssh command.
    4. Confirm Vincent’s public key is stored in the ~/.ssh/authorized_keys file.
    5. Check the ~/.ssh/config configuration file.

  4. Anton is struggling to determine why a particular user cannot log into a CentOS system, where SELinux is disabled and auditd is not used. Which of the following are the best log files to peruse? (Choose two.)

    1. /var/log/audit/audit.log
    2. /var/log/messages
    3. /var/log/auth
    4. /var/log/secure
    5. /var/log/lastlog

  5. Tarissa needs to run a shell script, which has the permissions of rwxr--r--, is owned by root, and belongs to the wheel group. Tarissa’s user account is T2T1000, and she is a member of the admin group. What can be done to allow her to run this script? (Choose all that apply.)

    1. Add Tarissa to the wheel group.
    2. Create a new account for Tarissa named wheel.
    3. Add w to the script file’s group permissions.
    4. Add x to the script file’s group permissions.
    5. Nothing. Tarissa can run the script now.

  6. Miles needs to change his present working directory to the /home/miles directory. He does not own the directory, nor is he a member of its group. Assuming needed parent directory permissions are set, what needs to take place for this to successfully occur?

    1. Nothing. The /home/miles directory is Miles’s home directory, so he can access it by default.
    2. The execute (x) permission needs to be added.
    3. The write (w) permission needs to be added.
    4. The read (r) permission needs to be added.
    5. The dash (-) permission needs to be added.

  7. Sarah, a system administrator, attempts to create a file and receives an error message indicating the file cannot be created. Which of the following might be the problem? (Choose all that apply.)

    1. The filesystem on which she is attempting to create the file has quotas set, and she is past her quota and grace period.
    2. The filesystem on which she is attempting to create the file has run out of space.
    3. The file that she is attempting to create has the immutable bit set and therefore cannot be created.
    4. The action is triggering either a SELinux or an AppArmor policy violation.
    5. The filesystem is experiencing inode exhaustion and therefore cannot accommodate any new files.

  8. A user cannot delete one of her files but is able to delete other files in her directory. John, a system admin, is attempting to troubleshoot this issue. What command should he use first on the file?

    1. chown
    2. chattr
    3. chmod
    4. umask
    5. lsattr

  9. Melissa wants to set her default editor to the vim editor and wants this to stay set when she enters a subshell. What should she do?

    1. Put EDITOR='vim' in the /etc/profile file.
    2. Put export EDITOR='vim' in the /etc/profile file.
    3. Put EDITOR='vim' in her ~/.profile file.
    4. Put export EDITOR='vim' in her ~/.profile file.
    5. Put export EDITOR='vim' in her ~/. bash.bashrc file.

  10. Mark Watney, a system administrator, has his account, MW2015, modified by a new system admin intern. When Mark logs into the system and tries to group a list of commands, it no longer works. No one else is having this problem. He suspects his account’s default shell has been changed from /bin/bash to /bin/tcsh. Which of the following will help determine if his suspicion is correct? (Choose all that apply.)

    1. cat /etc/profile
    2. echo $SHELL
    3. sudo grep tcsh$ /etc/passwd
    4. sudo getent shadow MW2015
    5. sudo getent passwd MW2015
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset