Exploring Common Issues

If you are fairly new to Linux system administration, most likely you are unaware of common storage problems. The following can help you prepare.

Degraded Storage/Mode Degraded storage refers to the storage medium’s gradual decay due to time or improper use, which causes data degeneration or loss. For example, an SSD has limited endurance due to its finite number of program/erase (PE) cycles. Thus, employing an SSD in your swap space is unwise.

Degraded mode refers to a situation in which one or more disks in a RAID array have failed. In this case, troubleshooting efforts require you to employ the mdadm -D command to view a particular array’s detailed status. If the state contains the word degraded, add another partition to the array so it can recover.

Missing Devices Storage devices can go “missing” on Linux, but the cause varies. If it is network attached storage (NAS), check your network first (see Chapter 20).

If it is a locally attached device and other utilities, such as lsblk, are not displaying it, use super user privileges and try the lspci -M command. This command will perform a thorough scan of all PCI attached devices.

The conduit to Linux devices is through the device files, such as /dev/sdb. Ensure that the particular partition’s device file is available and not corrupted. If needed, rebuild it via the mknod command.

Check that you (or the utility configuration) are using the correct device file name. A whole disk is referred to by the device file name with no numbers, such as /dev/sdc. A disk partition is specified by the device file name and its number, like /dev/sdc2. When using an NvME SSD, the device file name, such as /dev/nvme0n1p1, has extra items, including the namespace.

Missing Volumes Another form of a lost device is a missing volume. If you perform a pvscan on the physical devices that make up a logical volume and get a “Couldn’t find device” message, you’ve got a missing volume. Typically, the cause is a failed or unintentionally removed disk.

If a disk that was part of a logical volume’s group has failed, the missing disk’s UUID will display in the pvscan message. You can replace the failed volume (pvcreate), restore the group’s metadata (vgcfgrestore), recover the group (vgscan), and then activate it (vgchange) via LVM tools.

Missing Mount Points A “Mount point does not exist” error message implies the obvious—the directory on which you are attempting to mount the filesystem does not exist. It either was deleted or never created in the first place. Simply make it with appropriate privileges via the mkdir command.

However, this error message can also be generated for a not-so-obvious problem. It centers on employing the bind option, either at the command line via the mount command or in the /etc/fstab file. This option moves a filesystem from its current mount point to a new mount point. If it is not already mounted somewhere, you’ll get a “Mount point does not exist” error message.

Storage Integrity A bad block (also called a bad sector) is a small chunk of a disk drive that will not respond to I/O requests due to corruption or physical damage. A random bad block does not indicate a drive is failing, but these storage devices need monitoring, because increasing bad sectors indicate it needs replacing.

Besides using the fsck command (covered in Chapter 11), you can employ the badblocks utility to monitor a drive. It is different from fsck in that it focuses on a particular partition and does not perform any repairs. It is wise to back up and unmount a partition prior to checking for bad sectors. Use the non-destructive test, by issuing the badblocks -nsv partition-device-file command. The utility provides progress as it runs, and when the tests are complete, it issues a final bad blocks status.

In addition, a disk’s manufacturer often provides its own set of testing programs. Typically these programs let you know whether or not to replace the drive but do not provide detailed data on bad sectors.

Performance Issues Poor storage performance adversely affects applications. Besides using utilities such as iostat, ioping, iotop, and sar (covered in Chapter 20) to monitor storage performance problems, you can also employ hdparm to determine a drive’s read speeds. This utility is useful for PATA or SATA drives. SCSI drives that have SCSI/ATA command translation are also supported.

The dstat utility is similar to iostat but provides additional helpful data for troubleshooting storage performance problems. For example, this tool displays throughput stats associated with network use or per individual LV drives.

Another handy utility that works specifically with logical volumes is the dmstats utility. This tool allows the setup and management of statistics for any devices charted by the device mapper. You can determine device mapper file names associated with logical volumes via the lsblk -p utility.

A GUI tool that gauges disk performance is the gnome-disks utility. However, back up any of the disk’s data prior to performing a write benchmark.

Resource Exhaustion Resource exhaustion is a situation in which a system’s finite resources are committed and unavailable to others. Running out of inode numbers or disk space (covered in Chapter 20) are two examples.

Dealing with Specialized Issues

One of the first things you should check for an older storage device experiencing problems is whether or not the device’s manufacturer has a new driver or firmware available. Often this can resolve a tricky issue.

Another item to check is the device’s Linux module (driver). If it is not loaded or built into the Linux kernel, your device will not function. Start with the dmesg utility to gain some clues. A snipped example is shown in Listing 24.1.

Listing 24.1: Looking up disk information via the dmesg command

# dmesg | grep sde
[…]
 [    5.566479] sd 6:0:0:0: [sde] Attached SCSI disk

The dmesg utility’s output is searched using grep to find information concerning the sde disk (/dev/sde). The important clue here is that the disk is an attached SCSI disk.

The available SCSI disk driver information is stored within a /sys/ directory as shown in Listing 24.2.

Listing 24.2: Determining the driver via the ls and udevadm commands

# ls /sys/bus/scsi/drivers
sd  sr
#
# udevadm info -an /dev/sde | grep DRIVERS | grep sd
    DRIVERS=="sd"

Notice that the sd and sr drivers are used for SCSI devices. The udevadm command confirms which one is employed for the /dev/sde disk.

After the driver (module) is determined, use the lsmod command to see if it is currently loaded into the kernel. A snipped example is shown in Listing 24.3.

Listing 24.3: Determining if the module is loaded via the lsmod command

# lsmod | grep sd
sd_mod                 46322  5
[…]
# modinfo sd_mod
filename:       /lib/modules/3.10.0-
862.11.6.el7.x86_64/kernel/drivers/scsi/sd_mod.ko.xz
[…]
description:    SCSI disk (sd) driver
[…]

If the module is not loaded, it may be built into the kernel. You can check this by looking at the modules.builtin file as shown snipped in Listing 24.4.

Listing 24.4: Determining if the module is built in via the cat command

$ cat /lib/modules/$(uname -r)/modules.builtin | grep sd_mod
kernel/drivers/scsi/sd_mod.ko

If the module is not loaded nor built into the kernel, dynamically load it using super user privileges and the modprobe command (Chapter 14).

# udevadm info -an /dev/sda | grep -i drivers
    DRIVERS=="sd"
[…]
    DRIVERS=="ahci"
[…]
#
# lsmod | grep ahci
ahci                   34056  3
[…]
#
# modinfo ahci
[…]
description:    AHCI SATA low-level driver
[…]

# lsblk -S
NAME HCTL       TYPE VENDOR   MODEL             REV TRAN
[…]
sde  6:0:0:0    disk ATA      VBOX HARDDISK    1.0  sata
[…]
#
# echo '- - -' > /sys/class/scsi_host/host6/scan

Versioning

Typically, application software programs and operating systems are continually updated. These updates may improve performance or add additional functionality. To keep track of the various application updates, a technique called versioning is employed. Versioning is the management of multiple application software updates through a numbering process. Different versions (releases) of an application have different numbers, which typically increase for newer application updates.

For example, the Linux kernel version 2.6.0 was released in December 2003. Current kernel versions can be found at www.kernel.org and have numerically higher numbers compared to the 2.6.0 version. These higher numbers indicate newer releases.

You can use versioning to determine if application updates or patches have been released. This is helpful when troubleshooting application software issues.

Updating Issues

If an application is experiencing problems, check for a new software update. If the application is available through a repository, use your distro’s particular package management to check for a new version (see Chapter 13).

Consider setting up a test system with the application environment and apply application updates to it. You can conduct thorough planned tests prior to updating production applications. Before applying any production application updates, ensure that you have an excellent backup. These two items should protect you from most bad situations.

If an application begins experiencing problems after a recent update, review the distro’s package management history information. For example, on a system using RPM, using super user privileges you can issue the command rpm -q package-name --last to see the latest update history for the package-name package. On a Debian system, check the /var/log/apt/history.log file. You’ll need to uninstall any installed packages or libraries causing the problem using the appropriate package management utility (see Chapter 13).

When one software package depends upon another package or library to operate properly, it is called a dependency. A broken dependency (also called an unmet dependency) is an undesirable situation, where a software package has been installed but one or more of its needed packages or libraries are not installed. Sometimes a package upgrade can break dependencies, resulting in what is called a broken package.

To check for broken dependencies, on a Debian package management system, use the apt-get check command. The YUM package manager will not update programs that cause a broken dependency, but if you installed a program using the rpm utility that caused problems, you can issue the rpm -aV command on any Red Hat package managed system to see damaged software packages.

Patching

A patch refers to program changes or configuration file updates for a particular application or system service. Patches may correct serious problems or fix security vulnerabilities and are often issued out of the normal software update cycle. Patching is the act of applying a patch. It does not necessarily involve updating all your system’s software. There are many conflicting theories on patch management, but at the heart of the issue is keeping your applications and Linux system running smoothly, safely, and effectively for your users.

A kernel patch release is a little different. It is a special source code package that only contains the changes applied to the major kernel source code release. You just download the patch source code package, use the Linux patch command to apply the patch updates to the existing kernel source code files on your system, and then recompile the kernel. Typically, your package manager handles this for you when you update packages (software) on the system.

Dealing with Libraries

Application functions are often split into separate library files (shared libraries) so that multiple applications that use the same functions can share these library files. Libraries were first covered in Chapter 13.

If an application begins experiencing problems after a software upgrade, it may be related to a recently upgraded shared library the application employs. You can check which libraries a program uses by typing in ldd program-name at the command line. It is helpful to redirect this command’s output into another file. Use the grep command to search package management log files to determine if one of the application’s libraries was recently updated. An example is shown snipped in Listing 24.7.

Listing 24.7: Using ldd and grep to discover a recently upgraded library

$ which ssh
/usr/bin/ssh
$
$ ldd /usr/bin/ssh > lib.txt
$
$ cat lib.txt
[…]
        libk5crypto.so.3 […]
[…]
$
$ grep -B 2 -A 2 libk5crypto /var/log/apt/history.log
Start-Date: 2019-01-22  13:37:45
Commandline: /usr/bin/unattended-upgrade
Upgrade: libk5crypto3:amd64 (1.16-2build1, 1.16-2ubuntu0.1)
End-Date: 2019-01-22  13:37:49

$

The application used in this example is the OpenSSH command. Notice one of its libraries was recently upgraded. (The -B and -A options on the grep command allow you to pull additional lines below and above the found content.) If it began experiencing problems shortly after this library upgrade, you have a probable cause. Check to see if a new upgrade or patch is available for this library. If not, you may have to uninstall it and install an earlier version.

Exploring Environment Variable Issues

If you have a newly installed application that is not executing, check the PATH environment variable. This particular variable determines what directories are searched for a program that the Bash shell does not directly handle. An example of displaying the variable’s contents is shown in Listing 24.8.

Listing 24.8: Viewing the PATH environment variable

$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:
/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
$

If you need to modify this parameter for everyone, create a Bash script file (Bash scripts are covered in Chapter 25) in the /etc/profile.d/ directory. Be sure to use the .sh file extension. The file must be owned by root and belong to the root group. Set the file’s other (world) permissions to r so that all users can read the file. The script is read by the /etc/profile or /etc/bashrc file, depending on your distribution, when a user logs into the system or starts a new shell.

If only certain users need this particular PATH modification, make it in their ~/.profile, ~/.bash_profile, or ~./bash_login file. Environment files were discussed in Chapter 10.

Gaining GCC Compatibility

The most common tool used for compiling programs in Linux is the GNU Compiler Collection (GCC). If you have problems compiling an application on Linux with GCC, there are several potential causes. They are as follows:

• GCC uses the system C library, which might not be compliant with the ISO C standard.
• There are several notable incompatibilities between GNU C and non-ISO versions of C.
• GCC uses corrected versions of system header files, which can cause issues.

Note that besides these issues, you might be using an older version of gcc and need to update it. For example, if your system distro is CentOS 7 and you are using gcc v4.4.*, you need to upgrade the GCC package.

You can find detailed documentation on the GCC compiler at its website— https://gcc.gnu.org/. This site includes FAQ and other useful information.

Perusing Repository Problems

The very first thing to check when you get an odd error message concerning a package that cannot be found, updated, or installed is your network connection. Often a system that is not network-connected causes this problem. However, various package repositories (also called repos) can become corrupted.

On a system using a Debian package manager, such as Ubuntu, if you get a message saying it cannot download repository information or something similar, use the apt-get clean command. This command cleans up the database and any temporary download files. After that, try to update the local repository with apt-get update, which attempts to retrieve updated information about packages in the repository.

On a system using a Red Hat package manager, such as CentOS, you can employ the yum clean all or zypper clean -a command, depending upon your distro. Next update the local repository with the yum check-update or zypper refresh command.

If you are attempting to install or update packages from a non-standard repository, you may need to enable that repo on your system. To see a list of the enabled repositories on your system, use the yum repolist or zypper repos command on Red Hat package systems. For Debian package systems, you’ll need to issue the grep -v "#" /etc/apt/sources.list command to see the enabled repositories.

Before you add any additional non-standard repositories, back up the repository file(s), such as these:

• /etc/apt/sources.list
• /etc/yum.repos.d/*.*
• /etc/zypp/repos.d/*.*

You have to manually edit the sources.list file to add and enable a new repository. To add and enable a new repo with YUM, use the yum-config-manager --add-repo repository-url command. The zypper command is similar but also requires a repository name alias—zypper addrepo repository-url alias.

Keep in mind that we have only touched on a few of the more common problems you can run into with programs and their various dependencies. Be sure to employ your distribution’s man pages for additional help.

Unrestricting ACLs

A firewall ACL identifies a network packet by reviewing its control information along with other network data. Therefore, when troubleshooting an application issue related to a firewall, you’ll need to gather the following information for the application packets traveling back and forth:

• Source address or hostname
• Destination address or hostname
• Network protocol(s) used
• Inbound port(s) used
• Outbound port(s) used

You also need to know both your source and destination systems’ firewall application in use. When you’ve gathered this information, you can review the firewall settings on both sides to determine if the ACLs are overly restrictive.

For example, if you are using firewalld on your application’s host system, you can quickly check the current default zone, as shown in Listing 24.9.

Listing 24.9: Viewing the default zone with the firewall-cmd command

$ firewall-cmd --get-default-zone
drop
$

Notice that this system has its firewalld default set to the drop zone. This means all incoming network packets are dropped and only outbound network connections are allowed. If the application receives data or connections from other systems, then this firewall ACL setting is overly restrictive.

Unblocking Ports

If your application relies on another system service (daemon), you’ll want to check rules related to the service port. Blocking a port needed by the external service would adversely affect the application. If your application is designed to use a port that is not dedicated to a well-known service, check it as well.

For example, if you are providing public web services on your system, you need to allow incoming and outbound packets associated with the HTTPS protocol port 443. If your system is using the iptables firewall software, you can view the current ACL rules via the iptables -L command. If the packet filter is blocking port 443 via a particular rule or policy, you can modify the chain via a command similar to the one in Listing 24.10, which opens up port 443.

Listing 24.10: Modifying the firewall with the iptables command

$ iptables -A INPUT -p tcp --dport 443 -j ACCEPT
$

Keep in mind you also need to modify the OUTPUT chain rules to allow your web server to establish connections. In addition, if your application allows HTTP traffic, you must modify rules for port 80 as well.

Unblocking Protocols

Besides ports, be aware of the various protocols, such as UDP, TCP, and ICMP, that your application employs. If it uses another system service, you must know the protocols it uses as well. The /etc/services file can help.

For example, an application is working with a DNS caching server on the local network. DNS protocol uses port 53. Check the /etc/services (well-known ports) file to find the transport protocols it employs as shown snipped in Listing 24.11.

Listing 24.11: Checking DNS’s protocols in the /etc/services file

$ grep 53 /etc/services
domain          53/tcp                       # Domain Name Server
domain          53/udp
[…]
$

Unblock port 53 on the DNS server system for both TCP and UDP, since DNS listens for requests using those two transport protocols. Also, unblock them for both inbound and outbound packets.

Looking at Helpful Hardware Commands

When you are troubleshooting hardware problems, there are many Linux command-line tools that can help. The lspci, lsusb, and lsdev commands are a few, which were introduced in Chapter 23. We’ll cover a couple more great utilities, dmidecode and lshw, here.

# ls /sys/firmware/dmi/tables
DMI  smbios_entry_point
#

$ dmidecode -h
Usage: dmidecode [OPTIONS]
Options are:
 -d, --dev-mem FILE     Read memory from device FILE (default: /dev/mem)
 -h, --help             Display this help text and exit
 -q, --quiet            Less verbose output
 -s, --string KEYWORD   Only display the value of the given DMI string
 -t, --type TYPE        Only display the entries of given type
 -u, --dump             Do not decode the entries
     --dump-bin FILE    Dump the DMI data to a binary file
     --from-dump FILE   Read the DMI data from a binary file
     --no-sysfs         Do not attempt to read DMI data from sysfs files
 -V, --version          Display the version and exit
$

# dmidecode -t memory
# dmidecode 3.0
Getting SMBIOS data from sysfs.
SMBIOS 2.5 present.

#
# dmidecode -t system
# dmidecode 3.0
Getting SMBIOS data from sysfs.
SMBIOS 2.5 present.

Handle 0x0001, DMI type 1, 27 bytes
System Information
        Manufacturer: innotek GmbH
        Product Name: VirtualBox
        Version: 1.2
        Serial Number: 0
        UUID: 3909BE96-5CA6-4801-8236-D6113BB5D2CF
        Wake-up Type: Power Switch
        SKU Number: Not Specified
        Family: Virtual Machine

# lshw -short
H/W path          Device      Class       Description
=====================================================
                              system      VirtualBox
/0                            bus         VirtualBox
/0/0                          memory      128KiB BIOS
/0/1                          memory      4GiB System memory
/0/2                          processor   Intel(R) Core(TM) […]
/0/100                        bridge      440FX - 82441FX PMC [Natoma]
/0/100/1                      bridge      82371SB PIIX3 ISA [Natoma/Triton II]
/0/100/1.1        scsi1       storage     82371AB/EB/MB PIIX4 IDE
/0/100/1.1/0.0.0  /dev/cdrom  disk        CD-ROM
/0/100/2                      display     VirtualBox Graphics Adapter
/0/100/3          enp0s3      network     82540EM Gigabit Ethernet Controller
/0/100/4                      generic     VirtualBox Guest Service
/0/100/5                      multimedia  82801AA AC'97 Audio Controller
/0/100/6                      bus         KeyLargo/Intrepid USB
/0/100/6/1        usb1        bus         OHCI PCI host controller
/0/100/7                      bridge      82371AB/EB/MB PIIX4 ACPI
/0/100/8          enp0s8      network     82540EM Gigabit Ethernet Controller
/0/100/d          scsi2       storage     82801HM/HEM (ICH8M/ICH8M-E) SATA […]
/0/100/d/0        /dev/sda    disk        16GB VBOX HARDDISK
/0/100/d/0/1                  volume      1GiB Linux filesystem partition
/0/100/d/0/2      /dev/sda2   volume      13GiB Linux LVM Physical Volume partition
[…]
/0/4                          input       PnP device PNP0f03
/1                virbr0-nic  network     Ethernet interface
[…]
#

$ sudo lshw -class display
  *-display
       description: VGA compatible controller
       product: VirtualBox Graphics Adapter
       vendor: InnoTek Systemberatung GmbH
[…]
       configuration: driver=vboxvideo latency=0
[…]
$

Investigating Other Hardware Problems

Occasionally you have a hardware problem that is uncommon. Being able to quickly address these unique issues will make you stand out from your peers.

Memory Physical problems with RAM are tricky to diagnose. Some symptoms of this issue include a system’s performance slows over time, the system appears to hang when a memory-intensive application is running or at boot, kernel panics or segmentation faults occur intermittently, files are sporadically corrupted, and/or program installations fail.

First make sure it is not a memory capacity issue, which often shows symptoms similar to hardware problems. Check via the free and vmstat utilities.

You can quickly determine hardware information concerning your RAM via the lshw utility. Just issue the lshw -class memory command.

If you recently added new memory, most likely you obtained a faulty component. Damage can also be done to RAM via power spikes or outages. In any case, you’ll want to conduct a test on the memory. Typically you can conduct such a test via a system reboot and accessing the memtest or memtest86+ option in the server’s boot menu. If this option is not available, you can employ the memtester utility. This command-line utility is typically not installed by default, but it is available either in your distribution’s repository or as an RPM or dpkg file (see Chapter 13). When using this utility, you’ll need to shut down any production applications and test the memory in chunks.

Printers External hardware devices are typically plug and play for Linux, but odd problems do arise. When dealing with printers, the issue typically comes down to either an outdated/incorrect driver (PPD) or a bad connection. Start by checking the kernel ring buffer with dmesg and taking a look at the printer error log files, such as /var/log/cups/error_log.

If the printer was recently installed, check its configuration. You can do this via a web browser, if available, on the system by entering 127.0.0.1:631 in the address bar. If your system does not have a GUI, you can look through the /etc/cups/printers.conf file to review the printer’s configuration.

Determine how the printer connects to the system. Is it a network printer? Does it attach via a USB cable? Is the printer directly connected into a parallel port? If it is a network-connected printer, first check that the network is operational. If the printer is attached via a USB cable, start with troubleshooting the USB connections (covered later in this chapter). If it uses a parallel port on your system, it may be a bad adapter. If possible, consider switching it to a different connection type, such as USB, or obtaining a new printer with a more modern configuration.

Check if the printer’s PostScript Printer Definition (PPD) file or driver needs updating. Go to the manufacturer or open-source driver website to determine if an update is available. You can view all the currently available printer drivers on your system via the lpinfo -m command. Keep in mind the problem may involve a needed printer firmware update, so check for those as well.

Video Hardware issues with video show up in sluggish displays, audio lag, glitches on the screen, and so on. You may even see a black screen or receive no audio output. Some problems can even cause the system to crash or hang.

As with many other hardware problems, first check the kernel ring buffer (dmesg) and video log files. If your system is still using X11, check the journal file or the /var/log/Xorg.0.log file. If you are employing Wayland, check the journal file via the journalctl command.

To find out what graphics card driver your system is using, just type in lspci -vnn at the command line, redirect STDOUT to a file. Peruse the file for the word VGA. This will show your graphics card driver data. You can also employ lshw -class display (or video) and look for the driver information there. When you have the driver’s name, find out additional details through the modinfo driver-name command.

Check the manufacturer’s or open-source site to see if there is an updated graphics card driver available. If not, try testing the card on another system to see if you need to replace it.

Communications Ports A communications port is a serial communications port. Though a rarity nowadays, it is often used to connect hardware such as point-of-sale devices. The device files that represent these serial ports are /dev/ttyS#.

When experiencing problems with a serial communications device, start the troubleshooting process by issuing dmesg | grep ttyS to find the device file name in use. When you have the full device file name, employ the setserial utility. This will provide detailed information on the serial device. Use super user privileges, and type in setserial -a device-file-name at the command line. Look for the interrupt request (IRQ) number in the output.

When you know the IRQ of the serial device, you can check the interrupts file. If you do not find the IRQ number in the /proc/interrupts file, this indicates that the appropriate driver for the serial device is not loaded.

If the driver is loaded, check the manufacturer’s website for a newly updated driver. Also, check the serial device’s recommended configuration and make any modifications needed via the setserial utility.

USB If you have a USB device, such as a printer, directly attached to your system and problems occur, there are some simple troubleshooting techniques you can employ. First ensure that the USB module (driver) is loaded into the kernel by using super user privileges and typing in lsmod | grep usb at the command line. If you get a response, it is loaded. If you just get a prompt back, then employ the modprobe command to load the module (Chapter 14).

If the driver is already loaded, try detaching the device’s USB cable from the system. Watch the journal file via the journalctl -f command. If you are on an older Linux system, use the tail -f command on the appropriate log file, such as /var/log/syslog or /var/log/rsyslog. After the watch is in place, plug back in the USB device’s cable and see what log messages are generated. If the USB device is a printer, also check the /var/log/cups/error_log for any pertinent information. You may uncover some important details here.

When you have completed that activity, employ the lsusb -v command to see if the device is showing up on the USB bus. If you see the device’s manufacturer and product information, then Linux can see the device. If the lsusb utility is not installed on your system, look through the kernel ring buffer via dmesg.

Check the USB’s device files for corruption. This topic was covered earlier in the chapter in the Missing Devices list item in the section “Exploring Common Issues.”

If your USB device is still not working, try attaching it to a different USB port. However, prior to doing so, put another watch on the appropriate journal or log files. You may also want to try switching out your USB cable to see if that resolves the issue(s).

Keyboard Mapping If you press a key on your keyboard and a different letter appears on the screen, most likely you have a keyboard mapping issue. The fix depends on the particular distribution you are using.

For Red Hat–based distros, type in localectl with no options and your current key map will display. To see the list of available key maps, enter localectl list-keymaps and a list of available key mappings will display. This list can be rather large, so you might want to redirect STDOUT to the less utility for your perusal. When you find the appropriate key mapping name, permanently set it by typing in localectl set-keymap keymap-name at the command line.

For Debian-based distros, use super user privileges and enter the dpkg-reconfigure keyboard-configuration command. This will enter you into a text-based menu system where you can select the appropriate keyboard mapping.

Hardware or Software Compatibility Issues Before you purchase any new hardware (or software for that matter), make sure it will work with your Linux distribution. Keep in mind that while Linux is the number-one operating system kernel for super computers and a strong contender in the server world, it does not always get the attention it deserves from hardware manufacturers. Therefore, often drivers are not available for brand-new devices, or you may end up with a manufacturer’s poorly written device driver. Check with the Linux community to find well-developed drivers and hardware device recommendations. You’ll save yourself a lot of trouble and headaches.