12.3. Target System

The target system is the entity under test. As robustness relates to input and environmental stresses, the target system definition is a key issue. Typically, a layered model of a system is used to indicate the different entities in the system. The OS is split into a suitably large number of components, as in Figure 12.1, depending on the goals of the study. A model similar to the one in Figure 12.1 allows for defining the input/output interfaces in the system and to isolate the target system under test.

The model chosen should be general enough to make the approach applicable to more than a specific target. This becomes even more important when the goal is to benchmark a system, as the benchmark is used to compare multiple systems. In Koopman and DeVale [20], the POSIX interface (an API to OS services) is used as a benchmark target (i.e., the components “below” this level in the system are considered to be the target system). Any system supporting this API can then potentially be benchmarked.

Often the OS is split into components, allowing characterization of intercom-ponent interaction. Arlat et al. [30] considers microkernel-based systems, where interactions between the different functional components comprising the kernel are studied. Gu et al. [16, 41] study the Linux OS, dividing it into functional subsystems, allowing characterization of error propagation across subsystems.

12.3.1. Case Study

The system used in this case study consists of the OS software and hardware components depicted in Figure 12.2. The goal is comparing device drivers and system services on their effect on OS robustness. The target system is the OS. The interfaces to the system under test thus become the software interfaces between the OS and the drivers and the interface between applications and the OS, as depicted in Figure 12.2. The first step (evaluation goals), has already pointed out device drivers as being separate from the rest of the OS, and thus Figure 12.2 highlights this key interface.

Drivers handle the interaction between the OS and the hardware. They are responsible for implementing the interface that the OS expects and are typically implemented by the hardware vendors. We do not consider direct interaction by the OS (or applications for that matter) with the hardware. This type of interaction is indeed possible in some OSs, but we focus on the more common model using device drivers.

The drivers in the system are labeled D₁, D₂,..., D_N in Figure 12.2. A driver exports services (service ds_x.y is the y^th service of driver D_x) that the OS uses to interact with the driver. For implementing its functionality, a driver may use other services provided by the OS (os_x.y), for instance, system calls. The two types of service interactions are considered individually in the case study.

The OS layer includes shared libraries present in the system. This corresponds to a programmer’s view of the system, that is, the libraries are present in the system to provide services for applications (and drivers). The OS layer is part of two important interfaces: the OS-Application and OS-Driver interfaces. The OS-Application interface (also known as API) provides services s₁, s₂,..., s_s to be used by applications. The OS-Driver interface contains the services the OS provides for drivers to use. For a specific driver D_x, the OS services it uses are labeled os_x.1, os_x.2,..., os_x.K. Note that it is possible for the OS to provide the same service for applications as well as for drivers.

The applications executing in the system provide services to users by the use of application code and services provided by the OS. Since different OS services may have different failure characteristics, the set of used services influences the behavior of the application when faults are present in the system. For the robustness evaluation presented in this chapter, the workload is defined as a set of applications and Section 12.4 discusses the choice of workload in more detail.