Kernel functionality
The kernel provides the glue that makes all components in Blue Gene/Q system work together. This chapter provides an overview of the functionality that is implemented as part of the Compute Node Kernel (CNK) and the I/O node kernel, which includes information about the following topics:
2.1 Compute Node Kernel
The CNK is a flexible, lightweight kernel for Blue Gene/Q compute nodes that can support diagnostic modes and user applications. It provides an operating system that is similar to the Linux operating system and supports a large subset of Linux‑compatible system calls. This subset is based on the IBM Blue Gene/P® system, which demonstrated good compatibility and portability with the Linux operating system. The CNK is tuned for the capabilities and performance of the Blue Gene/Q application-specific integrated circuit (ASIC).
As part of the Blue Gene/Q system, the CNK supports threads and dynamic linking for further compatibility with the Linux operating system. Figure 2-1 shows the interaction between the application space and the kernel space.
Figure 2-1 Compute Node Kernel overview
When running a user application, the CNK connects to the I/O node through the torus network. This connection communicates to a set of processes called Control and I/O Services (CIOS) that run on the I/O node. All function-shipped system calls are forwarded to the CIOS process and executed on the I/O node.
At the user-application level, the Compute Node Kernel supports the following application programming interfaces (APIs) among others:
•Message Passing Interface (MPI) support between nodes using MPI library support
•Open multi-processing (OpenMP) API
•Standard IBM XL family of compilers support with XLC/C++, XLF, and GNU Compiler Collection
•Highly optimized mathematical libraries, such as IBM Engineering and Scientific Subroutine Library (ESSL)
•GNU Compiler Collection (GCC) C Library, or GLIBC, which is the C standard library and the GCC interface.
The CNK provides the following services:
•Torus direct memory access (DMA), which provides memory access for reading, writing, or doing both independently of the processing unit. The DMA torus interfaces are made available to the user space, which allows communication libraries to send messages directly from the application without involving the kernel. The kernel, with the hardware, implements secure limit registers that prevent the DMA from targeting memory outside the application. These constraints, along with the electrical partitioning of the torus, provide security between applications.
•Shared-memory access on a local node
•Hardware configuration
•Memory management
•MPI topology
•File I/O
•Sockets connection
•Signals
•Thread management
•Transport layer through the torus network
2.1.1 Stateless compute nodes
The Blue Gene/Q hardware is a stateless system with no embedded read-only memories (ROMs) or resident basic input/output system (BIOS). When the hardware is reset, the Control System must load the operating system into the memory of each compute node. It accomplishes this process in two phases:
•Phase 1 loads a small firmware component into the embedded random access memory (RAM) on each compute node. This firmware starts executing and initializes critical pieces of the Blue Gene/Q chip.
•Phase 2 communicates over a custom protocol to download the rest of the kernel images. These kernel images are then executed, allowing for connectivity over the torus network.
2.1.2 Firmware
The firmware component provides low-level services that are both specific to Blue Gene and common to the Linux operating system and the Compute Node Kernel. As such, these services provide a consistent implementation across node types while insulating the kernels from the details of the Control System. The common node services provide the same low-level hardware initialization and setup interfaces to both the Linux operating system and the Compute Node Kernel.
2.2 Role of the I/O node kernel
The I/O node kernel provides I/O services to compute nodes and runs on I/O nodes. The I/O nodes also play an important role in starting and stopping jobs and in coordinating activities with debug and monitoring tools.
The operating system that runs on the I/O nodes is a distribution of Red Hat Enterprise
Linux 6 for IBM PowerPC. IBM System Blue Gene Solution: Blue Gene/Q System Administration, SG24-7869 describes setup and configuration of the operating system. The I/O node is not apparent to the application, but it is important to consider the I/O node when tuning I/O performance.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset