Similar to most programming languages and frameworks, the OpenCL specification provides support for optional extensions. They represent a small but important set of OpenCL extensions. In this chapter, we discuss a number of these extensions that can provide programmers an extended set of tools to implement OpenCL applications.

OpenCL defines three types of extensions:

Extensions can be associated with a given OpenCL platform and in those cases are likely to be always enabled. For example, AMD's Event Callback extension (cl_amd_event_callback) is always enabled. Other extensions can be specific to a particular set of devices (e.g., double precision support (cl_khr_fp64)).

In general, the application must query either the platform or a particular device to determine what extensions are supported. Using the C++ Wrapper API, the following code demonstrates how to query both the platform and a device for the extension information:

Of course, because we now have a value of type std::string for platform and device extensions, we can simply use the method find(). For example, to check that a device supports 64-bit Atomics (defined with the name cl_khr_int64_base_atomics), you could use the following code:

Because extensions are optional, there is no requirement for an implementation to provide any externally defined symbols that the host application can call. Accordingly, a host program cannot be statically linked against any extension APIs. Instead, we can query the particular function's address equipped with the following API:

This returns the address of the extension function named by funcname. The pointer returned can be cast to a function pointer of the expected type. Typedefs for each function pointer are declared for all extensions that add API entry points. For example, the EXT extension Atomic Counters, exposed in the OpenCL extension header cl_ext.h, provides low-latency atomic append/consume counters and defines the following:

Then, assuming that a particular OpenCL device supports the extension cl_ext_atomic_counters, the following call will set clCreateCounterEXT_pfn to a non-NULL value:

In practice, it is often useful to use the following macro to enable quick allocation of these functions:

To get the same behavior as the function pointer definition previously, we can write the following:

The examples presented later in this chapter go a step further and use the OpenCL C++ Wrapper API, which, if a particular extension is enabled, directly exposes a class interface and implicitly takes care of allocating and querying function pointers (as required).

Once an application has verified that a particular extension is supported by a device and has allocated any required API function addresses, it can then use any corresponding OpenCL C source extensions. However, these extension must also be enabled, this time in the OpenCL C source itself. These features are controlled using a pragma directive. The pragma is of the form

where extension_name is the corresponding name of the extension being enabled or disabled, such as cl_khr_int64, and behavior can be either enable or disable. The impact of enabling or disabling an extension follows from the position in the source code of the last seen pragma for a given extension. For example, to enable 64-bit atomics in an OpenCL C source program, the following code would do the job:

Images are the one exception to the rules described previously. To check if a device supports this feature (which is optional), a separate device query is required:

If the result of this call is true, then images are supported by the device in question and the OpenCL C source is not required to explicitly enable this feature.

There are many KHR, EXT, and vendor extensions—too many to discuss in detail in this chapter. The remainder of this chapter discusses two important extensions in wide use today:

The EXT extension Device Fission (Bellows et al., 2010) provides an interface for subdividing an OpenCL device into multiple subdevices. As an example, consider an AMD six-core Istanbul x86 CPU shown symbolically in Figure 11.1A. We view all six cores as a single OpenCL device by default, as shown in Figure 11.1B. Using Device Fission, the six cores can be subdivided into six OpenCL devices as shown in Figure 11.1C, each capable of supporting one or more command queues. Because these command queues are asynchronous and run in their own threads, it is possible to use Device Fission to build a portable and powerful threading application based on task parallelism. To date, Device Fission is supported only on CPU-like devices, but in the future this functionality will spread across all devices in the platform, including GPUs. Next, we motivate and describe the basic use of Device Fission through an example.

An interesting application of Device Fission is to use it to build and study concurrent runtimes, similar to Microsoft's Concurrent Runtime (ConcRT) (Microsoft, 2010) and Intel's Threading Building Blocks (TBB) (Intel, 2011). We present a case study of building a simple C++ class that supports a parallel for construct that distributes work across a set of devices created using Device Fission. Although this example has little in common with the kind of production functionality provided by runtimes such as ConcRT and TBB, it provides enough to show that with only a small amount of effort we can build portable, scalable, parallel applications that go beyond basic data-parallel computations. For this example, the goal is to implement a simple class Parallel that provides a parallelFor method to distribute work across x86 cores, each having a corresponding OpenCL device generated by Device Fission. The class Parallel has the following public interface:

We demonstrate its use with a simple application that counts the number of prime numbers in an input array of randomly generated integers. The implementation uses the OpenCL C++ Wrapper API (Gaster, 2010). To enable Device Fission, we simply need to define USE_CL_DEVICE_FISSION and include cl.hpp:¹

As described previously, we encapsulate the parallel features of our simple runtime in the class Parallel. All OpenCL state for the program is held in the private part of this class definition. The default constructor initializes OpenCL and divides the CPU device into single core subdevices. As is standard with all OpenCL initialization, we need to first query a platform, then create a context (in this case, using CL_DEVICE_TYPE_CPU), and finally inspect the list of devices (as shown in Figure 11.1B, this will always be presented as a single element):

Before we can use the Device Fission extension, the program must check that it is an exported extension for the device:

Given an OpenCL cl::Device, in this case devices[0], the method createSubDevices creates subdevices,

which, given a list of partition properties (as defined in Table 11.1), creates a set of subdevices, returned in the parameter devices.

The following code then creates the subdevices, using the partition CL_DEVICE_PARTITION_EQUALLY_EXT to equally subdivide (in this case), producing a one-to-one mapping between subdevice and core:

The following code concludes the default constructor definition by iterating through the list of subdevices, creating a command queue for each one:

Figure 11.2 shows each CPU core paired with its corresponding command queue. It is important to note that commands submitted via each queue run asynchronously and concurrently with each other.

The implementation for atomicAdd is straightforward and is provided later. Here, we focus on the definition of parallelFor, which takes two arguments. The first argument represents the bounds of the iteration space, and the second argument is a function object that will be applied to each index across this space. This second argument is a native C++ function object, ² but OpenCL clEnqueueNDRangeKernel operations are valid only for OpenCL C kernels, and so we need another approach to support native C++ functions. Fortunately, OpenCL has just such a function, clEnqueueNativeKernel, which can enqueue a C function. With some careful marshaling, this can also be made to work for a C++ function. Native kernels were described in Chapter 5.

Using clEnqeueNativeKernel, the definition of parallelFor is straightforward: ³

The first part of the function sets up the argument, which in this case is the actual function we want to run. This is required because it is a C++ function object (often called a functor), and OpenCL is expecting a C function. Thus, we provide a wrapper function called funcWrapper that takes as an argument a pointer to the functor. The wrapper function “unboxes” it and calls the function object when executed. The main body of the function is the loop executing the functor for the 0 … range, with the inner loop mapping some subset of this to the number of actual subdevices. Note that these are all submitted asynchronously, and each call returns an event that we wait on at the end of each set of submissions. A more optimized version might wait until a large number of submissions have happened and control a finer grain of control using queue.flush() and so on. We leave this as an exercise for the reader.

Finally, we put this all together in Listing 11.1, which shows the complete implementation of our primes checking example, including the main function. One interesting and important aspect of this example is that no OpenCL C device code is used, and it is simply using the OpenCL runtime as a portable threading library.

Floating point formats were created to allow programmers to work with very large and very small non-integral data values. For many applications, single precision floating point does not provide enough range for the targeted application. Many applications (particularly in science and engineering) require part or all of a particular computation to use double precision. OpenCL does not require that a particular compute device support double precision. For critical applications in which double precision is required, OpenCL provides the optional cl_khr_fp64 extension. This is enabled by including the following directive before any double precision use in an OpenCL C program:

Once enabled, the double precision support provides access to the following data types:

The double type conforms to the IEEE 754 double precision storage format.

There is a one-to-one mapping with the corresponding single precision float types. On an AMD CPU, when the OpenCL device is CL_DEVICE_TYPE_CPU, vector types are mapped directly to SSE and AVX packed registers. In the case in which a vector type is larger than the underlying hardware vector, the resulting implementation is expanded into multiples of the hardware width. All of the conversion rules defined by the OpenCL specification for float types are also defined for doubles. The built-in math functions provided for float have been extended to include appropriate versions that work on double, double2, double3, double4, double8, and double16 as arguments and return values.

As a simple example, Listing 11.2 shows OpenCL C code to implement a block matrix multiple using double precision. Note that the first line of the OpenCL source file enables the extension.

The host OpenCL program, written using the OpenCL C++ Wrapper API, is given in Listing 11.3, with the shared header, matrixmul.h, given in Listing 11.4. The example is straightforward, but two points are worth discussing further. First, to avoid unexpected runtime errors, the application must check that the device supports the extension cl_khr_fp64, and this is achieved with the following code:

In this case, if the device does not support the extension, then the application simply exits. A more robust solution might drop back to the host to perform the computation if necessary. Second, this example uses the profiling API for command queues to collect information on the time to execute the matrix multiple.