• The addition of the ability to automatically offload from the processors to coprocessors, which is described in this chapter.

• Use all the threads to get best performance. You do not need to avoid the core where the OS resides, as you should when using offload, because the OS is much less likely to be using significant computational resources. For examples, on a 60-core Intel Xeon Phi coprocessor use the following:
 OMP_NUM_THREADS=240

• Use thread affinity to avoid thread migration. For examples, on a 60-core Intel Xeon Phi coprocessor use the following:
 KMP_AFFINITY=explicit,granularity=fine,proclist=[1-239:1,0]

• Consider enabling huge paging for memory allocation by using libhugetlbfs.so or using the mmap() system call (see Chapter 9).

Automatic offload works better when matrix size is right

Matrix size is critical for automatic offload to get good performance. In fact, automatic offload does not even start when matrices are too small (generally this is when row or column size is smaller than 2048). This is because in this situation the overhead of data transferring overshadows any performance benefit offloading can bring. If matrices are sufficiently large, then the best performance is typically achieved when the matrices are square. The MKL environment variable MKL_MIC_MAX_MEMORY may be valuable to guide MKL on memory usage to best suit your application and system.

Debug or test by forcing execution to exit if offload fails

By default, if offload does not occur either because the runtime cannot find a coprocessor or because it cannot be properly initialized, then the runtime will automatically do all the work on the processors. This behavior is part of the transparent execution model provided by automatic offload. There is an environment variable OFFLOAD_REPORT to set to have an offloading report dumped out during runtime for seeing what is being offloaded. Users can also override the default fallback behavior by setting the environment variable (MKL_MIC_DISABLE_HOST_FALLBACK=1). Then, if offload cannot take place the program will exit with an error message indicating that the automatic offload could not be initialized or otherwise performed.

When not to use automatic offload

There are situations where compiler-assisted offload is more appropriate:

How to disable and re-enable automatic offload

Automatic offload only needs to be enabled once and its effect lasts until the end of the execution for all automatic offload-aware functions (excluding those explicitly offloaded using compiler-assisted offload). To disable automatic offload in the middle of a program, user code should call the support function mkl_mic_disable(). Alternatively, user code can call mkl_mic_set_workdivision() to set 100 percent of work to be done on the processors:

rc = mkl_mic_set_workdivision(MKL_TARGET_HOST,

              coprocessor_number, 1.0);

Later on, automatic offload can be re-enabled by calling mkl_mic_enable() or the workdivision function again to specify a different percentage for the processors or for a coprocessor. The environment variable OFFLOAD_DEVICES can be used to regulate the compiler and MKL library usage of devices as well.

Use automatic and compiler-assisted offload together

In addition to automatic offload, Intel MKL supports compiler-assisted offload. That is, offload can be explicitly specified using compiler pragmas provided in Intel® Fortran Compiler XE and Intel® C/C++ Compiler XE. Compiler-assisted offload requires more effort from programmers, but it provides more control and is more flexible. In sophisticated applications, cases may exist where mixing automatic offload and compiler-assisted offloading in one application is necessary. Intel MKL does allow this usage model. However, when automatic offload and compiler-assisted offload are used in the same program, and you may need to explicitly specify work division for automatic offload-aware functions using support functions but not via environment variables. The default work division setting is to run all the work on the processors.

Avoid oversubscription

To help avoid performance drops caused by oversubscribing Intel Xeon Phi coprocessors, Intel MKL limits the number of threads it uses to parallelize computations:

In these, Number-of-Cores is the number of core in an Intel Xeon Phi coprocessor.

Optimize openMP and threading settings

To improve performance of Intel MKL routines, use the following OpenMP and threading settings:

Data alignment and leading dimensions

Additionally, for data issues, to improve performance of Intel MKL you may wish to:

Specifically for FFT functions, follow these additional recommendations:

Favor LAPACK unpacked routines

The routines with the names that contain the letters HP, OP, PP, SP, TP, or UP in the matrix type and storage position (the second and third letters respectively) operate on the matrices in the packed format (see LAPACK “Routine Naming Conventions” sections in the Intel MKL Reference Manual). Their functionality is equivalent to the functionality of the unpacked routines with the names containing the letters HE, OR, PO, SY, TR, or UN in the same positions, but the performance is significantly lower.

If the memory restriction is not too tight, use an unpacked routine for better performance. In this case, you need to allocate N²/2 more memory than the memory required by a respective packed routine, where N is the problem size (the number of equations).

For example (Fortran code shown), to speed up solving a symmetric eigenproblem with an expert driver, use the unpacked routine:

call dsyevx(jobz, range, uplo, n, a, lda, vl, vu, il, iu, abstol, m, w, z, ldz, work, lwork, iwork, ifail, info)

where a is the dimension lda-by-N, which is at least N² elements, instead of the packed routine use this:

call dspevx(jobz, range, uplo, n, ap, vl, vu, il, iu, abstol, m, w, z, ldz, work, iwork, ifail, info)

where ap is an array with size of N×(N+1)/2 holding the triangular part of a symmetric matrix.

• Use data persistence to avoid unnecessary data copying and memory allocations and deallocations.

• Use thread affinity and avoid using the core where the OS is doing its work when using offload. For examples, on a 60-core Intel Xeon Phi coprocessor use the following:
 MIC_KMP_AFFINITY=explicit,granularity=fine,proclist=[1-235:1,0]

• Consider using larger pages by setting MIC_USE_2MB_BUFFERS=64K

• –fast–transcendentals for transcendentals (sin, log, pow, …)

• –no–prec–sqrt for square roots

• –no–prec–div for division

• –fimf–precision set desired accuracy to high, medium or low

Basics

The Intel Xeon Phi coprocessor supports the same floating-point data types as the Intel Xeon processor. Single (32 bit) and double (64 bit) precision are supported in hardware; quadruple (128 bit) precision is supported through software. Extended (80 bit) precision is supported through the x87 instruction set. Denormalized numbers and gradual underflow are supported, but abrupt underflow is the default at all optimization levels except –O0. The same set of rounding modes is supported as for Intel Xeon processors.

Floating-point exceptions

The biggest differences arise in the treatment of floating-point exceptions. The vector floating-point unit on the Intel Xeon Phi coprocessor flags but does not support trapping of floating-point exceptions. The corresponding bit in the VXCSR register is protected; attempts to modify it result in a segmentation fault. Some compiler options such as –fp–trap (C/C++) or –fpe0 (Fortran) that would unmask floating-point exceptions on Intel Xeon processors are unsupported on Intel® MIC architecture. The options –fp–model except or –fp–model strict still request strict, standard-conforming semantics for floating-point exceptions. This is achieved by generating x87 code for floating-point operations instead of code that makes use of Intel Xeon Phi coprocessor vector instructions. Because such code cannot be vectorized, this may have a substantial impact on performance. Nevertheless, these options may be useful for application debugging. For similar reasons, the options –ansi and –fmath–errno may result in calls to math functions that are implemented in x87 rather than the vector instructions.

In the Intel Fortran compiler version 13.0, the IEEE_FEATURES, IEEE_ARITHMETIC and IEEE_EXCEPTIONS modules are not yet updated for the properties of the Intel Xeon Phi coprocessor.

The –fp–model switch

The same settings of the –fp–model switch are supported as for Intel Xeon processors. The default setting is –fp–model fast=1 for both. The behavior of the –fp–model precise option is the same, though the impact on performance may be somewhat greater for Intel Xeon Phi coprocessors, because of the larger vector width on Intel MIC architecture and larger potential performance gain from the vectorization of reduction loops and loops containing transcendental math functions, square roots or division. The impact of –fp–model except and –fp–model strict on performance may be greater, for the reasons noted above.

The setting –fp–model fast=2 sets the –fimf–domain–exclusion switch and enables faster, inlined versions of some math functions for the Intel Xeon Phi Coprocessor, see the section “Precision of Math Functions” below.

The –fp–model precise switch enables arithmetic using denormalized numbers and disables abrupt underflow; abrupt [gradual] underflow can be enabled (but not required) explicitly using –[no–]ftz. This behavior is the same as on Intel Xeon processors.

Fused multiply-add

Intel Xeon processors up to and including Intel® 3rd Generation Core™ processors do not have a fused multiply-add (FMA) instruction. The FMA instruction on Intel Xeon Phi coprocessors only performs a single rounding on the final result, so can yield results that differ very slightly from separate addition and multiplication instructions.

In principle, the –fp–model strict switch would disable fused multiply-add (FMA) instructions. But since, as noted above, –fp–model strict suppresses vector instructions in favor of legacy x87 floating-point instructions, this additional behavior is moot. FMA operations are enabled by default, but may be disabled directly by the switch –no–fma. FMA operations are not disabled by –fp–model precise.

Precision of math functions

In the Intel® Composer XE 2013 product, the compiler for Intel Xeon Phi coprocessors invokes medium accuracy (< 4 ulp) transcendental functions for both scalar and vector code by default, mostly as calls to libsvml. For Intel Xeon processors, the default is libm (< 0.55 ulp) for scalar code and medium accuracy libsvml (< 4 ulp) for vector code. On Intel Xeon Phi coprocessors, division defaults to medium accuracy inlined code that uses a reciprocal instruction, whereas in the initial 13.0 compiler release, square roots call a medium accuracy SVML function by default (this will change to inlined code in a forthcoming update). An inlined version of square root may be obtained with –fimf–domain–exclusion=15:sqrt (double precision) or –fimf–domain–exclusion=15:sqrtf (single precision). See compiler documentation for details of the excluded domains. The –fimf–domain–exclusion switch may also be used to obtain inlined versions of certain other math functions. High accuracy (typically 0.6 ulp) vectorizable SVML versions of divide, square root, and transcendental functions may be obtained with –fimf–precision=high.

Using –fp–model precise results in high accuracy (<0.55 ulp), scalar calls to libm for transcendentals and to libsvml for square roots. For division, it results in an x87 division instruction. The switches –no–fast–transcendentals, –prec–sqrt, and –prec–div respectively have the same effect. In forthcoming updates, vectorizable inlined code sequences using Intel Xeon Phi coprocessor vector instructions may be used for division and square roots in preference to x87 instructions for these switches and for –fp–model precise. x87 instructions will continue to be generated for –fp–model strict.

Medium accuracy, vectorizable math functions can still be obtained in conjunction with –fp–model precise by specifying –fp–model precise –fast–transcendentals –no–prec–sqrt –no–prec–div. Higher accuracy, vectorizable versions are obtained by adding –fimf–precision=high.

Comparing floating-point results between Intel Xeon Phi coprocessors and Intel Xeon processors

In general, floating-point computations on an Intel Xeon Phi coprocessor may not give bit-for-bit identical results to the equivalent computations on an Intel Xeon processor, even though underlying hardware instructions conform to the same standards. Compiler optimizations may be implemented differently, math functions may be implemented differently, and so on. The fused multiply-add (FMA) instruction available on the Intel Xeon Phi coprocessor is a common source of differences. Nevertheless, the following guidelines may help to minimize differences between results obtained on different platforms, at a cost in performance:

These guidelines are intended to help enhance reproducibility and minimize variations in floating-point results between different platforms. The observed variations do not represent the true numerical uncertainty in your result, which may be much greater.