Intel® Math Kernel Library

for Windows* OS
User’s Guide

March 2009

Document Number: 315930-008US

World Wide Web: http://www.intel.com/software/products/

 Version Version Information Date

-001 Original issue. Documents Intel® Math Kernel Library (Intel® MKL) 9.1 beta January 2007
release.

-002 Documents Intel® MKL 9.1 gold release. Document restructured. More June 2007
aspects of ILP64 interface discussed. Section “Selecting Between Static and
Dynamic Linking” added to chapter 5; section “Changing the Number of Pro-
cessors for Threading at Run Time” and details on redefining memory func-
tions added to chapter 6; section ”Calling LAPACK, BLAS, and CBLAS Routines
from C Language Environments” added to chapter 7. Cluster content is orga-
nized into one separate chapter 9 “Working with Intel® Math Kernel Library
Cluster Software” and restructured, appropriate links added.

-003 Documents Intel® MKL 10.0 Beta release. Layered design model has been September 2007
described in chapter 3 and the content of the entire book adjusted to the
model. New Intel MKL threading controls have been described in chapter 6.
The User’s Guide for Intel MKL merged with the one for Intel MKL Cluster Edi-
tion to reflect consolidation of the respective products.

-004 Documents Intel® MKL 10.0 Gold release. Intel® Compatibility OpenMP* run- October 2007
time compiler library (libiomp) has been described.

-005 Documents Intel® MKL 10.1 beta release. Information on dummy libraries in May 2008
Table "High-level directory structure" has been further detailed. Information
on the Intel MKL configuration file removed. Instructions on creation/configur-
ing of a project running an Intel MKL example in the Microsoft Visual Studio*
IDE have been added to chapter 4. Section "Support for Boost uBLAS Matrix-
Matrix Multiplication" has been added to chapter 7. Chapter “Getting Assis-
tance for Programming in the Microsoft Visual Studio* IDE” has been added.

-006 Documents Intel® MKL 10.1 gold release. Linking examples for IA-32 archi- August 2008
tecture and section "Linking with Computational Libraries" have been added to
chapter 5. Integration of DSS/PARDISO into the layered structure has been
documented. Two Fortran code examples have been added. Integration of a
Microsoft Visual Studio IDE project with Intel MKL has been described.

-007 Documents Intel® MKL 10.2 beta release. Prebuilt Fortran 95 interface librar- January 2009
ies and modules for BLAS and LAPACK have been described. Support for
Intel® Advanced Vector Extensions (Intel® AVX) has been documented. Dis-
continuation of support for dummy libraries and legacy linking model has been
also documented. Chapter 5 has been restructured.

-008 Documents Intel® MKL 10.2 gold release. The document has been consider- March 2009
ably restructured. The "Getting Started" chapter has been enhanced, as well
as the description of the layered model concept. Description of the SP2DP
interface has been added to Chapter 3. The Web-based linking advisor has
been described and referenced in chapters 2 and 5.

ii
INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL® PRODUCTS. NO LICENSE, EXPRESS OR
IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT
AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY
WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL
PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY,
OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
UNLESS OTHERWISE AGREED IN WRITING BY INTEL, THE INTEL PRODUCTS ARE NOT DESIGNED NOR INTENDED FOR ANY
APPLICATION IN WHICH THE FAILURE OF THE INTEL PRODUCT COULD CREATE A SITUATION WHERE PERSONAL INJURY OR
DEATH MAY OCCUR.
Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the
absence or characteristics of any features or instructions marked "reserved" or "undefined." Intel reserves these for future
definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The
information here is subject to change without notice. Do not finalize a design with this information.
The products described in this document may contain design defects or errors known as errata which may cause the product to
deviate from published specifications. Current characterized errata are available on request.
Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.
Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be
obtained by calling 1-800-548-4725, or by visiting Intel's Web Site.

Intel processor numbers are not a measure of performance. Processor numbers differentiate features within each processor
family, not across different processor families. See http://www.intel.com/products/processor_number for details.

This document contains information on products in the design phase of development.

BunnyPeople, Celeron, Celeron Inside, Centrino, Centrino Atom, Centrino Atom Inside, Centrino Inside, Centrino logo, Core
Inside, FlashFile, i960, InstantIP, Intel, Intel logo, Intel386, Intel486, IntelDX2, IntelDX4, IntelSX2, Intel Atom, Intel Atom
Inside, Intel Core, Intel Inside, Intel Inside logo, Intel. Leap ahead., Intel. Leap ahead. logo, Intel NetBurst, Intel NetMerge,
Intel NetStructure, Intel SingleDriver, Intel SpeedStep, Intel StrataFlash, Intel Viiv, Intel vPro, Intel XScale, IPLink, Itanium,
Itanium Inside, MCS, MMX, Oplus, OverDrive, PDCharm, Pentium, Pentium Inside, skoool, Sound Mark, The Journey Inside,
VTune, Xeon, and Xeon Inside are trademarks of Intel Corporation in the U.S. and other countries.

* Other names and brands may be claimed as the property of others.

Copyright © 2007 - 2009, Intel Corporation. All rights reserved.

Microsoft product screen shot(s) reprinted with permission from Microsoft Corporation.

iii
Contents

Chapter 1 Overview
Technical Support ....................................................................... 1-1
About This Document .................................................................. 1-1
Document Organization ........................................................... 1-2
Term and Notational Conventions .............................................. 1-3

Chapter 2 Getting Started

Before You Begin ........................................................................ 2-1
Compiler Support ....................................................................... 2-3
Steps to Get Started ................................................................... 2-4
Check Your Installation ............................................................ 2-4
Set Environment Variables ....................................................... 2-5
Use a Web-based Linking Advisor.............................................. 2-5
Use Intel MKL Code Examples................................................... 2-6

Chapter 3 Intel® Math Kernel Library Structure

High-level Directory Structure ...................................................... 3-1
Layered Model Concept................................................................ 3-3
Sequential Mode of the Library ..................................................... 3-5
Support for ILP64 Programming.................................................... 3-5
Architecture Support ................................................................... 3-7
Directory Structure in Detail......................................................... 3-8
Contents of the Documentation Directory ................................. 3-21

iv
Intel® Math Kernel Library User’s Guide

Chapter 4 Configuring Your Development Environment

Configuring the Microsoft Visual C/C++* Development System to Link
with Intel® MKL ....................................................................... 4-1
Configuring Intel® Visual Fortran to Link with Intel® MKL ............... 4-2
Automatically Configuring a Microsoft Visual Studio* IDE Project with
Intel® MKL.............................................................................. 4-2
Configuring a Project to Run an Intel® MKL Example in the Microsoft
Visual Studio* IDE 2008............................................................ 4-4
Creating and Configuring the Intel® C/C++ and/or Microsoft Visual
C++* 2008 Project............................................................... 4-5
Creating and Configuring the Intel® Visual Fortran Project........... 4-8
Support Files for Intel MKL Examples ...................................... 4-11
Known Limitations ................................................................ 4-11
Configuring the Out-of-Core (OOC) DSS/PARDISO* Solver ............. 4-11

Chapter 5 Linking Your Application with the Intel® Math Kernel Library
Selecting Libraries to Link............................................................ 5-2
Linking with Fortran 95 Interface Libraries ................................. 5-2
Linking with Threading Libraries ............................................... 5-2
Linking with Computational Libraries ......................................... 5-4
Linking with Compiler Support RTLs .......................................... 5-6
Linking with System Libraries ................................................... 5-6
Linking Examples ................................................................... 5-6
Building a Custom DLL ................................................................ 5-9
Intel MKL Custom DLL Builder .................................................. 5-9
Specifying Makefile Parameters ................................................ 5-9
Specifying a List of Functions ................................................. 5-11

Chapter 6 Managing Performance and Memory

Using the Intel® MKL Parallelism .................................................. 6-1
Techniques to Set the Number of Threads .................................. 6-3
Avoiding Conflicts in the Execution Environment ......................... 6-3
Setting the Number of Threads Using OpenMP* Environment
Variable .............................................................................. 6-5
Changing the Number of Threads at Run Time............................ 6-5

v
 Contents

Using Additional Threading Control ........................................... 6-8

Dispatching Intel® Advanced Vector Extensions (Intel® AVX) ......... 6-13
Tips and Techniques to Improve Performance ................................ 6-14
Coding Techniques................................................................. 6-14
Hardware Configuration Tips ................................................... 6-15
Managing Multi-core Performance ............................................ 6-16
Operating on Denormals......................................................... 6-17
FFT Optimized Radices ........................................................... 6-17
Using the Intel® MKL Memory Management.................................. 6-17
Redefining Memory Functions.................................................. 6-18

Chapter 7 Language-specific Usage Options

Using Language-Specific Interfaces with Intel® MKL ....................... 7-1
Mixed-language Programming with Intel® MKL .............................. 7-4
Calling LAPACK, BLAS, and CBLAS Routines from C Language
Environments ...................................................................... 7-5
Using Complex Types in C/C++ ................................................ 7-7
Calling BLAS Functions that Return the Complex Values in C/C++
Code .................................................................................. 7-8
Support for Boost uBLAS Matrix-matrix Multiplication ................. 7-11
Invoking Intel® MKL Functions from Java* Applications.............. 7-13

Chapter 8 Coding Tips

Aligning Data for Numerical Stability ............................................. 8-1

Chapter 9 Working with the Intel® Math Kernel Library Cluster Soft-
ware
MPI support............................................................................... 9-1
Linking with ScaLAPACK and Cluster FFTs ...................................... 9-2
Setting the Number of Threads .................................................... 9-3
Using DLLs ................................................................................ 9-4
ScaLAPACK Tests........................................................................ 9-4
Examples for Linking with ScaLAPACK and Cluster FFT .................... 9-5
Examples for Linking a C Application ......................................... 9-5
Examples for Linking a Fortran Application ................................. 9-5

vi
Intel® Math Kernel Library User’s Guide

Chapter 10 Getting Assistance for Programming in the Microsoft Visual

Studio* IDE
Viewing Intel® MKL Documentation in Visual Studio ..................... 10-1
Using Context-Sensitive Help ..................................................... 10-3
Using the IntelliSense* Capability ............................................... 10-5

Chapter 11 LINPACK and MP LINPACK Benchmarks

Intel® Optimized LINPACK Benchmark for Windows OS* ............... 11-1
Contents ............................................................................. 11-1
Running the Software ........................................................... 11-2
Known Limitations ................................................................ 11-3
Intel® Optimized MP LINPACK Benchmark for Clusters .................. 11-4
Contents ............................................................................. 11-5
Building the MP LINPACK ....................................................... 11-6
New Features....................................................................... 11-7
Benchmarking a Cluster ........................................................ 11-7

Appendix A Intel® Math Kernel Library Language Interfaces Support

Appendix B Support for Third-Party Interfaces

GMP* Functions ......................................................................... B-1
FFTW Interface Support .............................................................. B-1

Index

List of Tables
Table 1-1 Notational Conventions................................................. 1-4
Table 2-1 What You Need to Know Before You Begin....................... 2-1
Table 3-1 High-level Directory Structure ....................................... 3-1
Table 3-2 Compiling for the ILP64 and LP64 Interfaces ................... 3-6
Table 3-3 Integer Types.............................................................. 3-6
Table 3-4 Architecture-specific Implementations ............................ 3-7
Table 3-5 Detailed Structure of the IA-32 Architecture-specific
Directory ia32 ......................................................................... 3-9

vii
 Contents

Table 3-6 Detailed Structure of the Intel® 64 Architecture-specific

Directory em64t ..................................................................... 3-13
Table 3-7 Detailed Structure of the IA-64 Architecture-specific
Directory ia64 ........................................................................ 3-18
Table 3-8 Contents of the doc Directory ....................................... 3-21
Table 5-1 Typical Link Libraries .................................................... 5-1
Table 5-2 Selecting Threading Libraries ........................................ 5-3
Table 5-3 Computational Libraries to Link, by Function Domain ........ 5-4
Table 6-1 How to Avoid Conflicts in the Execution Environment for Your
Threading Model ..................................................................... 6-4
Table 6-2 Environment Variables for Threading Controls.................. 6-9
Table 6-3 Interpretation of MKL_DOMAIN_NUM_THREADS Values ... 6-12
Table 7-1 Interface Libraries and Modules ..................................... 7-1
Table 9-1 Possible Values of MKL_BLACS_MPI................................ 9-4
Table 11-1 Contents of the LINPACK Benchmark ........................... 11-2
Table 11-2 Contents of the MP LINPACK Benchmark ...................... 11-5

List of Examples
Example 6-1 Changing the Number of Threads .............................. 6-5
Example 6-2 Setting the Number of Threads to One...................... 6-10
Example 6-3 Setting An affinity Mask by Operating System Means
Using the Intel® Compiler ....................................................... 6-16
Example 6-4 Redefining Memory Functions .................................. 6-19
Example 6-5 Redefining Memory Functions for Dynamically Linked
Intel® MKL............................................................................ 6-20
Example 7-1 Calling a Complex BLAS Level 1 Function from C ......... 7-9
Example 7-2 Calling a Complex BLAS Level 1 Function from C++.... 7-10
Example 7-3 Using CBLAS Interface Instead of Calling BLAS Directly
from C .................................................................................. 7-11
Example 8-1 Aligning Addresses at 16-byte Boundaries .................. 8-2

viii
Intel® Math Kernel Library User’s Guide

List of Figures
Figure 4-1 Automatically configuring a Microsoft Visual Studio* IDE
project with Intel® MKL ........................................................... 4-3
Figure 7-1 Column-major Order versus Row-major Order................ 7-6
Figure 10-1 Intel® MKL Help in the Microsoft Visual Studio* IDE ... 10-2
Figure 10-2 Help Collections filtered by the "Intel" filter ................ 10-3
Figure 10-3 Context-Sensitive Help for Intel® MKL in the Microsoft
Visual Studio* IDE................................................................. 10-4
Figure 10-4 IntelliSense* Parameter Info.................................... 10-6
Figure 10-5 IntelliSense* Complete Word ................................... 10-7

ix
Overview 1
The Intel® Math Kernel Library (Intel® MKL) offers highly optimized, thread-safe math
routines for science, engineering, and financial applications that require maximum
performance.

Technical Support
Intel provides a support web site, which contains a rich repository of self help information,
including getting started tips, known product issues, product errata, license information,
user forums, and more. Visit the Intel® MKL support website at
http://www.intel.com/software/products/support/ .

About This Document

The Intel MKL User's Guide provides usage information for the library. The usage
information covers the organization, configuration, performance, and accuracy of Intel
MKL, specifics of routine calls in mixed-language programming, linking, and more.
This guide:
• Focuses on the usage information needed to call Intel MKL routines from user's
applications running on the Windows* OS.
• Describes OS-specific usage of Intel MKL, along with OS-independent features.
• Contains usage information for all Intel MKL function domains, listed in Table A-1 (in
Appendix A).
• Assumes you have completed the installation of Intel MKL on your system. If you have
not completed the installation, see the Intel® Math Kernel Library Installation Guide
(file Install.htm).
To reference how to use the library in your application, use this guide in conjunction with
the following documents:

1-1
1 Intel® Math Kernel Library User’s Guide

• The Intel MKL Reference Manual, which provides reference information on routine
functionalities, parameter descriptions, interfaces, calling syntaxes, and return values.
• The Intel® Math Kernel Library for Windows* Release Notes.
This User’s Guide is intended to assist you in mastering the usage of Intel MKL on
Windows. In particular, it:
• Describes post-installation steps to help you start using the library
• Shows you how to configure the library with your development environment
• Acquaints you with the library structure
• Explains how to link your application to the library and provides simple usage scenarios
• Describes how to code, compile, and run your application with Intel MKL for Windows.
This guide is intended for Windows programmers with beginner to advanced experience in
software development.

Document Organization
The document contains the following chapters and appendices:
Chapter 1 Overview. Introduces the Intel MKL usage information and
describes this document’s notational conventions.
Chapter 2 Getting Started. Describes post-installation steps and gives
information needed to start using Intel MKL after its installation.
Chapter 3 Intel® Math Kernel Library Structure. Discusses the structure of
the Intel MKL directory after installation.
Chapter 4 Configuring Your Development Environment. Explains how to
configure Intel MKL with your development environment. In
particular, discusses configuring of a project running an Intel MKL
example in the Microsoft Visual Studio* IDE.
Chapter 5 Linking Your Application with the Intel® Math Kernel Library.
Explains which libraries should be linked with your application for
your particular platform; discusses how to build custom dynamic
libraries.
Chapter 6 Managing Performance and Memory. Discusses Intel MKL
threading; shows coding techniques and gives hardware
configuration tips for improving performance of the library;
explains features of the Intel MKL memory management and, in
particular, shows how to replace memory functions that the library
uses by default with your own ones.
Chapter 7 Language-specific Usage Options. Discusses mixed-language
programming and the use of language-specific interfaces.

1-2
 Overview 1
Chapter 8 Coding Tips. Presents coding tips that may be helpful to your
specific needs.
Chapter 9 Working with the Intel® Math Kernel Library Cluster Software.
Discusses usage of ScaLAPACK and Cluster FFTs; in particular,
describes linking of your application with these function domains,
including C- and Fortran-specific linking examples; gives
information on the supported MPI.
Chapter 10 Getting Assistance for Programming in the Microsoft Visual Studio*
IDE. Discusses Intel MKL features that software engineers can
benefit from when working in the Microsoft Visual Studio* IDE.
Chapter 11 LINPACK and MP LINPACK Benchmarks. Describes the Intel®
Optimized LINPACK Benchmark for Windows* and Intel®
Optimized MP LINPACK Benchmark for Clusters.
Appendix A Intel® Math Kernel Library Language Interfaces Support.
Summarizes information on language interfaces that Intel MKL
provides for each function domain, including the respective header
files.
Appendix B Support for Third-Party Interfaces. Describes some interfaces that
Intel MKL supports.
The document also includes an Index.

Term and Notational Conventions

The following term is used to refer to the operating system:
Windows* OS This term refers to information that is valid on all
supported Windows* operating systems.
The following notation is used in reference to Intel MKL directories:
<mkl_directory> The main directory where Intel MKL is installed.
Replace this placeholder with the specific pathname in
the configuring, linking, and building instructions. For
more information, see Getting Started.
<Intel Compiler Pro directory> The installation directory for the Intel® C++ Compiler
Professional Edition or Intel® Visual Fortran Compiler
Professional Edition. For more information, see Getting
Started.

1-3
1 Intel® Math Kernel Library User’s Guide

Table 1-1 lists the other notational conventions:

Table 1-1 Notational Conventions

Italic Italic is used for emphasis and also indicates document names in body text, for
example:
see Intel MKL Reference Manual
Monospace Indicates commands and command-line options, for example:
lowercase mixed ifort myprog.f mkl_blas95.lib mkl_c.lib libguide40.lib
with uppercase filenames, directory names and pathnames, for example:
C:\Program Files\mkl\10.2.0.004
C/C++ code fragments, for example:
a = new double [SIZE*SIZE];
UPPERCASE Indicates system variables, for example, $MKLPATH
MONOSPACE
Monospace italic Indicates a parameter in discussions: routine parameters, for example, lda;
makefile parameters, for example, functions_list; etc.
When enclosed in angle brackets, indicates a placeholder for an identifier, an
expression, a string, a symbol, or a value, for example, <mkl directory>.
Substitute one of these items for the placeholder.
[ items ] Square brackets indicate that the items enclosed in brackets are optional.
{ item | item } Braces indicate that only one of the items listed between braces should be
selected. A vertical bar ( | ) separates the items

1-4
Getting Started 2
This chapter provides some basic usage information and describes post-installation steps
to help you start using the Intel® Math Kernel Library (Intel® MKL) on Windows* OS.

Before You Begin

Before you begin using Intel MKL, learning a few important concepts will help you get off to
a good start, as shown in Table 2-1.

Table 2-1 What You Need to Know Before You Begin

Target platform Action: Identify the architecture of your target machine:
• IA-32 or compatible
• Intel® 64 or compatible
• IA-64 (Itanium® processor family)
Reason: Because Intel MKL libraries are located in directories corresponding to
your particular architecture (see Architecture Support), you should provide proper
paths on your link lines (see Linking Examples). To configure your development
environment for the use with Intel MKL, set your environment variables using the
script corresponding to your architecture (see Set Environment Variables for
details).

2-1
2 Intel® Math Kernel Library User’s Guide

Table 2-1 What You Need to Know Before You Begin (continued)
Mathematical Action: Identify all Intel MKL function domains that you require:
problem • BLAS
• Sparse BLAS
• LAPACK
• PBLAS
• ScaLAPACK
• Sparse Solver routines
• Vector Mathematical Library functions
• Vector Statistical Library functions
• Fourier Transform functions (FFT)
• Cluster FFT
• Trigonometric Transform routines
• Poisson, Laplace, and Helmholtz Solver routines
• Optimization (Trust-Region) Solver routines
• GMP* arithmetic functions
Reason: The function domain you intend to use narrows the search in the
Reference Manual for specific routines you need. Additionally, if you are using the
Intel MKL cluster software, your link line is function-domain specific (see Working
with the Intel® Math Kernel Library Cluster Software). Coding tips may also depend
on the function domain (see Tips and Techniques to Improve Performance).
Programming Action: Though Intel MKL provides support for both Fortran and C/C++
language programming, not all the function domains support a particular language
environment, for example, C/C++ or Fortran 90/95. Identify the language
interfaces that your function domains support (see Intel® Math Kernel Library
Language Interfaces Support).
Reason: In case your function domain does not directly support the needed
environment, you can use mixed-language programming (see Mixed-language
Programming with Intel® MKL).
For a list of language-specific interface libraries and modules and an example how
to generate them, see also Using Language-Specific Interfaces with Intel® MKL.
Range of integer data Action: If your system is based on the Intel 64 or IA-64 architecture, identify
whether your application performs calculations with huge data arrays (of more than
231-1 elements).
Reason: To operate on huge data arrays, you need to select the ILP64 interface,
where integers are 64-bit; otherwise, use the default, LP64, interface, where
integers are 32-bit (see Support for ILP64 Programming).

2-2
 Getting Started 2
Table 2-1 What You Need to Know Before You Begin (continued)
Threading model Action: Identify whether and how your application is threaded:
• Threaded with the Intel® compiler
• Threaded with a third-party compiler
• Not threaded
Reason: The compiler you use to thread your application determines which
threading library you should link with your application. For applications threaded
with a third-party compiler you may need to use Intel MKL in the sequential mode
(for more information, see Sequential Mode of the Library and Linking with
Threading Libraries).
Action: Determine the number of threads you want Intel MKL to use.
Reason: Intel MKL is based on the OpenMP* threading. By default, the OpenMP*
software sets the number of threads that Intel MKL uses. If you need a different
number, you have to set it yourself using one of the available mechanisms. For
more information, see Using the Intel® MKL Parallelism.
Linking model Action: Decide which linking model is appropriate for linking your application with
Intel MKL libraries:
• Static
• Dynamic
Reason: For information on the benefits of each linking model, linking examples,
link libraries, and other linking topics, like how to save disk space by creating a
custom dynamic library, see Linking Your Application with the Intel® Math Kernel
Library.
MPI used Action: Decide what MPI you will use with the Intel MKL cluster software. You are
strongly encouraged to use Intel® MPI 3.x.
Reason: To link your application with ScaLAPACK and/or Cluster FFT, the libraries
corresponding to your particular MPI should be listed on the link line (see Working
with the Intel® Math Kernel Library Cluster Software).

Compiler Support
Intel MKL supports compilers identified in the Release Notes. However, the library has been
successfully used with other compilers as well.

NOTE. There is currently no support for compilers that require OMF file
format.

Although Compaq no longer supports the Compaq Visual Fortran* (CVF) compiler, Intel
MKL still preserves the CVF interface in IA-32 architecture implementations. This interface
can be used with the Intel® Fortran Compiler by employing the /Gm compiler option.

2-3
2 Intel® Math Kernel Library User’s Guide

There are both cdecl (default interface of the Microsoft Visual C* application) and stdcall
(default CVF interface) versions of the library. The cdecl version of Intel MKL can be
obtained by linking with interface library mkl_intel_c.lib and the stdcall version can be
obtained by linking with mkl_intel_s.lib. The dynamic counterparts of these interface
libraries are mkl_intel_c_dll.lib and mkl_intel_s_dll.lib. Whether you choose to
link with cdecl or stdcall, depends on factors that only you can determine.
Below are compiler-specific recommendations for linking with the cdecl or stdcall version of
the library:
• Using a C compiler
If you are using mkl_intel_s[_dll].lib, call Intel MKL routines from C with a
statement like
extern __stdcall name( <prototype variable1>, <prototype variable2>, .. );
where stdcall is actually the CVF compiler default compilation, which differs from
stdcall in how strings are passed to the routine. Because the default CVF format is not
identical with stdcall, you must handle strings in the calling sequence specially.
Dealing with this issue is complex, and you are advised to refer to sections on
interfaces in the CVF documentation.
If you are using mkl_intel_c[_dll].lib, call the routines with a similar declaration:
<type> name( <prototype variable1>, <prototype variable2>, .. );
• Using the CVF compiler
Similarly, the CVF compiler will link with mkl_intel_s[_dll].lib if routines are
compiled with the default interface. However, if you compile with the option
/iface=(cref,nomixed_str_len_arg), the compiler will link with
mkl_intel_c[_dll].lib
• Using the Intel® Fortran compiler
The Intel Fortran compiler will link with mkl_intel_c[_dll].lib by default. If the
/Gm option is used, call mkl_intel_s[_dll].lib (/Gm enables CVF and Powerstation
calling convention compatibility, so does /iface:cvf )
Intel MKL provides a set of include files to simplify program development by specifying
enumerated values and prototypes for the respective functions (for the list of include files,
see Table A-2). Calling Intel MKL functions from your application without an appropriate
include file may lead to incorrect behavior of the functions.

Steps to Get Started

This section helps you to get started with Intel MKL on Windows* OS.

Check Your Installation

After installing Intel MKL, verify that the library has been properly installed and configured.

2-4
 Getting Started 2
1. Check that the directory you chose for the installation has been created. The Intel MKL
default installation directory may be one of the following:
— C:\Program Files\Intel\MKL\RR.r.y.xxx, where RR.r is the version
number, y is the release-update number, and xxx is the package number, for
example, C:\Program Files\Intel\MKL\10.2.0.004
— <Intel Compiler Pro directory>\MKL, for example, C:\Program
Files\Intel\Complier\11.1\015\MKL.
2. If you choose to keep multiple versions of Intel MKL installed on your system, update
your build scripts so that they point to the desired version.
3. Check that the following three files are placed in the tools/environment directory:
mklvars32.bat
mklvarsem64t.bat
mklvars64.bat
You can use these files to set environment variables in the current user shell.
4. To check the high-level and detailed structure of the Intel MKL installation directory,
see Chapter 3.

Set Environment Variables

When the installation of Intel MKL for the Windows* OS is complete, you can use files
mklvars32.bat, mklvarsem64t.bat, and mklvars64.bat in the tools\environment
directory to set the environment variables path, lib, include, LIBRARY_PATH, CPATH, and
FPATH in the user shell. For further configuring the library, see Chapter 4.

Use a Web-based Linking Advisor

Intel MKL provides a web-based linking advisor to help you choose the libraries and options
to specify on a link line for your application.
The tool is available at http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor.
The advisor types to a screen the link line depending on your Intel MKL usage scenario,
specifically: the operating system, architecture, compiler, static or dynamic linking model,
length of integers (32-bit or 64-bit), threaded or sequential mode of Intel MKL operation,
and so on.
Copy-paste the output to your link line.
For more information on linking with Intel MKL, see Chapter 5 and specifically Table 5-1 for
a list of typical link libraries.

2-5
2 Intel® Math Kernel Library User’s Guide

Use Intel MKL Code Examples

Intel MKL package includes code examples, located in the examples subdirectory of the
Intel MKL installation directory. The examples provide the most direct way for you to find
out:
• Whether Intel MKL is working on your system
• How you should call the library
• How to link the library
The examples are grouped in subdirectories mainly by Intel MKL function domains and
programming languages. For instance, subdirectory examples\spblas contains Sparse
BLAS examples, and subdirectory examples\vmlc contains VML examples in C. Source
code for the examples is in the next level sources subdirectory.
To compile, build, and run the examples, use the makefile provided. For information on
how to use it, refer to the makefile header.
See also:
High-level Directory Structure
Configuring a Project to Run an Intel® MKL Example in the Microsoft Visual Studio* IDE
2008.

2-6
Intel® Math Kernel Library
Structure 3
The chapter discusses the structure of the Intel® Math Kernel Library (Intel® MKL),
including the Intel MKL directory structure, architecture-specific implementations,
supported programming interfaces, and more.
Starting with version 10.0, Intel MKL employs a layered model to streamline the library
structure, reduce its size, and add usage flexibility.
See also: Layered Model Concept.

High-level Directory Structure

Table 3-1 shows a high-level directory structure of Intel MKL after installation.

Table 3-1 High-level Directory Structure

Directory Comment
<mkl directory> Intel MKL main directory

<mkl directory>\benchmarks\linpack Contains a Shared-Memory (SMP) version of the LIN-

PACK benchmark
<mkl Contains an MPI version of the LINPACK benchmark
directory>\benchmarks\mp_linpack
<mkl directory>\doc Contains documentation for a stand-alone Intel MKL
<mkl directory>\em64t\bin Contains DLLs for applications running on processors
with Intel® 64 architecture (formerly, Intel® EM64T)
<mkl directory>\em64t\lib Contains static libraries and static interfaces to DLLs for
the Intel® 64 architecture
<mkl directory>\examples A source and data for examples

3-1
3 Intel® Math Kernel Library User’s Guide

Table 3-1 High-level Directory Structure (continued)

Directory Comment
<mkl directory>\help Contains Help2-format files for integration of the docu-
mentation for a stand-alone Intel MKL with the Microsoft
Visual Studio* IDE.
<mkl directory>\ia32\bin Contains DLLs for applications running on processors
with the IA-32 architecture
<mkl directory>\ia32\lib Contains static libraries and static interfaces to DLLs for
the IA-32 architecture
<mkl directory>\ia64\bin Contains DLLs for applications running on processors
with the IA-64 architecture
<mkl directory>\ia64\lib Contains static libraries and static interfaces to DLLs for
the IA-64 architecture
<mkl directory>\include Contains INCLUDE files for the library routines, as well
as for tests and examples
<mkl directory>\include\em64t\ilp64 Contains BLAS951 and LAPACK952 .mod files for the
Intel® 64 architecture, Intel® Fortran compiler, and
ILP64 interface
<mkl directory>\include\em64t\lp64 Contains BLAS95 and LAPACK95 .mod files for the
Intel® 64 architecture, Intel® Fortran compiler, and
LP64 interface
<mkl directory>\include\ia32 Contains BLAS95 and LAPACK95 .mod files for the
IA-32 architecture and Intel® Fortran compiler
<mkl directory>\include\ia64\ilp64 Contains BLAS95 and LAPACK95 .mod files for the IA-64
architecture, Intel® Fortran compiler, and ILP64
interface
<mkl directory>\include\ia64\lp64 Contains BLAS95 and LAPACK95 .mod files for the
IA-64 architecture, Intel® Fortran compiler, and LP64
interface
<mkl directory>\interfaces\blas95 Contains Fortran 95 wrappers for BLAS and a makefile
to build the library
<mkl Contains wrappers for MPI FFTW version 2.x to call Intel
directory>\interfaces\fftw2x_cdft MKL Cluster FFT interface

<mkl directory>\interfaces\fftw2xc Contains wrappers for FFTW version 2.x (C interface) to

call Intel MKL FFTs
<mkl directory>\interfaces\fftw2xf Contains wrappers for FFTW version 2.x (Fortran inter-
face) to call Intel MKL FFTs
<mkl directory>\interfaces\fftw3xc Contains wrappers for FFTW version 3.x (C interface) to
call Intel MKL FFTs
<mkl directory>\interfaces\fftw3xf Contains wrappers for FFTW version 3.x (Fortran inter-
face) to call Intel MKL FFTs

3-2
 Intel® Math Kernel Library Structure 3
Table 3-1 High-level Directory Structure (continued)
Directory Comment
<mkl directory>\interfaces\lapack95 Contains Fortran 95 wrappers for LAPACK and a make-
file to build the library
<mkl directory>\tests Contains source and data files for tests

<mkl directory>\tools\builder Contains tools for creating custom dynamically linkable

libraries
<mkl directory>\tools\environment Contains batch files to set environmental variables in
the user shell
<Intel Compiler Pro directory>\ Contains documentation for Intel MKL included in the
documentation\en_US\MKL Intel® C++ Compiler Professional Edition or Intel®
Visual Fortran Compiler Professional Edition.
<Intel Compiler Pro For Intel MKL included in the Intel® C++ Compiler Pro-
directory>\vshelp\ fessional Edition or Intel® Visual Fortran Compiler Pro-
1033\intel.mkldocs.en fessional Edition, contains Help2-format files for
integration of the Intel MKL documentation with the
Microsoft Visual Studio* IDE.
1. Fortran 95 interface to BLAS
2. Fortran 95 interface to LAPACK

Layered Model Concept

Starting with release 10.0, Intel MKL employs a layered model.
There are four essential parts of the library:
1. Interface layer
2. Threading layer
3. Computational layer
4. Compiler Support Run-time libraries.
The first part adapts Intel MKL to interface-related issues, for example, whether to use
32-bit or 64-bit integer types, or how different compilers return function values. The
second part adapts the library to the OpenMP* implementations used by different
threading compilers or to the non-threaded usage mode. The Computational layer is the
bulk of the library, which is isolated in a separate part to save space. Rather than generate
one library for each of the cases within those parts and thus create a geometric explosion
of possibilities, Intel MKL has broken down each part into independent libraries
corresponding to those cases. You can combine these libraries to meet your needs. Once
the interface library is selected, the threading library you select picks up that interface, and
the computational libraries use interfaces and threading chosen in the first two layers.

3-3
3 Intel® Math Kernel Library User’s Guide

Interface Layer. This layer provides matching between compiled code of your application
and the threading and/or computational parts of the library. This layer provides:
• The cdecl and CVF default interfaces
• The LP64 and ILP64 interfaces
(see Support for ILP64 Programming for details)
• Compatibility with compilers that return function values differently
• A mapping between single-precision names and double-precision names for applications using
Cray*-style naming (SP2DP interface)

NOTE. SP2DP interface supports Cray-style naming in applications

targeted for the Intel 64 or IA-64 architecture and using the ILP64
interface. SP2DP interface provides a mapping between single-precision
names (for both real and complex types) in the application and
double-precision names in Intel MKL BLAS and LAPACK. Function names
are mapped as shown in the following example for BLAS functions *GEMM:
SGEMM -> DGEMM
DGEMM -> DGEMM
CGEMM -> ZGEMM
ZGEMM -> ZGEMM
Mind that no changes are made to double-precision names.

Threading Layer. This layer provides:

• A way to link threaded Intel MKL with different threading compilers.
• An ability for the user to link with a threaded or sequential mode of the library.
This layer is compiled for different environments (threaded or sequential) and compilers
(from Intel, Microsoft, and so on).

Computational Layer. This layer is the heart of Intel MKL. For any given processor
architecture (IA-32, IA-64, or Intel® 64) and OS, this layer has only one computational
library to link with, regardless of the Interface and Threading layer. The Computational
layer accommodates multiple architectures through identification of architecture features
and chooses the appropriate binary code at run time.

Compiler Support Run-time Libraries (RTL, for brevity). This layer provides RTL support. Not
all RTLs are delivered with Intel MKL. Except those that are relevant to the Intel MKL
cluster software, the only RTLs provided are Intel® compiler RTLs: Intel® Compatibility
OpenMP* run-time library (libiomp) and Intel® Legacy OpenMP* run-time library

3-4
 Intel® Math Kernel Library Structure 3
(libguide). To thread using third-party threading compilers, use libraries in the Threading
layer or an appropriate compatibility library (for more information, see Linking with
Threading Libraries).

Sequential Mode of the Library

You can use Intel MKL in a sequential (non-threaded) mode. It requires no Compatibility
OpenMP* or Legacy OpenMP* run-time library, and does not respond to the environment
variable OMP_NUM_THREADS or its Intel MKL equivalents. In this mode, Intel MKL runs
unthreaded code. However, it is thread-safe1, which means that you can use it in a parallel
region from your own OpenMP* code. You should use the library in the sequential mode
only if you have a particular reason not to use Intel MKL threading. The sequential mode
may be helpful when using Intel MKL with programs threaded with some non-Intel
compilers or in other situations where you may, for various reasons, need a non-threaded
version of the library (for instance, in some MPI cases). To obtain Intel MKL running in the
sequential mode, in the Threading layer, choose the *sequential.* library to link.
See also:
Directory Structure in Detail
Using the Intel® MKL Parallelism
Avoiding Conflicts in the Execution Environment
Linking Examples.

Support for ILP64 Programming

The Intel MKL ILP64 libraries use the 64-bit integer type (necessary for indexing huge
arrays, with more than 231-1 elements), whereas the LP64 libraries index arrays with the
32-bit integer type.
The LP64 and ILP64 interfaces are supported in the Interface layer (see Layered Model
Concept and Directory Structure in Detail for more information).
The ILP64 interface is provided for the following two reasons:
• To support huge data arrays (with more than 231-1 elements)
• To enable compiling your Fortran code with the /4I8 compiler option

1. Except for LAPACK deprecated routines ?lacon, ?lasq3, and ?lasq4.

3-5
3 Intel® Math Kernel Library User’s Guide

It is up to you to choose which interface to use. Choose the LP64 interface for compatibility
with the previous Intel MKL versions because "LP64" is just a new name for the only
interface that the Intel MKL versions lower than 9.1 provided. Choose the ILP64 interface if
your application uses Intel MKL for calculations with huge data arrays or the library may be
used so in future.
Intel MKL provides the same include directory for the ILP64 and LP64 interfaces.

Compiling for LP64/ILP64

Table 3-2 shows how to compile for the ILP64 and LP64 interfaces:

Table 3-2 Compiling for the ILP64 and LP64 Interfaces

Fortran
Compiling for ILP64 ifort /4I8 /I<mkl drectory>\include …
Compiling for LP64 ifort /I<mkl drectory>\include …
C or C++
Compiling for ILP64 icl /DMKL_ILP64 /I<mkl directory>\include …
Compiling for LP64 icl /I<mkl directory>\include …

NOTE. Linking of the application compiled with the /4I8 or /DMKL_ILP64

option to the LP64 libraries may result in unpredictable consequences and
erroneous output.

Coding for ILP64

You do not need to change existing code if you are not using the ILP64 interface.
To migrate to ILP64 or write new code for ILP64, use appropriate types for parameters of
the Intel MKL functions and subroutines (see Table 3-3):

Table 3-3 Integer Types

Fortran C or C++
32-bit integers INTEGER*4 or int
INTEGER(KIND=4)
Universal integers for ILP64/LP64: INTEGER MKL_INT
• 64-bit for ILP64 without specifying KIND
• 32-bit otherwise

3-6
 Intel® Math Kernel Library Structure 3
Table 3-3 Integer Types (continued)
Fortran C or C++
Universal integers for ILP64/LP64: INTEGER*8 or MKL_INT64
• 64-bit integers INTEGER(KIND=8)
FFT interface integers for ILP64/LP64 INTEGER MKL_LONG
without specifying KIND

Browsing the Intel MKL Include Files

Given a function with integer parameters, the Reference Manual does not explain which
parameters become 64-bit and which remain 32-bit for ILP64, and you are encouraged to
browse the include files, examples, and tests for the ILP64 interface details. For their
location, see Table 3-1.
You are encouraged to start with browsing the include files, listed in Table A-2.
Some function domains that support only a Fortran interface (see Table A-1), provide
header files for C/C++ in the include directory. Such *.h files enable using a Fortran binary
interface from C/C++ code. These files can also be used to understand the ILP64 usage.

Limitations
All Intel MKL function domains support ILP64 programming with the following exceptions:
• FFTW interfaces to Intel MKL:
— FFTW 2.x wrappers do not support ILP64.
— FFTW 3.2 wrappers support ILP64 by a dedicated set of functions plan_guru64.
• GMP* arithmetic functions do not support ILP64.

Architecture Support
Intel MKL for Windows* OS provides three architecture-specific implementations. Table 3-4
lists the supported architectures and directories where each architecture-specific
implementation is located.

Table 3-4 Architecture-specific Implementations

Architecture Location
IA-32 or compatible <mkl directory>\ia32
Intel® 64 or compatible <mkl directory>\em64t
IA-64 <mkl directory>\ia64

3-7
3 Intel® Math Kernel Library User’s Guide

See a detailed structure of these directories in Table 3-5, Table 3-6, and Table 3-7.

Directory Structure in Detail

The information in the tables below shows a detailed structure of the Intel MKL
architecture-specific directories. For the list of additional interface libraries that can be
generated in these directories using makefiles in the interfaces directory, see Using
Language-Specific Interfaces with Intel® MKL. For the contents of the doc directory, see
Contents of the Documentation Directory. For the contents of subdirectories in the
benchmarks directory, see LINPACK and MP LINPACK Benchmarks. Note that in MKL 10.2,
libraries that provided link line compatibility with the Intel MKL versions 9.x and lower were
removed.

3-8
 Intel® Math Kernel Library Structure 3

Table 3-5 Detailed Structure of the IA-32 Architecture-specific Directory ia32

File Contents
Static Libraries
Interface layer
lib\mkl_blas95.lib Fortran 95 interface library for BLAS for the Intel® Fortran compiler

lib\mkl_intel_c.lib cdecl interface library

lib\mkl_intel_s.lib CVF default interface library

lib\mkl_lapack95.lib Fortran 95 interface library for LAPACK for the Intel® Fortran compiler

Threading layer
lib\mkl_intel_thread. Threading library for the Intel® compilers
lib
lib\mkl_pgi_thread.lib Threading library for the PGI* compiler

lib\mkl_sequential.lib Sequential library

Computational layer
lib\mkl_cdft_core. Cluster version of FFTs
lib
lib\mkl_core.lib Kernel library for IA-32 architecture

lib\mkl_scalapack_ ScaLAPACK routines

core.lib
lib\mkl_solver.lib Iterative Sparse Solver, Trust Region Solver, and GMP routines

lib\mkl_solver_ Sequential version of Iterative Sparse Solver and Trust Region Solver
sequential.lib routine library

RTL
lib\libguide.lib Intel® Legacy OpenMP* run-time library for static linking

lib\libiomp5mt.lib Intel® Compatibility OpenMP* run-time library for static linking

lib\mkl_blacs_ BLACS routines supporting Intel MPI

intelmpi.lib
lib\mkl_blacs_ BLACS routines supporting MPICH2
mpich2.lib

3-9
3 Intel® Math Kernel Library User’s Guide

Table 3-5 Detailed Structure of the IA-32 Architecture-specific Directory ia32

(continued)
File Contents
Dynamic Libraries
Interface layer
lib\mkl_intel_c_dll. cdecl interface library for dynamic linking
lib
lib\mkl_intel_s_dll. CVF default interface library for dynamic
lib linking

Threading layer
bin\mkl_intel_thread. Dynamic threading library for the Intel compilers
dll
bin\mkl_pgi_thread.dll Dynamic threading library for the PGI* compiler

bin\mkl_sequential.dll Dynamic sequential library

lib\mkl_intel_thread_ Threading library for dynamic linking with the Intel compilers
dll.lib
lib\mkl_pgi_thread_ Threading library for dynamic linking with the PGI* compiler
dll.lib
lib\mkl_sequential_ Sequential library for dynamic linking
dll.lib

3-10
 Intel® Math Kernel Library Structure 3
Table 3-5 Detailed Structure of the IA-32 Architecture-specific Directory ia32
(continued)
File Contents
Computational layer
bin\libimalloc.dll Dynamic library to support renaming of
memory functions
bin\mkl_cdft_core. Cluster FFT dynamic library
dll
bin\mkl_def.dll Default kernel (Intel® Pentium®, Pentium® Pro, Pentium® II, and
Pentium® III processors)
bin\mkl_lapack.dll LAPACK and DSS/PARDISO routines and drivers

bin\mkl_p4.dll Pentium® 4 processor kernel

bin\mkl_p4m.dll Kernel for processors based on the Intel® Core™ microarchitecture

(except Intel® Core™ Duo and Intel® Core™ Solo processors for
which mkl_p4p.dll is intended)
bin\mkl_p4m3.dll Kernel for the Intel® Core™ i7 processors

bin\mkl_p4p.dll Kernel for the Intel® Pentium® 4 processor with Streaming SIMD
Extensions 3 (SSE3)
bin\mkl_scalapack_ ScaLAPACK routines
core.dll
bin\mkl_vml_def.dll VML/VSL part of default kernel for old Intel® Pentium® processors

bin\mkl_vml_ia.dll VML/VSL default kernel for newer Intel® architecture processors

bin\mkl_vml_p4.dll VML/VSL part of Pentium® 4 processor kernel

bin\mkl_vml_p4m.dll VML/VSL for processors based on the Intel® Core™ microarchitecture

(except Intel® Core™ Duo and Intel® Core™ Solo processors for
which mkl_vml_p4p.dll is intended).
bin\mkl_vml_p4m2.dll VML/VSL for 45nm Hi-k Intel® Core™2 and Intel Xeon® processor
families
bin\mkl_vml_p4m3.dll VML/VSL for the Intel® Core™ i7 processors

bin\mkl_vml_p4p.dll VML/VSL for Pentium® 4 processor with Streaming SIMD Extensions 3

(SSE3)
lib\mkl_cdft_core_ Cluster FFT library for dynamic linking
dll.lib
lib\mkl_core_dll.lib Library dispatcher for dynamic load of processor-specific kernel library

lib\mkl_scalapack_ ScaLAPACK routine library for dynamic linking

core_dll.lib

3-11
3 Intel® Math Kernel Library User’s Guide

Table 3-5 Detailed Structure of the IA-32 Architecture-specific Directory ia32

(continued)
File Contents
RTL
bin\libguide40.dll Intel® Legacy OpenMP* run-time dynamic library

bin\libiomp5md.dll Intel® Compatibility OpenMP* run-time dynamic library

bin\mkl_blacs.dll BLACS routines

bin\mkl_blacs_ BLACS routines supporting Intel MPI

intelmpi.dll
bin\mkl_blacs_ BLACS routines supporting MPICH2
mpich2.dll
bin\1033\mkl_msg.dll Catalog of Intel MKL messages in English

bin\1041\mkl_msg.dll Catalog of Intel MKL messages in Japanese

lib\libguide40.lib Intel® Legacy OpenMP* run-time library for dynamic linking

lib\libiomp5md.lib Intel® Compatibility OpenMP* run-time library for dynamic linking

lib\mkl_blacs_dll. BLACS interface library for dynamic linking

lib

3-12
 Intel® Math Kernel Library Structure 3

Table 3-6 Detailed Structure of the Intel® 64 Architecture-specific Directory em64t

File Contents
Static Libraries
Interface layer
lib\mkl_blas95_ilp64.lib Fortran 95 interface library for BLAS for the Intel® Fortran
compiler. Supports the ILP64 interface
lib\mkl_blas95_lp64. Fortran 95 interface library for BLAS for the Intel® Fortran
lib compiler. Supports the LP64 interface

lib\mkl_intel_ilp64.lib ILP64 interface library for the Intel compilers

lib\mkl_intel_lp64.lib LP64 interface library for the Intel compilers

lib\mkl_lapack95_ Fortran 95 interface library for LAPACK for the Intel® Fortran
ilp64.lib compiler. Supports the ILP64 interface

lib\mkl_lapack95_lp64. Fortran 95 interface library for LAPACK for the Intel® Fortran
lib compiler. Supports the LP64 interface

Threading layer
lib\mkl_intel_thread.lib Threading library for the Intel compilers

lib\mkl_pgi_thread.lib Threading library for the PGI* compiler

lib\mkl_sequential.lib Sequential library

3-13
3 Intel® Math Kernel Library User’s Guide

Table 3-6 Detailed Structure of the Intel® 64 Architecture-specific Directory em64t (continued)
File Contents
Computational layer
lib\mkl_cdft_core.lib Cluster version of FFTs

lib\mkl_core.lib Kernel library for the Intel® 64 architecture

lib\mkl_scalapack_ilp64. ScaLAPACK routine library supporting the ILP64 interface

lib
lib\mkl_scalapack_ ScaLAPACK routine library supporting the LP64 interface
lp64.lib
lib\mkl_solver_ilp64.lib Iterative Sparse Solver and Trust Region Solver routine library
supporting the ILP64 interface
lib\mkl_solver_ilp64_ Sequential version of Iterative Sparse Solver and Trust Region
sequential.lib Solver routine library supporting the ILP64 interface

lib\mkl_solver_lp64.lib Iterative Sparse Solver, Trust Region Solver, and GMP routine
library supporting the LP64 interface
lib\mkl_solver_lp64_ Sequential version of Iterative Sparse Solver, Trust Region Solver,
sequential.lib and GMP routine library supporting the LP64 interface

RTL
lib\libguide.lib Intel® Legacy OpenMP* run-time library for static linking

lib\libiomp5mt.lib Intel® Compatibility OpenMP* run-time library for static linking

lib\mkl_blacs_ ILP64 version of BLACS routines supporting Intel MPI

intelmpi_ilp64.lib
lib\mkl_blacs_intelmpi_ LP64 version of BLACS routines supporting Intel MPI
lp64.lib
lib\mkl_blacs_mpich2_ ILP64 version of BLACS routines supporting MPICH2
ilp64.lib
lib\mkl_blacs_mpich2_ LP64 version of BLACS routines supporting MPICH2
lp64.lib
lib\mkl_blacs_msmpi_ ILP64 version of BLACS routines supporting Microsoft* MPI
ilp64.lib
lib\mkl_blacs_msmpi_ LP64 version of BLACS routines supporting Microsoft* MPI
lp64.lib

3-14
 Intel® Math Kernel Library Structure 3
Table 3-6 Detailed Structure of the Intel® 64 Architecture-specific Directory em64t (continued)
File Contents
Dynamic Libraries
Interface layer
lib\mkl_intel_ilp64_dll. ILP64 interface library for dynamic linking with the Intel
lib compilers

lib\mkl_intel_lp64_ LP64 interface library for dynamic linking with the Intel compilers
dll.lib
Threading layer
bin\mkl_intel_thread.dll Dynamic threading library for the Intel compilers

bin\mkl_pgi_thread.dll Dynamic threading library for the PGI* compiler

bin\lib\mkl_sequential.dll Dynamic sequential library

lib\mkl_intel_thread_ Threading library for dynamic linking with the Intel compilers
dll.lib
lib\mkl_pgi_thread_dll.lib Threading library for dynamic linking with the PGI* compiler

lib\mkl_sequential_dll.lib Sequential library for dynamic linking

3-15
3 Intel® Math Kernel Library User’s Guide

Table 3-6 Detailed Structure of the Intel® 64 Architecture-specific Directory em64t (continued)
File Contents
Computational layer
bin\libimalloc.dll Dynamic library to support renaming of
memory functions
bin\mkl_avx.dll Kernel optimized for the Intel® Advanced Vector Extensions
(Intel® AVX).
bin\mkl_cdft_core.dll Cluster FFT dynamic library

bin\mkl_def.dll Default kernel for the Intel® 64 architecture

bin\mkl_lapack.dll LAPACK and DSS/PARDISO routines and drivers

bin\mkl_mc.dll Kernel for processors based on the Intel® Core™ microarchitec-

ture
bin\mkl_mc3.dll Kernel for the Intel® Core™ i7 processors

bin\mkl_p4n.dll Kernel for the Intel® Xeon® processor using the Intel® 64 archi-
tecture
bin\mkl_scalapack_ ScaLAPACK routine library supporting the ILP64 interface
ilp64.dll
bin\mkl_scalapack_ ScaLAPACK routine library supporting the LP64 interface
lp64.dll
bin\mkl_vml_avx.dll VML/VSL optimized for the Intel® Advanced Vector Extensions
(Intel® AVX).
bin\mkl_vml_def.dll VML/VSL part of default kernel

bin\mkl_vml_mc.dll VML/VSL for processors based on the Intel® Core™ microarchi-

tecture
bin\mkl_vml_mc2.dll VML/VSL for 45nm Hi-k Intel® Core™2 and Intel Xeon® proces-
sor families
bin\mkl_vml_mc3.dll VML/VSL for the Intel® Core™ i7 processors

bin\mkl_vml_p4n.dll VML/VSL for the Intel® Xeon® processor using the Intel® 64
architecture
lib\mkl_cdft_core_ Cluster FFT library for dynamic linking
dll.lib
lib\mkl_core_dll.lib Library dispatcher for dynamic load of processor-specific kernel
library
lib\mkl_scalapack_ilp64_ ScaLAPACK routine library for dynamic linking supporting the
dll.lib ILP64 interface

lib\mkl_scalapack_lp64_ ScaLAPACK routine library for dynamic linking supporting the

dll.lib LP64 interface

3-16
 Intel® Math Kernel Library Structure 3
Table 3-6 Detailed Structure of the Intel® 64 Architecture-specific Directory em64t (continued)
File Contents
RTL
bin\libguide40.dll Intel® Legacy OpenMP* run-time dynamic library

bin\libiomp5md.dll Intel® Compatibility OpenMP* run-time dynamic library

bin\mkl_blacs_ilp64. ILP64 version of BLACS routines

dll
bin\mkl_blacs_intelmpi_ ILP64 version of BLACS routines supporting Intel MPI
ilp64.dll
bin\mkl_blacs_intelmpi_ LP64 version of BLACS routines supporting Intel MPI
lp64.dll
bin\mkl_blacs_lp64.dll LP64 version of BLACS routines

bin\mkl_blacs_mpich2_ ILP64 version of BLACS routines supporting MPICH2

ilp64.dll
bin\mkl_blacs_mpich2_ LP64 version of BLACS routines supporting MPICH2
lp64.dll
bin\mkl_blacs_msmpi_ ILP64 version of BLACS routines supporting Microsoft* MPI
ilp64.dll
bin\mkl_blacs_msmpi_ LP64 version of BLACS routines supporting Microsoft* MPI
lp64.dll
bin\1033\mkl_msg.dll Catalog of Intel MKL messages in English

bin\1041\mkl_msg.dll Catalog of Intel MKL messages in Japanese

lib\libguide40.lib Intel® Legacy OpenMP* run-time library for dynamic linking

lib\libiomp5md.lib Intel® Compatibility OpenMP* run-time library for dynamic

linking
lib\mkl_blacs_ilp64_ ILP64 version of BLACS interface library for dynamic linking
dll.lib
lib\mkl_blacs_lp64_dll. LP64 version of BLACS interface library for dynamic linking
lib

3-17
3 Intel® Math Kernel Library User’s Guide

Table 3-7 Detailed Structure of the IA-64 Architecture-specific Directory ia64

File Contents
Static Libraries
Interface layer
lib\mkl_blas95_ilp64. Fortran 95 interface library for BLAS for the Intel® Fortran
lib compiler. Supports the ILP64 interface

lib\mkl_blas95_lp64.lib Fortran 95 interface library for BLAS for the Intel® Fortran
compiler. Supports the LP64 interface
lib\mkl_intel_ilp64.lib ILP64 interface library for the Intel compilers

lib\mkl_intel_lp64.lib LP64 interface library for the Intel compilers

lib\mkl_lapack95_ilp64. Fortran 95 interface library for LAPACK for the Intel® Fortran
lib compiler. Supports the ILP64 interface

lib\mkl_lapack95_lp64. Fortran 95 interface library for LAPACK for the Intel® Fortran
lib compiler. Supports the LP64 interface

Threading layer
lib\mkl_intel_thread.lib Threading library for the Intel compilers

lib\mkl_sequential.lib Sequential library

3-18
 Intel® Math Kernel Library Structure 3
Table 3-7 Detailed Structure of the IA-64 Architecture-specific Directory ia64
(continued)
File Contents
Computational layer
lib\mkl_cdft_core.lib Cluster version of FFTs

lib\mkl_core.lib Kernel library for the IA-64 architecture

lib\mkl_scalapack_ ScaLAPACK routine library supporting the ILP64 interface

ilp64.lib
lib\mkl_scalapack_lp64. ScaLAPACK routine library supporting the LP64 interface
lib
lib\mkl_solver_ilp64. Iterative Sparse Solver and Trust Region Solver routine library
lib supporting the ILP64 interface

lib\mkl_solver_ilp64_ Sequential version of Iterative Sparse Solver and Trust Region

sequential.lib Solver routine library supporting the ILP64 interface

lib\mkl_solver_lp64.lib Iterative Sparse Solver, Trust Region Solver, and GMP routine
library supporting the LP64 interface
lib\mkl_solver_lp64_ Sequential version of Iterative Sparse Solver, Trust Region Solver,
sequential.lib and GMP routine library supporting the LP64 interface

RTL
lib\libguide.lib Intel® Legacy OpenMP* run-time library for static linking

lib\libiomp5mt.lib Intel® Compatibility OpenMP* run-time library for static linking

lib\mkl_blacs_mpich2_ ILP64 version of BLACS routines supporting MPICH2

ilp64.lib
lib\mkl_blacs_mpich2_ LP64 version of BLACS routines supporting MPICH2
lp64.lib

3-19
3 Intel® Math Kernel Library User’s Guide

Table 3-7 Detailed Structure of the IA-64 Architecture-specific Directory ia64

(continued)
File Contents
Dynamic Libraries
Interface layer
lib\mkl_intel_ilp64_dll. ILP64 interface library for dynamic linking with the Intel
lib compilers

lib\mkl_intel_lp64_dll. LP64 interface library for dynamic linking with the Intel compilers
lib
Threading layer
bin\mkl_intel_thread.dll Dynamic threading library for the Intel compilers

bin\mkl_sequential.dll Dynamic sequential library

lib\mkl_intel_thread_dll. Threading library for dynamic linking with the Intel compilers
lib
lib\mkl_sequential_dll.lib Sequential library for dynamic linking
Computation layer
bin\libimalloc.dll Dynamic library to support renaming of
memory functions
bin\mkl_cdft_core.dll Cluster FFT dynamic library

bin\mkl_i2p.dll Kernel library for the IA-64 architecture

bin\mkl_lapack.dll LAPACK and DSS/PARDISO routines and drivers

bin\mkl_scalapack_ ScaLAPACK routine library supporting the ILP64 interface

ilp64.dll
bin\mkl_scalapack_lp64. ScaLAPACK routine library supporting the LP64 interface
dll
bin\mkl_vml_i2p.dll VML part of processor kernel for the IA-64 architecture

lib\mkl_cdft_core_dll. Cluster FFT library for dynamic linking

lib
lib\mkl_core_dll.lib Library dispatcher for dynamic load of processor-specific kernel
library
lib\mkl_scalapack_ilp64_ ScaLAPACK routines for dynamic linking supporting the ILP64
dll.lib interface

lib\mkl_scalapack_lp64_ ScaLAPACK routines for dynamic linking supporting the LP64

dll.lib interface

3-20
 Intel® Math Kernel Library Structure 3
Table 3-7 Detailed Structure of the IA-64 Architecture-specific Directory ia64
(continued)
File Contents
RTL
bin\libguide40.dll Intel® Legacy OpenMP* run-time dynamic library

bin\libiomp5md.dll Intel® Compatibility OpenMP* run-time dynamic library

bin\mkl_blacs_ilp64.dll ILP64 version of BLACS routines

bin\mkl_blacs_lp64.dll LP64 version of BLACS routines

bin\1033\mkl_msg.dll Catalog of Intel MKL messages in English

bin\1041\mkl_msg.dll Catalog of Intel MKL messages in Japanese

lib\libguide40.lib Intel® Legacy OpenMP* run-time library for dynamic linking

lib\libiomp5md.lib Intel® Compatibility OpenMP* run-time library for dynamic

linking
lib\mkl_blacs_ilp64_ ILP64 version of BLACS interface library for dynamic linking
dll.lib
lib\mkl_blacs_lp64_ LP64 version of BLACS interface library for dynamic linking
dll.lib

Contents of the Documentation Directory

Table 3-8 shows the contents of the doc subdirectory in the Intel MKL installation
directory:

Table 3-8 Contents of the doc Directory

File name Comment
Install.htm Intel MKL Installation Guide

mkl_documentation.htm Overview and links for the Intel MKL documentation

mklEULA.rtf Intel MKL end user license

mklman.chm HTML Help version of the Intel MKL Reference Manual

mklman.pdf Intel MKL Reference Manual

mklman90_j.pdf Intel MKL Reference Manual in Japanese

mklsupport.txt Information on package number for customer support reference

redist.txt List of redistributable files

Release_Notes.pdf Intel MKL Release Notes

userguide.pdf Intel MKL User’s Guide, this document.

3-21
Configuring Your
Development Environment 4
This chapter explains how to configure your development environment, namely, the Intel®
Visual Fortran compiler, Microsoft Visual C++* IDE, and a Microsoft Visual Studio* IDE
project, for the use with the Intel® Math Kernel Library (Intel® MKL).
Chapter 2 explains how to set environment variables path, lib, include, LIBRARY_PATH,
CPATH, and FPATH.
For information on how to set up environment variables for threading, see Setting the
Number of Threads Using OpenMP* Environment Variable.

Configuring the Microsoft Visual C/C++* Development

System to Link with Intel® MKL
To configure the Microsoft Visual C++*.NET 2003, Visual C++* 2005, or Visual C++* 2008
development system to link with Intel MKL, perform the following steps. Though some
menu items mentioned below may slightly differ for different versions of Visual C++, the
fundamental configuring steps are applicable to all these versions.
1. Select View > Solution Explorer (and make sure this window is active).
2. Select Tools > Options > Projects > VC++ Directories > Show directories for:
> Include Files. Add the directory for the Intel MKL include files, that is, <mkl
directory>\include".
3. Select Tools > Options > Projects > VC++ Directories > Show directories for:
> Library Files. Add the directory for the Intel MKL library files, for example, "<mkl
directory>\ia32\lib".
4. Select Tools > Options > Projects > VC++ Directories > Show directories for:
> Executable Files. Add the directory for the Intel MKL executable files, for example,
"<mkl directory>\ia32\bin".

4-1
4 Intel® Math Kernel Library User’s Guide

5. Select Project > Properties > Configuration Properties > Linker > Input >
Additional Dependencies. Add the libraries required (for example, mkl_intel_c.lib
mkl_intel_thread.lib mkl_core.lib libiomp5md.lib). To learn how to choose
the libraries, see Selecting Libraries to Link.

Configuring Intel® Visual Fortran to Link with Intel® MKL

To ensure that Intel® Visual Fortran (IVF) works correctly with Intel MKL, perform the
following steps:
1. Select Project > Properties > Linker > General > Additional Library
Directories. Add the architecture-specific lib folder,
for example, <mkl directory>\ia32\lib.
2. Select Project > Properties > Linker > Input > Additional Dependencies. Insert
mkl_intel_c.lib mkl_intel_thread.lib mkl_core.lib libiomp5md.lib.

Automatically Configuring a Microsoft Visual Studio*

IDE Project with Intel® MKL
Intel MKL enables you to automatically configure Microsoft Visual Studio* IDE projects with
the library. This feature supports Microsoft Visual Studio* IDE versions 2005 or higher. To
enable automatic configuring, select the Microsoft Visual Studio ... environment
integration add-in component during Intel MKL installation. The component is selected
during the default installation.
To automatically configure your C++ project with Intel MKL,
1. Open Solution Explorer
2. Select <project name> > Intel MKL Project Settings > Add Intel MKL
<version> (see Figure 4-1).
The message informs you that the configuring is complete.

4-2
 Configuring Your Development Environment 4
Figure 4-1 Automatically configuring a Microsoft Visual Studio* IDE project with Intel® MKL

The configuring updates the project settings with the Intel MKL libraries and paths needed
for programming in Microsoft Visual Studio* on your system. In particular:

4-3
4 Intel® Math Kernel Library User’s Guide

• The <project name> > Properties > Configuration Properties > C++ > General
> Additional Include Directories setting is appended with <mkl
directory>\include.
• The <project name> > Properties > Configuration Properties > Linker >
General > Additional Library Directories setting is appended with a path to Intel
MKL libraries, for example, <mkl directory>\ia32\lib for IA-32 architecture.
• The <project name> > Properties > Configuration Properties > Linker >
Input > Additional Dependencies setting is appended with a set of Intel MKL link
libraries required for your architecture, for example, mkl_intel_c.lib
mkl_intel_thread.lib mkl_core.lib libiomp5md.lib for IA-32 architecture.
• The <project name> > Properties > Configuration Properties > Debugging >
Environment setting contains the PATH environment variable appended with the path
to the bin directory depending on your architecture, for example, PATH=<mkl
directory>\ia32\bin\$(PATH) for IA-32 architecture.
To remove the project settings related to Intel MKL, select <project name> > Intel MKL
Project Settings > Remove Intel MKL.
A message informs you of removing Intel MKL-related settings.

Configuring a Project to Run an Intel® MKL Example in the

Microsoft Visual Studio* IDE 2008
This section provides hints for creating projects with the Intel MKL examples in Microsoft
Visual Studio* 2008. To run the Intel MKL C examples in Microsoft Visual Studio 2008, it is
recommended to have the Intel® C/C++ Compiler installed and integrated into Visual
Studio. It is also possible to run C examples using Microsoft Visual C++* 2008 Compiler
integrated into Visual Studio*. To run the Intel MKL Fortran examples in Microsoft Visual
Studio 2008, you must have the Intel® Visual Fortran Compiler installed and integrated
into Visual Studio. Integration of the compiler into Visual Studio is performed automatically
during the default installation, provided that Visual Studio is installed on your system. If
not, request Visual Studio integration when prompted.
The instructions provided assume standard configuration of mouse buttons. If you are
using reversed mouse buttons, properly adjust the instructions, for example, read
"right-click" as "left-click". For Intel MKL examples where the instructions do not work, see
Known Limitations.

4-4
 Configuring Your Development Environment 4
Creating and Configuring the Intel® C/C++ and/or Microsoft
Visual C++* 2008 Project
This section demonstrates how to create a Visual C/C++ project using an Intel MKL
example in Microsoft Visual Studio 2008.

NOTE. The instructions below creates a Win32/Debug project running

one Intel MKL example in a Console window. For details on creation of
different kinds of Microsoft Visual Studio projects, refer to MSDN Visual
Studio documentation at http://www. microsoft. com.

To create and configure a Win32/Debug project running an Intel MKL C example with the
Intel C/C++ Compiler integrated into Visual Studio and/or Microsoft Visual C++* 2008,
perform the following steps:
1. Creating a C Project:
a. Open Visual Studio 2008.
b. On the main menu, select File > New > Project, which opens the New Project
window.
c. In the New Project window, select Visual C++ > Win32 in Project Types:,
then select Win32 Console Application in Templates:. Type <project name>
in the Name: field, for example, MKL_CBLAS_CAXPYIX, and click OK. This closes
the New Project window and opens the Win32 Application Wizard - <project
name> window.
d. In the Win32 Application Wizard - <project name> window, select Next,
then select Application Settings, check Empty project in Additional options:,
and click Finish, which closes the Win32 Application Wizard - <project
name> window.
The next steps are performed inside the Solution Explorer window. To open it, select
View > Solution Explorer from the main menu.
2. (optional) To switch to the Intel C/C++ project, right-click <project name> and
select Convert to use Intel® C++ Project System from the drop-down menu. (The
menu item is available if the Intel® C/C++ Compiler is integrated into Visual Studio.).
3. Adding sources of the Intel MKL example to the project:

4-5
4 Intel® Math Kernel Library User’s Guide

a. Right-click the Source Files folder under <project name> and select Add >
Existing Item… from the drop-down menu, which opens the Add Existing Item
- <project name> window.
b. In the Add Existing Item - <project name> window, go to the Intel MKL
examples directory, that is, "<mkl directory>\examples". Next go to the
subdirectory that contains the example you want to run, for instance,
\cblas\source. Select the example file and supporting files with extension ".c"
(C sources), for example, select files cblas_caxpyix.c and common_func.c. For
the list of supporting files in each example directory, see Support Files for Intel
MKL Examples. Click Add, which closes the Add Existing Item - <project
name> window and causes selected files to appear in the Source Files folder in
Solution Explorer.
The next steps adjust the properties of the project.
4. Select <project name>.
5. On the main menu, select Project > Properties, which opens the <project name>
Property Pages window.
6. Setting Intel MKL Include dependencies:
a. In the <project name> Property Pages window, select Configuration
Properties > C/C++ > General. In the right-hand part of the window, select
Additional Include Directories > … (the button with dots …), which opens the
Additional Include Directories window.
b. In the Additional Include Directories window, click the New Line button (the
first button in the uppermost row). When the new line appears in the window, click
the button with dots, which opens the Select Directory window.
c. In the Select Directory window, navigate to the Intel MKL include directory, that
is, <mkl_directory>\include". When the name "include" appears in the
Folder: field, click OK, which closes the Select Directory window and causes the
full path to the Intel MKL include directory to appear in the Additional Include
Directories window.
d. In the Additional Include Directories window, click OK, which closes the
Additional Include Directories window.
7. Setting library dependencies:
a. In the <project name> Property Pages window, select Configuration
Properties > Linker > General. In the right-hand part of the window, select
Additional Library Directories > … (the button with dots …), which opens the
Additional Library Directories window.
b. In the Additional Library Directories window, click the New Line button (the
first button in the uppermost row). When the new line appears in the window, click
the button with dots, which opens the Select Directory window.

4-6
 Configuring Your Development Environment 4
c. In the Select Directory window, go to the directory with the Intel MKL libraries,
that is, "<mkl_directory>\<architecture>\lib", where <architecture> is
one of {ia32, em64t, ia64}, which indicates the architecture of your computer,
for example: "<mkl_directory>\ia32\lib". Usually, for laptop and desktop
computers, <architecture> is ia32. When the name "lib" appears in the
Folder: field, click OK, which closes the Select Directory window and causes the
full path to the Intel MKL libraries to appear in the Additional Library
Directories window.
d. In the Additional Library Directories window, click OK, which closes the
Additional Library Directories window.
e. In the <project name> Property Pages window, select Configuration
Properties > Linker > Input. In the right-hand part of the window, select
Additional Dependencies > … (the button with dots …), which opens the
Additional Dependencies window.
f. In the Additional Dependencies window, type the libraries required, for
example, for <architecture>=ia32, type mkl_intel_c.lib
mkl_intel_thread.lib mkl_core.lib libiomp5md.lib. For more details, see
Selecting Libraries to Link.
g. In the Additional Dependencies window, click OK, which closes the Additional
Dependencies window.
If the Intel MKL example directory does not contain a data directory, skip the next step.
8. Setting data dependencies for the Intel MKL example:
a. In the '<project name> Property Pages window, select Configuration
Properties > Debugging. In the right-hand part of the window, select
Command Arguments >  > <Edit…>, which opens the Command
Arguments window.
b. In the Command Arguments window, type the path to the proper data file in
quotes. The name of the data file is the same as the name of the example file, but
the extension is different, for example,
"<mkl_directory>\examples\cblas\data\cblas_caxpyix.d".
c. In the Command Arguments window, click OK, which closes the Command
Arguments window.
9. In the <project name> Property Pages window, click OK, which closes the
<project name> Property Pages window.
10. Certain examples do not pause before the end of execution. To see the results printed
in the Console window, set a breakpoint at the very last 'return 0;' statement or add
a statement like a call to 'getchar();' before the last 'return 0;' statement.
11. To build the solution, select Build > Build Solution.

4-7
4 Intel® Math Kernel Library User’s Guide

NOTE. You may see warnings about unsafe functions and variables. To
get rid of these warnings, go to Project > Properties, and when the
<project name> Property Pages window opens, go to Configuration
Properties > C/C++ > Preprocessor. In the right-hand part of the
window, select Preprocessor Definitions, add
_CRT_SECURE_NO_WARNINGS, and click OK.

12. To run the example, select Debug > Start Debugging, which opens the Console
window.
13. You can see the results of the example in the Console window. If you used the
'getchar();' statement to pause execution of the program, press Enter to complete
the run. If you used a breakpoint to pause execution of the program, select Debug >
Continue, which closes the Console window.

Creating and Configuring the Intel® Visual Fortran Project

This section demonstrates how to create an Intel Visual Fortran project running an Intel
MKL example in Microsoft Visual Studio 2008.

NOTE. The instructions below create a Win32/Debug project running one

Intel MKL example in a Console window. For details on creation of
different kinds of Microsoft Visual Studio projects, refer to MSDN Visual
Studio documentation at http://www. microsoft. com.

To create and configure a Win32/Debug project running an Intel MKL Fortran example with
the Intel Visual Fortran Compiler integrated into Visual Studio, perform the following steps:
1. Creating a Visual Fortran Project:
a. Open Visual Studio 2008.
b. On the main menu, select File > New > Project, which opens the New Project
window.

4-8
 Configuring Your Development Environment 4
c. In the New Project window, select Intel® Fortran > Console Application in
Project Types:, then select Empty Project in Templates:. When done, type
<project name> in Name:, for example,
MKL_PDETTF_D_TRIG_TRANSFORM_BVP, and click OK, which closes the New
Project window.
The next steps are performed inside the Solution Explorer window. To open it, select View
> Solution Explorer from the main menu.
2. Adding sources of Intel MKL example to the project:
a. Right-click the Source Files folder under <project name> and select Add >
Existing Item… from the drop-down menu, which opens the Add Existing Item
- <project name> window.
b. In the Add Existing Item - <project name> window, go to the Intel MKL
examples directory, that is, "<mkl_directory>\examples". Next go to the
subdirectory that contains an example you want to run, for instance,
\pdettf\source. Select the example file and supporting files with extension ".f"
or ".f90" (Fortran sources). For example, select the d_trig_tforms_bvp.f90
file. For the list of supporting files in each example directory, see Support Files for
Intel MKL Examples. Click Add, which closes the Add Existing Item - <project
name> window and causes the selected files to appear in the Source Files folder
in Solution Explorer.
Certain examples with the "use" statements require the next two steps.
c. Right-click the Header Files folder under <project name> and select Add >
Existing Item… from the drop-down menu, which opens the Add Existing Item
- <project name> window.
d. In the Add Existing Item - <project name> window, go to the Intel MKL
include directory, that is, <mkl_directory>\include". Select the header files
that appear in the "use" statements. For example, select the mkl_dfti.f90 and
mkl_trig_transforms.f90 files. Click Add, which closes the Add Existing
Item - <project name> window and causes the selected files to appear in the
Header Files folder in Solution Explorer.
The next steps adjust the properties of the project:
3. Select the <project name>.
4. On the main menu, select Project > Properties, which opens the <project name>
Property Pages window.
5. Setting the Intel MKL Include dependencies:

4-9
4 Intel® Math Kernel Library User’s Guide

a. In the <project name> Property Pages window, select Configuration

Properties > Fortran > General. In the right-hand part of the window, select
Additional Include Directories >  > <Edit…>, which opens the Additional
Include Directories window.
b. In the Additional Include Directories window, type the Intel MKL include
directory in quotes, that is, is "<mkl_directory>\include". Click OK, which
closes the Additional Include Directories window.
6. In the <project name> Property Pages window, select Configuration Properties
> Fortran > Preprocessor. In the right-hand part of the window, select Preprocess
Source File > Yes (default is No). This step is recommended because certain
examples require preprocessing.
7. Setting library dependencies:
a. In the <project name> Property Pages window, select Configuration
Properties > Linker > General. In the right-hand part of the window, select
Additional Library Directories >  > <Edit…>, which opens the Additional
Library Directories window.
b. In the Additional Library Directories window, type the directory with the Intel
MKL libraries in quotes, that is, "<mkl_directory>\<architecture>\lib",
where <architecture> is one of {ia32, em64t, ia64}, which indicates the
architecture of your computer, for example: "<mkl_directory>\ia32\lib".
Usually, for laptop and desktop computers <architecture> is ia32. Click OK,
which closes the Additional Library Directories window.
c. In the <project name> Property Pages window, select Configuration
Properties > Linker > Input. In the right-hand part of the window, select
Additional Dependencies and type the libraries required, for example, for
<architecture>=ia32, type mkl_intel_c.lib mkl_intel_thread.lib
mkl_core.lib libiomp5md.lib.
8. In the <project name> Property Pages window, click OK, which closes the
Property Pages window.
9. Certain examples do not pause before the end of execution. To see the results printed
in the Console window, set a breakpoint at the very end of the program or add the
'pause' statement before the last 'end' statement.
10. To build the solution, select Build > Build Solution.
11. To run the example, select Debug > Start Debugging, which opens the Console
window.
12. You can see the results of the example in the Console window. If you used 'pause'
statement to pause execution of the program, press Enter to complete the run. If you
used a breakpoint to pause execution of the program, select Debug > Continue,
which closes the Console window.

4-10
 Configuring Your Development Environment 4
Support Files for Intel MKL Examples
Below is the list of support files that have to be added to the project for respective
examples:
examples\cblas\source: common_func.c
examples\dftc\source: dfti_example_status_print.c dfti_example_support.c

Known Limitations
It is not possible to create a Visual Studio* project using the above instructions for
examples from the following directories:
examples\blas
examples\blas95
examples\cdftc
examples\cdftf
examples\dftf
examples\fftw2x_cdft
examples\fftw2xc
examples\fftw2xf
examples\fftw3xc
examples\fftw3xf
examples\java
examples\lapack
examples\lapack95

Configuring the Out-of-Core (OOC) DSS/PARDISO* Solver

When using the configuration file for the OOC DSS/PARDISO* Solver, be aware that the
maximum length of the path lines in the file is 1000 characters.
For more information, see the "Sparse Solver Routines" chapter in the Intel MKL Reference
Manual.

4-11
Linking Your Application
with the Intel® Math
Kernel Library 5
This chapter discusses linking your applications with the Intel® Math Kernel Library
(Intel® MKL) for the Windows* OS. The chapter provides information on the libraries that
should be linked with your application, presents linking examples, and explains building of
custom DLL.
To link with Intel MKL, which employs the layered linking model, choose one library from
the Interface layer, one library from the Threading layer, the library from the
Computational layer, and, if necessary, add run-time libraries. Table 5-1 lists typical sets of
libraries that suffice to link with Intel MKL.

Table 5-1 Typical Link Libraries

Computational
Interface layer Threading layer layer RTL
IA-32 mkl_intel_c.lib mkl_intel_ mkl_core.lib libiomp5md.lib
architecture, thread.lib
static linking
IA-32 mkl_intel_c_ mkl_intel_ mkl_core_dll. libiomp5md.lib
architecture, dll.lib thread_dll.lib lib
dynamic linking
Intel® 64 and mkl_intel_ mkl_intel_ mkl_core.lib libiomp5md.lib
IA-64 lp64.lib thread_lp64.lib
architectures,
static linking
Intel® 64 and mkl_intel_ mkl_intel_ mkl_core_dll. libiomp5md.lib
IA-64 lp64_dll.lib thread_lp64_ lib
architectures,
dll.lib
dynamic linking

For exceptions and alternatives to the libraries listed above, see Selecting Libraries to Link.

5-1
5 Intel® Math Kernel Library User’s Guide

Selecting Libraries to Link

This section recommends which libraries to link depending on your Intel MKL usage
scenario and provides details of the linking in subsections:
Linking with Fortran 95 Interface Libraries
Linking with Threading Libraries
Linking with Computational Libraries
Linking with Compiler Support RTLs
Linking with System Libraries
Linking Examples

Linking with Fortran 95 Interface Libraries

The mkl_blas95.lib, mkl_blas95_lp64.lib, mkl_blas95_ilp64.lib,
mkl_lapack95.lib, mkl_lapack95_lp64.lib, and mkl_lapack95_ilp64.lib libraries
contain Fortran 95 interfaces for BLAS and LAPACK, respectively, which are
compiler-dependent. In the Intel MKL package, they are prebuilt for the Intel® Fortran
compiler. If you are using a different compiler, build these libraries before using the
interface. (See Fortran 95 Interfaces to LAPACK and BLAS and Compiler-dependent
Functions and Fortran 90 Modules for details on building the libraries and on why source
code is distributed in this case.)

Linking with Threading Libraries

Several compilers that Intel MKL supports use the OpenMP* threading technology. Starting
with version 10.0, Intel MKL supports the implementations of OpenMP* that those
compilers provide. If an application using OpenMP* and compiled with such a compiler calls
Intel MKL 10.0 or lower, which is threaded with Intel® compilers, performance issues and
even failures may arise because threading libraries from different compilers are not
compatible. Starting with Intel MKL 10.0, solutions for these issues are provided from the
Threading Layer and the Compiler Support Run-time Libraries (RTL).

Threading Layer. Because of the internal structure of the library, threading represents a
small amount of code. This code is compiled by different compilers (Intel and PGI*
compilers on Windows*), and you should link in the appropriate threading library.

RTL. This layer includes run-time libraries of the Intel compiler: the Intel® Compatibility
OpenMP* run-time library libiomp and Intel® Legacy OpenMP* run-time library
libguide. The Compatibility library libiomp is an extension of libguide that provides
support for one additional threading compiler on Windows (Microsoft Visual C++*). That is,

5-2
 Linking Your Application with the Intel® Math Kernel Library 5
a program threaded with Microsoft Visual C++ can safely be linked with Intel MKL and
libiomp and execute efficiently and effectively. So, you are encouraged to use libiomp
rather than libguide.
Table 5-2 shows different scenarios, depending on the threading compiler used, and the
possibilities for each scenario to choose the threading libraries and RTL when using Intel
MKL (static cases only):

Table 5-2 Selecting Threading Libraries

Application RTL
Compiler Threaded? Threading Layer Recommended Comment
Intel Does not mkl_intel_thread.lib libiomp5md.lib
matter
PGI Yes mkl_pgi_thread.lib or PGI* supplied Use of mkl_
mkl_sequential.lib sequential.lib
removes threading
from Intel MKL calls.
PGI No mkl_intel_thread.lib libiomp5md.lib
PGI No mkl_pgi_thread.lib PGI* supplied
PGI No mkl_sequential.lib None
Microsoft Yes mkl_intel_thread.lib libiomp5md.lib If the application is
threaded with the
OpenMP* library of
the Microsoft Visual
Studio* IDE version
2005 or later.
Microsoft Yes mkl_sequential.lib None If the application is
threaded using
Win32 threading.
Microsoft No mkl_intel_thread.lib libiomp5md.lib
other Yes mkl_sequential.lib None
other No mkl_intel_thread.lib libiomp5md.lib

NOTE. To use threaded Intel MKL, it is highly recommended to compile

your code with the /MT option. The compiler driver will pass the option to
the linker and the latter will load multi-thread (MT) run-time libraries.

5-3
5 Intel® Math Kernel Library User’s Guide

Linking with Computational Libraries

Typically, with the layered linking model, you must link your application with only one
computational library. However, certain Intel MKL function domains require several
computational link libraries.
For each Intel MKL function domain, Table 5-3 lists computational libraries that you must
include in the link line. For more information on linking with ScaLAPACK and Cluster FFTs,
see also Linking with ScaLAPACK and Cluster FFTs.

Table 5-3 Computational Libraries to Link, by Function Domain

Function IA-32 Architecture Intel® 64 or IA-64 Architecture

domain Static Dynamic Static Dynamic
BLAS, mkl_core.lib mkl_core. mkl_core.lib mkl_core.
CBLAS, _dll.lib _dll.lib
Sparse BLAS,
LAPACK,
VML, VSL,
FFT,
Trigonometric
Transform
Functions,
Poisson
Library
Iterative mkl_solver.lib n/a See below n/a
Sparse or
Solvers, mkl_solver_
Trust Region sequential.lib
Solver, and
GMP routines mkl_core.lib
Iterative n/a n/a mkl_solver_ n/a
Sparse lp64.lib
Solvers, or
Trust Region mkl_solver_
Solver, and lp64_
GMP sequential.lib
routines,
LP64 mkl_core.lib
interface

5-4
 Linking Your Application with the Intel® Math Kernel Library 5
Table 5-3 Computational Libraries to Link, by Function Domain (continued)
IA-32 Architecture Intel® 64 or IA-64 Architecture
Function
domain Static Dynamic Static Dynamic
Iterative n/a n/a mkl_solver_ n/a
Sparse ilp64.lib
Solvers, or
Trust Region mkl_solver_
Solver, and ilp64_
GMP sequential.lib
routines,
ILP64 mkl_core.lib
interface
Direct Sparse mkl_core.lib mkl_core. mkl_core.lib mkl_core.
Solver/ _dll.lib _dll.lib
PARDISO*
Solver
ScaLAPACK1 mkl_scalapack mkl_scalapack See below See below
_core.lib _core_dll.lib
mkl_core.lib mkl_core_
dll.lib
ScaLAPACK, n/a n/a mkl_scalapack mkl_scalapack
LP64 _lp64.lib _lp64_dll.lib
interface1
mkl_core. mkl_core_
lib dll.lib
ScaLAPACK, n/a n/a mkl_scalapack mkl_scalapack_
ILP64 _ilp64.lib ilp64_dll.lib
interface1
mkl_core.lib mkl_core_
dll.lib
Cluster mkl_cdft_ mkl_cdft_ mkl_cdft_ mkl_cdft_
Fourier core.lib core_dll.lib core.lib core_dll.lib
Transform
Functions1
mkl_core.lib mkl_core_ mkl_core. mkl_core_
dll.lib lib dll.lib
1. Add also the library with BLACS routines corresponding to the used MPI. For details, see Linking with ScaLAPACK and Cluster FFTs.

See also:
Linking with Compiler Support RTLs.

5-5
5 Intel® Math Kernel Library User’s Guide

Linking with Compiler Support RTLs

You are strongly encouraged to dynamically link in the Intel Compatibility OpenMP*
run-time library libiomp or Intel Legacy OpenMP* run-time library libguide. Linking to
static OpenMP* run-time library is not recommended because it is very easy with complex
software to link in more than one copy of the library. This causes performance problems
(too many threads) and may cause correctness problems if more than one copy is
initialized.
You are advised to link with libiomp and libguide dynamically even if other libraries are
linked statically.
However, if you link with libiomp/libguide statically, the version of libiomp/libguide
you link with depends on which compiler you use:
• If you use the Intel compiler, link in the libiomp/libguide version that comes with
the compiler, that is, use the -openmp option.
• If you do not use the Intel compiler, link in the libiomp/libguide version that
comes with Intel MKL.
If you link with dynamic versions of libiomp/libguide (recommended), make sure PATH
is defined correctly. See Set Environment Variables for details.

Linking with System Libraries

If your system is based on the Intel® 64 or IA-64 architecture, note that new Microsoft
SDKs starting with 1289 include an additional library bufferoverflowu.lib to resolve
external references _security_cookie. Makefiles for examples and tests contain this
library with "BUF_LIB=bufferoverflowu.lib" macro. For older SDKs, leave this macro
empty "BUF_LIB=" or remove it from the linkage string.

Linking Examples
The section provides specific linking examples that use Intel® compilers on systems based
on the IA-32, Intel® 64, and IA-64 architectures. See also examples on linking with
ScaLAPACK and Cluster FFT in Chapter 9.
The following examples use the .f Fortran source file. C/C++ users should instead specify
a .cpp (C++) or .c (C) file and change the ifort linker to icc.
For assistance in finding the right link line, use the Web-based linking advisor available
from http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor.

5-6
 Linking Your Application with the Intel® Math Kernel Library 5
Linking on Systems Based on the IA-32 Architecture
1. Static linking of myprog.f and parallel Intel MKL supporting cdecl interface:
ifort myprog.f mkl_intel_c.lib mkl_intel_thread.lib mkl_core.lib
libiomp5md.lib
2. Dynamic linking of myprog.f and parallel Intel MKL supporting cdecl interface:
ifort myprog.f mkl_intel_c_dll.lib mkl_intel_thread_dll.lib
mkl_core_dll.lib libiomp5md.lib
3. Static linking of myprog.f and sequential version of Intel MKL supporting cdecl
interface:
ifort myprog.f mkl_intel_c.lib mkl_sequential.lib mkl_core.lib
4. Dynamic linking of myprog.f and sequential version of Intel MKL supporting cdecl
interface:
ifort myprog.f mkl_intel_c_dll.lib mkl_sequential_dll.lib
mkl_core_dll.lib
5. Static linking of user code myprog.f and parallel Intel MKL supporting stdcall
interface:
ifort myprog.f mkl_intel_s.lib mkl_intel_thread.lib mkl_core.lib
libiomp5md.lib
6. Dynamic linking of user code myprog.f and parallel Intel MKL supporting stdcall
interface:
ifort myprog.f mkl_intel_s_dll.lib mkl_intel_thread_dll.lib
mkl_core_dll.lib libiomp5md.lib
7. Static linking of myprog.f, Fortran 95 LAPACK interface1, and parallel Intel MKL
supporting cdecl interface:
ifort myprog.f mkl_lapack95.lib mkl_intel_c.lib mkl_intel_thread.lib
mkl_core.lib libiomp5md.lib
8. Static linking of myprog.f, Fortran 95 BLAS interface1, and parallel Intel MKL
supporting cdecl interface:
ifort myprog.f mkl_blas95.lib mkl_intel_c.lib mkl_intel_thread.lib
mkl_core.lib libiomp5md.lib
9. Static linking of myprog.f, parallel version of an iterative sparse solver, and parallel
Intel MKL supporting cdecl interface:
ifort myprog.f mkl_solver.lib mkl_intel_c.lib mkl_intel_thread.lib
mkl_core.lib libiomp5md.lib

1. See Fortran 95 Interfaces to LAPACK and BLAS for information on how to build Fortran 95 LAPACK and
BLAS interface libraries.

5-7
5 Intel® Math Kernel Library User’s Guide

10. Static linking of myprog.f, sequential version of an iterative sparse solver, and
sequential Intel MKL supporting cdecl interface:
ifort myprog.f mkl_solver_sequential.lib mkl_intel_c.lib
mkl_sequential.lib mkl_core.lib

Linking on Systems Based on the Intel® 64 and IA-64 Architectures

1. Static linking of myprog.f and parallel Intel MKL supporting LP64 interface:
ifort myprog.f mkl_intel_lp64.lib mkl_intel_thread.lib mkl_core.lib
libiomp5md.lib
2. Dynamic linking of myprog.f and parallel Intel MKL supporting LP64 interface:
ifort myprog.f mkl_intel_lp64_dll.lib mkl_intel_thread_dll.lib
mkl_core_dll.lib libiomp5md.lib
3. Static linking of myprog.f and sequential version of Intel MKL supporting LP64
interface:
ifort myprog.f mkl_intel_lp64.lib mkl_sequential.lib mkl_core.lib
4. Dynamic linking of myprog.f and sequential version of Intel MKL supporting LP64
interface:
ifort myprog.f mkl_intel_lp64_dll.lib mkl_sequential_dll.lib
mkl_core_dll.lib
5. Static linking of myprog.f and parallel Intel MKL supporting ILP64 interface:
ifort myprog.f mkl_intel_ilp64.lib mkl_intel_thread.lib mkl_core.lib
libiomp5md.lib
6. Dynamic linking of myprog.f and parallel Intel MKL supporting ILP64 interface:
ifort myprog.f mkl_intel_ilp64_dll.lib mkl_intel_thread_dll.lib
mkl_core_dll.lib libiomp5md.lib
7. Static linking of myprog.f, Fortran 95 LAPACK interface1, and parallel Intel MKL
supporting LP64 interface:
ifort myprog.f mkl_lapack95_lp64.lib mkl_intel_lp64.lib
mkl_intel_thread.lib mkl_core.lib libiomp5md.lib
8. Static linking of myprog.f, Fortran 95 BLAS interface1, and parallel Intel MKL
supporting LP64 interface:
ifort myprog.f mkl_blas95_lp64.lib mkl_intel_lp64.lib
mkl_intel_thread.lib mkl_core.lib libiomp5md.lib
9. Static linking of myprog.f, parallel version of an iterative sparse solver, and parallel
Intel MKL supporting LP64 interface:

1. See Fortran 95 Interfaces to LAPACK and BLAS for information on how to build Fortran 95 LAPACK and
BLAS interface libraries.

5-8
 Linking Your Application with the Intel® Math Kernel Library 5
ifort myprog.f mkl_solver_lp64.lib mkl_intel_lp64.lib
mkl_intel_thread.lib mkl_core.lib libiomp5md.lib
10. Static linking of myprog.f, sequential version of an iterative sparse solver, and
sequential Intel MKL supporting LP64 interface:
ifort myprog.f mkl_solver_lp64_sequential.lib
mkl_intel_lp64.libmkl_sequential.lib mkl_core.lib

Building a Custom DLL

Custom DLL enable reducing the collection of functions available in Intel MKL libraries to
those required to solve your particular problems, which helps to save disk space and build
your own dynamic libraries for distribution.

Intel MKL Custom DLL Builder

Custom DLL builder enables creation of a dynamic library containing the selected functions
and located in the tools/builder directory. The builder contains a makefile and a
definition file with the list of functions. The makefile has three targets: ia32, ipf, and
em64t. Use the ia32 target for processors the use the IA-32 architecture; use ipf for
processors that use the IA-64 architecture; and use em64t for processors that use the
Intel® 64 architecture.

Specifying Makefile Parameters

There are several macros (parameters) for the makefile:
interface=cdecl/stdcall
Defines the interface only for the IA-32 architecture. The default value is cdecl.
export=functions_list
Specifies the full name of the file that contains the list of entry-point functions to be
included in the dll. The default name is functions_list (no extension).
name=mkl_custom
Specifies the name of the dll and interface library to be created. By default, the names
of the created libraries are mkl_custom.dll and mkl_custom.lib.
xerbla=user_xerbla.obj
Specifies the name of the object file that contains the user's error handler. This error
handler will be added to the library and used instead of the default Intel MKL error
handler xerbla. If you omit this parameter, the native Intel MKL xerbla is used. See
the description of the xerbla function in the Intel MKL Reference Manual on how to

5-9
5 Intel® Math Kernel Library User’s Guide

develop your own error handler. Please note that for the IA-32 architecture, the object
file should be in the appropriate interface (cdecl or stdcall) according to the interface
macro.
MKLROOT=<MKL_directory>
Specifies the location of Intel MKL libraries used to build the custom DLL. The
installation directory for the current Intel MKL release is used by default.
manifest
Manages the creation of a Microsoft manifest for the custom DLL:
if manifest=yes, the manifest file with the name defined by the name parameter
above and the "manifest" extension will be created,
if manifest=no, the manifest file will not be created,
if manifest=embed, the manifest will be embedded into the DLL,
if manifest is not specified (default), no special action regarding the manifest will
be taken.

CAUTION. Because manifest files were introduced in Microsoft Visual

Studio* 2005, do not specify the manifest parameter if you are using
Microsoft Visual Studio* 2003.

All parameters are optional.

In the simplest case, the command line is nmake ia32, and the missing parameters have
default values. This command creates the mkl_custom.dll and mkl_custom.lib
libraries with the cdecl interface for processors using the IA-32 architecture. The command
takes the list of functions from the functions_list file and uses the native Intel MKL
error handler xerbla.
An example of a more complex case follows:
nmake ia32 interface=stdcall export=my_func_list.txt name=mkl_small
xerbla=my_xerbla.obj
In this case, the command creates the mkl_small.dll and mkl_small.lib libraries with
the stdcall interface for processors using the IA-32 architecture. The command takes the
list of functions from my_func_list.txt file and uses the user's error handler
my_xerbla.obj.
The process is similar for processors using the Intel® 64 or IA-64 architecture.

5-10
 Linking Your Application with the Intel® Math Kernel Library 5
Specifying a List of Functions
Adjust entry points in the functions_list file to the required interface. For example, you
can list the cdecl entry points as follows:
DGEMM
DTRSM
DDOT
DGETRF
DGETRS
cblas_dgemm
cblas_ddot
Examples of entry points for the stdcall interface follow:
_DGEMM@60
_DDOT@20
_DGETRF@24

5-11
Managing Performance and
Memory 6
This chapter features different ways to obtain the best performance with the Intel® Math
Kernel Library (Intel® MKL): primarily, it discusses threading (see Using the Intel® MKL
Parallelism), then shows coding techniques and gives hardware configuration tips for
improving performance. The chapter also discusses the Intel MKL memory management
and shows how to redefine memory functions used by the library.

Using the Intel® MKL Parallelism

Intel MKL is threaded in a number of places:
• Direct sparse solver.
• LAPACK
— Linear equations, computational routines:
- factorization: *getrf, *gbtrf, *potrf, *pptrf, *sytrf, *hetrf, *sptrf, *hptrf
- solving: *gbtrs, *gttrs, *pptrs, *pbtrs, *pttrs, *sytrs, *sptrs, *hptrs,
*tptrs, *tbtrs.
— Orthogonal factorization, computational routines:
*geqrf, *ormqr, *unmqr, *ormlq, *unmlq, *ormql, *unmql, *ormrq, *unmrq.
— Singular Value Decomposition, computational routines: *gebrd, *bdsqr.
— Symmetric Eigenvalue Problems, computational routines:
*sytrd, *hetrd, *sptrd, *hptrd, *steqr, *stedc.
Note that a number of other LAPACK routines, which are based on threaded LAPACK or
BLAS routines, make effective use of parallelism: *gesv, *posv, *gels, *gesvd,
*syev, *heev, etc.

6-1
6 Intel® Math Kernel Library User’s Guide

• Level1 and Level2 BLAS functions:

— Level1 BLAS: *axpy, *copy, *swap, ddot/sdot, drot/srot
— Level2 BLAS: *gemv, *trmv, dsyr/ssyr, dsyr2/ssyr2, dsymv/ssymv
Note that these functions are threaded only for:
— Intel® 64 architecture
— Intel® Core™2 Duo and Intel® Core™ i7 processors
• All Level 3 BLAS and all Sparse BLAS routines except Level 2 triangular solvers.
• VML.
• FFT.
Because it is designed for multi-threaded programming, Intel MKL is thread-safe, which
means that all Intel MKL functions1 work correctly during simultaneous execution by
multiple threads. In particular, any chunk of threaded Intel MKL code provides access for
multiple threads to the same shared data, while permitting only one thread at any given
time to access a shared piece of data. Due to thread-safety, you can call Intel MKL from
multiple threads and not worry about the function instances interfering with each other.
The library uses OpenMP* threading software, so you can use the environmental variable
OMP_NUM_THREADS to specify the number of threads to use. There are different means to
set the number of threads. In Intel MKL releases earlier than 10.0, you could use the
environment variable OMP_NUM_THREADS (see Setting the Number of Threads Using
OpenMP* Environment Variable for details) or the equivalent OpenMP run-time function
calls (detailed in section Changing the Number of Threads at Run Time). Starting with
version 10.0, Intel MKL also offers variables that are independent of OpenMP, such as
MKL_NUM_THREADS, and equivalent Intel MKL functions for thread management (see Using
Additional Threading Control for details). The Intel MKL variables are always inspected first,
then the OpenMP variables are examined, and if neither are used, the OpenMP software
chooses the default number of threads. This is a change from the Intel MKL versions 9.x or
lower.

NOTE. Starting with Intel MKL 10.0, the OpenMP* software determines
the default number of threads. The default number of threads is equal to
the number of logical processors in your system for Intel OpenMP*
libraries.

To achieve higher performance, set the number of threads to the number of real processors
or physical cores, as summarized in Techniques to Set the Number of Threads.

1. Except LAPACK deprecated routines ?lacon, ?lasq3, and ?lasq4.

6-2
 Managing Performance and Memory 6
See also:
Managing Multi-core Performance.

Techniques to Set the Number of Threads

You can employ different techniques to change the number of threads to use in Intel MKL:
• Set one of the OpenMP or Intel MKL environment variables:
— OMP_NUM_THREADS
— MKL_NUM_THREADS
— MKL_DOMAIN_NUM_THREADS
• Call one of the OpenMP or Intel MKL functions:
— omp_set_num_threads()
— mkl_set_num_threads()
— mkl_domain_set_num_threads()
When choosing the appropriate technique, take into account the following rules:
• The Intel MKL threading controls take precedence over the OpenMP techniques.
• A function call takes precedence over any environment variables. The exception, which
is a consequence of the previous rule, is the OpenMP subroutine
omp_set_num_threads(), which does not have precedence over Intel MKL
environment variables, such as MKL_NUM_THREADS.
• The environment variables cannot be used to change run-time behavior in the course
of the run, because they are read only once at the first call to Intel MKL.

Avoiding Conflicts in the Execution Environment

Certain situations can cause conflicts in the execution environment that make the use of
threads in Intel MKL problematic. First, this section briefly discusses why the problems
exist.
If you thread the program using OpenMP directives and compile the program with Intel®
compilers, Intel MKL and the program will both use the same threading library. Intel MKL
tries to determine if it is in a parallel region in the program, and if it is, it does not spread
its operations over multiple threads unless the user specifically requests Intel MKL to do so
via the MKL_DYNAMIC functionality (see Using Additional Threading Control for details).
However, Intel MKL can be aware that it is in a parallel region only if the threaded program

6-3
6 Intel® Math Kernel Library User’s Guide

and Intel MKL are using the same threading library. If the user’s program is threaded by
some other means, Intel MKL may operate in multithreaded mode and the performance
may suffer due to overuse of the resources.
Here are several cases with recommendations depending on the threading model you
employ:

Table 6-1 How to Avoid Conflicts in the Execution Environment for Your Threading
Model
Threading model Discussion
You thread the program using OS threads (Win32* If more than one thread calls Intel MKL, and the
threads on the Windows* OS). function being called is threaded, it may be
important that you turn off Intel MKL threading. Set
the number of threads to one by any of the available
means (see Techniques to Set the Number of
Threads).
You thread the program using OpenMP directives This is more problematic because setting of the
and/or pragmas and compile the program using a OMP_NUM_THREADS environment variable affects
compiler other than a compiler from Intel. both the compiler's threading library and libiomp
(libguide). In this case, choose the threading
library that matches the layered Intel MKL with the
OpenMP compiler you employ (see Linking Examples
on how to do this). If this is not possible, use Intel
MKL in the sequential mode. To do this, you should
link with the appropriate threading library:
mkl_sequential.lib or
mkl_sequential.dll (see High-level Directory
Structure).
There are multiple programs running on a The threading software will see multiple processors
multiple-cpu system, for example, a parallelized on the system even though each processor has a
program that runs using MPI for communication in separate MPI process running on it. In this case, one
which each processor is treated as a node. of the solutions is to set the number of threads to
one by any of the available means (see Techniques to
Set the Number of Threads). Section Intel®
Optimized MP LINPACK Benchmark for Clusters
discusses another solution for a Hybrid (OpenMP* +
MPI) mode.

Note also that to use threaded Intel MKL, it is highly recommended to compile your code
with the /MT option.
See also:
Linking with Compiler Support RTLs.

6-4
 Managing Performance and Memory 6
Setting the Number of Threads Using OpenMP* Environment
Variable
You can set the number of threads using the environment variable OMP_NUM_THREADS. To
change the number of threads, in the command shell in which the program is going to run,
enter:
set OMP_NUM_THREADS=<number of threads to use> .
Some shells require the variable and its value to be exported:
export OMP_NUM_THREADS=<number of threads to use>
You can alternatively assign value to the environment variable using Microsoft* Windows*
Control Panel.
Note that you will not benefit from setting this variable on Microsoft* Windows* 98 or
Windows* ME because multiprocessing is not supported.
See Using Additional Threading Control on how to set the number of threads using Intel
MKL environment variables, for example, MKL_NUM_THREADS.

Changing the Number of Threads at Run Time

You cannot change the number of threads during run time using the environment variables.
However, you can call OpenMP API functions from your program to change the number of
threads during run time. The following sample code shows how to change the number of
threads during run time using the omp_set_num_threads() routine. See also Techniques
to Set the Number of Threads.
The following example shows both C and Fortran code examples. To run this example in the
C language, use the omp.h header file from the Intel® compiler package. If you do not
have the Intel compiler but wish to explore the functionality in the example, use Fortran
API for omp_set_num_threads() rather than the C version.

Example 6-1 Changing the Number of Threads

// ******* C language *******

#include "omp.h"
#include "mkl.h"
#include <stdio.h>

#define SIZE 1000

void main(int args, char *argv[]){

6-5
6 Intel® Math Kernel Library User’s Guide

Example 6-1 Changing the Number of Threads (continued)

double *a, *b, *c;
a = new double [SIZE*SIZE];
b = new double [SIZE*SIZE];
c = new double [SIZE*SIZE];
double alpha=1, beta=1;
int m=SIZE, n=SIZE, k=SIZE, lda=SIZE, ldb=SIZE, ldc=SIZE, i=0, j=0;
char transa='n', transb='n';
for( i=0; i<SIZE; i++){
for( j=0; j<SIZE; j++){
a[i*SIZE+j]= (double)(i+j);
b[i*SIZE+j]= (double)(i*j);
c[i*SIZE+j]= (double)0;
}
}
cblas_dgemm(CblasRowMajor, CblasNoTrans, CblasNoTrans,
m, n, k, alpha, a, lda, b, ldb, beta, c, ldc);
printf("row\ta\tc\n");
for ( i=0;i<10;i++){
printf("%d:\t%f\t%f\n", i, a[i*SIZE], c[i*SIZE]);
}

omp_set_num_threads(1);

for( i=0; i<SIZE; i++){

for( j=0; j<SIZE; j++){
a[i*SIZE+j]= (double)(i+j);
b[i*SIZE+j]= (double)(i*j);
c[i*SIZE+j]= (double)0;
}
}
cblas_dgemm(CblasRowMajor, CblasNoTrans, CblasNoTrans,
m, n, k, alpha, a, lda, b, ldb, beta, c, ldc);
printf("row\ta\tc\n");
for ( i=0;i<10;i++){
printf("%d:\t%f\t%f\n", i, a[i*SIZE], c[i*SIZE]);
}
omp_set_num_threads(2);
for( i=0; i<SIZE; i++){
for( j=0; j<SIZE; j++){
a[i*SIZE+j]= (double)(i+j);
b[i*SIZE+j]= (double)(i*j);
c[i*SIZE+j]= (double)0;
}
}
cblas_dgemm(CblasRowMajor, CblasNoTrans, CblasNoTrans,
m, n, k, alpha, a, lda, b, ldb, beta, c, ldc);

printf("row\ta\tc\n");
for ( i=0;i<10;i++){

6-6
 Managing Performance and Memory 6
Example 6-1 Changing the Number of Threads (continued)
printf("%d:\t%f\t%f\n", i, a[i*SIZE],
c[i*SIZE]);
}

delete [] a;
delete [] b;
delete [] c;
}

// ******* Fortran language *******

PROGRAM DGEMM_DIFF_THREADS

INTEGER N, I, J
PARAMETER (N=1000)
REAL*8 A(N,N),B(N,N),C(N,N)
REAL*8 ALPHA, BETA

INTEGER*8 MKL_MALLOC
integer ALLOC_SIZE

integer NTHRS

ALLOC_SIZE = 8*N*N
A_PTR = MKL_MALLOC(ALLOC_SIZE,128)
B_PTR = MKL_MALLOC(ALLOC_SIZE,128)
C_PTR = MKL_MALLOC(ALLOC_SIZE,128)

ALPHA = 1.1
BETA = -1.2

DO I=1,N
DO J=1,N
A(I,J) = I+J
B(I,J) = I*j
C(I,J) = 0.0
END DO
END DO

CALL DGEMM('N','N',N,N,N,ALPHA,A,N,B,N,BETA,C,N)

print *,'Row A C'

DO i=1,10
write(*,'(I4,F20.8,F20.8)') I, A(1,I),C(1,I)
END DO

CALL OMP_SET_NUM_THREADS(1);

DO I=1,N

6-7
6 Intel® Math Kernel Library User’s Guide

Example 6-1 Changing the Number of Threads (continued)

DO J=1,N
A(I,J) = I+J
B(I,J) = I*j
C(I,J) = 0.0
END DO
END DO

CALL DGEMM('N','N',N,N,N,ALPHA,A,N,B,N,BETA,C,N)

print *,'Row A C'

DO i=1,10
write(*,'(I4,F20.8,F20.8)') I, A(1,I),C(1,I)
END DO

CALL OMP_SET_NUM_THREADS(2);

DO I=1,N
DO J=1,N
A(I,J) = I+J
B(I,J) = I*j
C(I,J) = 0.0
END DO
END DO

CALL DGEMM('N','N',N,N,N,ALPHA,A,N,B,N,BETA,C,N)

print *,'Row A C'

DO i=1,10
write(*,'(I4,F20.8,F20.8)') I, A(1,I),C(1,I)
END DO

STOP
END

Using Additional Threading Control

Intel MKL has new optional threading controls, that is, the new environment variables and
service functions. They behave similar to their OpenMP equivalents, but take precedence
over them. By using these controls along with OpenMP variables, you can thread the part
of the application that does not call Intel MKL and the library independently from each
other.
These controls enable you to specify the number of threads for Intel MKL independently of
the OpenMP settings. Although Intel MKL may actually use a different number of threads
from the number suggested, the controls will also enable you to instruct the library to try
using the suggested number when the number used in the calling application is
unavailable.

6-8
 Managing Performance and Memory 6

NOTE. Intel MKL does not always have a choice on the number of
threads for certain reasons, such as system resources.

Use of the Intel MKL threading controls in your application is optional. If you do not use
them, the library will mainly behave the same way as Intel MKL 9.1 in what relates to
threading with the possible exception of a different default number of threads.
Section "Number of User Threads" in the "Fourier Transform Functions" chapter of the Intel
MKL Reference Manual shows how the Intel MKL threading controls help to set the number
of threads for the FFT computation.
Table 6-2 lists the Intel MKL environment variables for threading control, their equivalent
functions, and OMP counterparts:

Table 6-2 Environment Variables for Threading Controls

Equivalent
OpenMP*
Environment
Environment Variable Service Function Comment Variable
MKL_NUM_THREADS mkl_set_num_threads Suggests the number of OMP_NUM_THREADS
threads to use.
MKL_DOMAIN_NUM_ mkl_domain_set_num_ Suggests the number of
THREADS threads threads for a particular
function domain.
MKL_DYNAMIC mkl_set_dynamic Enables Intel MKL to OMP_DYNAMIC
dynamically change the
number of threads.

NOTE. The functions take precedence over the respective environment

variables.
In particular, if in your application, you want Intel MKL to use a given
number of threads and do not want users of your application to change
this via environment variables, set this number of threads by a call to
mkl_set_num_threads(), which will have full precedence over any
environment variables being set.

6-9
6 Intel® Math Kernel Library User’s Guide

The example below illustrates the use of the Intel MKL function mkl_set_num_threads()
to mimic the Intel MKL 9.x default behavior, that is, running on one thread.

Example 6-2 Setting the Number of Threads to One

// ******* C language *******
#include <omp.h>
#include <mkl.h>
…
mkl_set_num_threads ( 1 );

// ******* Fortran language *******

…
call mkl_set_num_threads( 1 )

The section further explains the Intel MKL environment variables for threading control. See
the Intel MKL Reference Manual for the detailed description of the threading control
functions, their parameters, calling syntax, and more code examples.

MKL_DYNAMIC
The default value of MKL_DYNAMIC is TRUE, regardless of OMP_DYNAMIC, whose default
value may be FALSE.
MKL_DYNAMIC being TRUE means that Intel MKL will always try to pick what it considers the
best number of threads, up to the maximum specified by the user.
MKL_DYNAMIC being FALSE means that Intel MKL will normally try not to deviate from the
number of threads the user requested. However, setting MKL_DYNAMIC=FALSE does not
ensure that Intel MKL will use the number of threads that you request mainly because the
library may have no choice on this number for such reasons as system resources.
Moreover, the library may examine the problem and pick a different number of threads
than the value suggested. For example, if you attempt to do a size 1 matrix-matrix multiply
across 8 threads, the library may instead choose to use only one thread because it is
impractical to use 8 threads in this event.
Note also that if Intel MKL is called in a parallel region, it will use only one thread by
default. If you want the library to use nested parallelism, and the thread within a parallel
region is compiled with the same OpenMP compiler as Intel MKL is using, you may
experiment with setting MKL_DYNAMIC to FALSE and manually increasing the number of
threads.
In general, you should set MKL_DYNAMIC to FALSE only under circumstances that Intel
MKL is unable to detect, for example, when nested parallelism is desired where the library
is called already from a parallel section.

6-10
 Managing Performance and Memory 6
MKL_DYNAMIC being TRUE, in particular, provides for optimal choice of the number of
threads in the following cases:
• If the requested number of threads exceeds the number of physical cores (perhaps
because of hyper-threading), and MKL_DYNAMIC is not changed from its default value
of TRUE, Intel MKL will scale down the number of threads to the number of physical
cores.
• If you are able to detect the presence of MPI, but cannot determine if it has been
called in a thread-safe mode (it is impossible to detect this with MPICH 1.2.x, for
instance), and MKL_DYNAMIC has not been changed from its default value of TRUE,
Intel MKL will run one thread.

MKL_DOMAIN_NUM_THREADS
MKL_DOMAIN_NUM_THREADS accepts a string value <MKL-env-string>, which must have
the following format:
<MKL-env-string> ::= <MKL-domain-env-string> { <delimiter>
<MKL-domain-env-string> }
<delimiter> ::= [ <space-symbol>* ] ( <space-symbol> | <comma-symbol> |
<semicolon-symbol> | <colon-symbol> ) [ <space-symbol>* ]
<MKL-domain-env-string> ::= <MKL-domain-env-name> <uses>
<number-of-threads>
<MKL-domain-env-name> ::= MKL_ALL | MKL_BLAS | MKL_FFT | MKL_VML
<uses> ::= [ <space-symbol>* ] ( <space-symbol> | <equality-sign> |
<comma-symbol>) [ <space-symbol>* ]
<number-of-threads> ::= <positive-number>
<positive-number> ::= <decimal-positive-number> | <octal-number> |
<hexadecimal-number>
In the syntax above, MKL_BLAS indicates the BLAS function domain, MKL_FFT indicates
non-cluster FFTs, and MKL_VML indicates the Vector Mathematics Library.
For example,
MKL_ALL 2 : MKL_BLAS 1 : MKL_FFT 4
MKL_ALL=2 : MKL_BLAS=1 : MKL_FFT=4
MKL_ALL=2, MKL_BLAS=1, MKL_FFT=4
MKL_ALL=2; MKL_BLAS=1; MKL_FFT=4
MKL_ALL = 2 MKL_BLAS 1 , MKL_FFT 4
MKL_ALL,2: MKL_BLAS 1, MKL_FFT,4 .

6-11
6 Intel® Math Kernel Library User’s Guide

The global variables MKL_ALL, MKL_BLAS, MKL_FFT, and MKL_VML, as well as the interface
for the Intel MKL threading control functions, can be found in the mkl.h header file.
Table 6-3 illustrates how values of MKL_DOMAIN_NUM_THREADS are interpreted.

Table 6-3 Interpretation of MKL_DOMAIN_NUM_THREADS Values

Value of
MKL_DOMAIN_NUM_THREADS Interpretation
MKL_ALL=4 All parts of Intel MKL are suggested to try using 4 threads. The actual
number of threads may be still different because of the MKL_DYNAMIC
setting or system resource issues. The setting is equivalent to
MKL_NUM_THREADS = 4.
MKL_ALL=1, MKL_BLAS=4 All parts of Intel MKL are suggested to use 1 thread, except for BLAS,
which is suggested to try 4 threads.
MKL_VML = 2 VML is suggested to try 2 threads. The setting affects no other part of
Intel MKL.

NOTE. The domain-specific settings take precedence over the overall

ones. For example, the "MKL_BLAS=4" value of
MKL_DOMAIN_NUM_THREADS suggests trying 4 threads for BLAS,
regardless of later setting MKL_NUM_THREADS, and a function call
"mkl_domain_set_num_threads ( 4, MKL_BLAS );" suggests the
same, regardless of later calls to mkl_set_num_threads().
However, pay attention to that a function call with input "MKL_ALL", such
as "mkl_domain_set_num_threads (4, MKL_ALL);" is equivalent to
"mkl_set_num_threads(4)", and thus it will be overwritten by later
calls to mkl_set_num_threads. Similarly, the environment setting of
MKL_DOMAIN_NUM_THREADS with "MKL_ALL=4" will be overwritten with
MKL_NUM_THREADS = 2.

Whereas the MKL_DOMAIN_NUM_THREADS environment variable enables you set several

variables at once, for example, "MKL_BLAS=4,MKL_FFT=2", the corresponding function
does not take string syntax. So, to do the same with the function calls, you may need to
make several calls, which in this example are as follows:
mkl_domain_set_num_threads ( 4, MKL_BLAS );
mkl_domain_set_num_threads ( 2, MKL_FFT );

6-12
 Managing Performance and Memory 6
Setting the Environment Variables for Threading Control
To set the environment variables used for threading control, in the command shell in which
the program is going to run, enter:
set <VARIABLE NAME>=<value>
For example,
set MKL_NUM_THREADS=4
set MKL_DOMAIN_NUM_THREADS="MKL_ALL=1, MKL_BLAS=4"
set MKL_DYNAMIC=FALSE
Some shells require the variable and its value to be exported:
export <VARIABLE NAME>=<value>
For example:
export MKL_NUM_THREADS=4
export MKL_DOMAIN_NUM_THREADS="MKL_ALL=1, MKL_BLAS=4"
export MKL_DYNAMIC=FALSE
You can alternatively assign values to the environment variables using Microsoft*
Windows* Control Panel.

Dispatching Intel® Advanced Vector Extensions (Intel®

AVX)
Intel MKL 10.2 introduced optimized kernels for Intel® AVX. To have the Intel AVX
instructions dispatched on Intel AVX-enabled hardware (or simulation), use the Intel MKL
service function mkl_enable_instructions(). This function enables dispatching new
Intel AVX instructions. Call this function before any other Intel MKL function call. For the
function description, see the Intel MKL Reference Manual.

NOTE. Successful execution of this function does not guarantee new

instructions to be dispatched. Dispatching of a particular instruction
depends on whether the hardware is Intel AVX-enabled and whether the
function is already optimized to dispatch this instruction. However, if you
do not call this function, new instructions will not be dispatched.

6-13
6 Intel® Math Kernel Library User’s Guide

As the Intel AVX instruction set is evolving, the behavior of

mkl_enable_instructions() may change with future Intel MKL releases. Refer to the
Release Notes for release-specific details of the function behavior.

Tips and Techniques to Improve Performance

To obtain the best performance with Intel MKL, follow the recommendations given in the
subsections below.

Coding Techniques
To obtain the best performance with Intel MKL, ensure the following data alignment in your
source code:
• Align arrays at 16-byte boundaries.
• Make sure leading dimension values (n*element_size) of two-dimensional arrays are
divisible by 16.
• For two-dimensional arrays, avoid leading dimension values divisible by 2048.

LAPACK Packed Routines

The routines with the names that contain the letters HP, OP, PP, SP, TP, UP in the matrix
type and storage position (the second and third letters respectively) operate on the
matrices in the packed format (see LAPACK "Routine Naming Conventions" sections in the
Intel MKL Reference Manual). Their functionality is strictly equivalent to the functionality of
the unpacked routines with the names containing the letters HE, OR, PO, SY, TR, UN in the
same positions, but the performance is significantly lower.
If the memory restriction is not too tight, use an unpacked routine for better performance.
In this case, you need to allocate N2/2 more memory than the memory required by a
respective packed routine, where N is the problem size (the number of equations).
For example, to speed up solving a symmetric eigenproblem with an expert driver by using
an unpacked routine:
call dsyevx(jobz, range, uplo, n, a, lda, vl, vu, il, iu, abstol, m, w,
z, ldz, work, lwork, iwork, ifail, info),
where a is the dimension lda-by-n, which is at least N2 elements, instead of
call dspevx(jobz, range, uplo, n, ap, vl, vu, il, iu, abstol, m, w, z,
ldz, work, iwork, ifail, info),
where ap is the dimension N*(N+1)/2.

6-14
 Managing Performance and Memory 6
FFT Functions
Additional conditions can improve performance of the FFT functions.

Applications based on the IA-32 or Intel® 64 architecture. The addresses of the first
elements of arrays and the leading dimension values, in bytes (n*element_size), of
two-dimensional arrays should be divisible by cache line size, which equals:
• 32 bytes for the Intel® Pentium® III processors
• 64 bytes for the Intel® Pentium® 4 processors and processors using Intel® 64
architecture

Applications based on the IA-64 architecture. Leading dimension values, in bytes

(n*element_size), of two-dimensional arrays are not a power of two.

Hardware Configuration Tips

Dual-Core Intel® Xeon® processor 5100 series systems. To get the best Intel MKL
performance on Dual-Core Intel® Xeon® processor 5100 series systems, enable the
Hardware DPL (streaming data) Prefetcher functionality of this processor. To configure this
functionality, use the appropriate BIOS settings where, as described in your BIOS
documentation.

The use of Hyper-Threading Technology. Hyper-Threading Technology (HT Technology) is

especially effective when each thread is performing different types of operations and when
there are under-utilized resources on the processor. However, Intel MKL fits neither of
these criteria because the threaded portions of the library execute at high efficiencies using
most of the available resources and perform identical operations on each thread. You may
obtain higher performance by disabling HT Technology. See Using the Intel® MKL
Parallelism for information on the default number of threads, changing this number, and
other relevant details.
If you run with HT enabled, performance may be especially impacted if you run on fewer
threads than physical cores. Moreover, if, for example, there are two threads to every
physical core, the thread scheduler may assign two threads to some cores and ignore the
other ones altogether. If you are using the OpenMP* library of the Intel Compiler, read the
respective User Guide on how to best set the thread affinity interface to avoid this
situation. For Intel MKL, you are recommended to set
KMP_AFFINITY=granularity=fine,compact,1,0.

6-15
6 Intel® Math Kernel Library User’s Guide

Managing Multi-core Performance

You can obtain best performance on systems with multi-core processors by requiring that
threads do not migrate from core to core. To do this, bind threads to the CPU cores by
setting an affinity mask to threads. You can do it using any of the following options:
• OpenMP facilities (recommended, if available), for instance, the KMP_AFFINITY
environment variable using the Intel OpenMP library
• A system function, as in the example below
Suppose:
• The system has two sockets with two cores each, for a total of 4 cores (CPUs).
• Performance of the four-thread parallel application using the Intel MKL LAPACK
happens to be unstable
The C code presented in Example 6-3 solves the problem. The code example calls the
system function SetThreadAffinityMask to bind the threads to appropriate cores, thus
preventing migration of the threads. After that the Intel MKL LAPACK routine is called.
Compile your application with the Intel compiler using the following command:
icl /Qopenmp test_application.c
where test_application.c is the filename for the application.
Build the application. Run it in 4 threads, for example, by using the environment variable to
set the number of threads:
set OMP_NUM_THREADS=4
test_application.exe

Example 6-3 Setting An affinity Mask by Operating System Means Using the Intel® Compiler

// Set affinity mask

#include <windows.h>
#include <omp.h>

int main(void) {
#pragma omp parallel default(shared)

{
int tid = omp_get_thread_num();

// 2 packages x 2 cores/pkg x 1 threads/core (4 total cores)

DWORD_PTR mask = (1 << (tid == 0 ? 0 : 2 ));
SetThreadAffinityMask( GetCurrentThread(), mask );
}

6-16
 Managing Performance and Memory 6
Example 6-3 Setting An affinity Mask by Operating System Means Using the Intel® Compiler
(continued)

// Call Intel MKL LAPACK routine

return 0;

See Windows API documentation at http ://msdn .microsoft .com/ for the restrictions on the
usage of Windows API routines and particulars of the SetThreadAffinityMask function
used in the above example.

Operating on Denormals
The IEEE 754-2008 standard, "An IEEE Standard for Binary Floating-Point Arithmetic",
defines denormal (or subnormal) numbers as non-zero numbers smaller than the smallest
possible normalized numbers for a specific floating-point format. Floating-point operations
on denormals are slower than on normalized operands because denormal operands and
results are usually handled through a software assist mechanism rather than directly in
hardware. This software processing causes Intel MKL functions that consume denormals to
run slower than with normalized floating-point numbers.
You can mitigate this performance issue by setting the appropriate bit fields in the MXCSR
floating-point control register to flush denormals to zero (FTZ) or to replace any denormals
loaded from memory with zero (DAZ). Check your compiler documentation to determine
whether it has options to control FTZ and DAZ. Note that these compiler options may
slightly affect accuracy.

FFT Optimized Radices

You can improve the performance of Intel MKL FFT if the length of your data vector permits
factorization into powers of optimized radices.
In Intel MKL, the optimized radices are 2, 3, 5, 7, and 11.

Using the Intel® MKL Memory Management

Intel MKL has memory management software that controls memory buffers for the use by
the library functions. New buffers that the library allocates when your application calls
certain functions (Level 3 BLAS or FFT) are not deallocated until the program ends. To get
the amount of memory allocated by the memory management software, call the

6-17
6 Intel® Math Kernel Library User’s Guide

mkl_mem_stat() function. If your program needs to free memory, call

mkl_free_buffers(). If another call is made to a library function that needs a memory
buffer, the memory manager will again allocate the buffers and they will again remain
allocated until either the program ends or the program deallocates the memory.
This behavior facilitates better performance. However, some tools may report this behavior
as a memory leak. In addition to calling the mkl_free_buffers() function, you can
release (free) memory in your program by setting an environment variable.
The memory management software is turned on by default, which leaves memory
allocated by calls to Level 3 BLAS and FFT until the program ends. To disable this behavior
of the memory management software, set the MKL_DISABLE_FAST_MM environment
variable to any value, which will cause memory to be allocated and freed from call to call.
Disabling this feature will negatively impact performance of routines such as the level 3
BLAS, especially for small problem sizes.
Using one of these methods to release memory will not necessarily stop programs from
reporting memory leaks, and, in fact, may increase the number of such reports in case you
make multiple calls to the library, thereby requiring new allocations with each call. Memory
not released by one of the methods described previously will be released by the system
when the program ends.

Redefining Memory Functions

C/C++ users of Intel MKL can replace memory functions that the library uses by default
with their own ones. The memory renaming feature enables this replacement.

Memory Renaming
Intel MKL memory management by default uses standard C run-time memory functions to
allocate or free memory. These functions can be replaced using memory renaming.
Intel MKL accesses the memory functions by pointers i_malloc, i_free, i_calloc, and
i_realloc, which are visible at the application level. These pointers initially hold
addresses of the standard C run-time memory functions malloc, free, calloc, and
realloc, respectively. You can programmatically redefine values of these pointers to the
addresses of your application's memory management functions.
Redirecting the pointers is the only correct way to use your own set of memory
management functions. If you call your own memory functions without redirecting the
pointers, the memory will get managed by two independent memory management
packages, which may cause unexpected memory issues.

How to Redefine Memory Functions

To redefine memory functions, use the following procedure:
If you are using the statically linked Intel MKL,

6-18
 Managing Performance and Memory 6
1. Include the i_malloc.h header file in your code.
This header file contains all declarations required for an application developer to
replace the memory allocation functions. The header file also describes how memory
allocation can be replaced in those Intel libraries that support this feature.
2. Redefine values of pointers i_malloc, i_free, i_calloc, i_realloc prior to the
first call to MKL functions:

Example 6-4 Redefining Memory Functions

#include "i_malloc.h"
. . .
i_malloc = my_malloc;
i_calloc = my_calloc;
i_realloc = my_realloc;
i_free = my_free;
. . .
// Now you may call Intel MKL functions

If you are using the dynamically linked Intel MKL,

1. Include the i_malloc_dll.h header file in your code.
2. Redefine values of pointers used in the previous example as well as similar pointers to
be used in dlls (their names have the _dll suffix) prior to the first call to MKL
functions:

6-19
6 Intel® Math Kernel Library User’s Guide

Example 6-5 Redefining Memory Functions for Dynamically Linked Intel® MKL
#include "i_malloc_dll.h"
. . .
i_malloc = my_malloc;
i_calloc = my_calloc;
i_realloc = my_realloc;
i_free = my_free;
i_malloc_dll = my_malloc;
i_calloc_dll = my_calloc;
i_realloc_dll = my_realloc;
i_free_dll = my_free;
. . .
// Now you may call Intel MKL functions

6-20
Language-specific Usage
Options 7
The Intel® Math Kernel Library (Intel® MKL) provides broad support for Fortran and
C/C++ programming. However, not all function domains support both Fortran and C
interfaces (see Table A-1 in Appendix A). For example, LAPACK has no C interface. You can
call functions comprising such domains from C using mixed-language programming.
If you want to use LAPACK or BLAS, which support Fortran, in the Fortran 95 environment,
additional effort may be initially required to build compiler-specific interface libraries and
modules from the source code provided with Intel MKL.
This chapter mainly focuses on mixed-language programming and the use of
language-specific interfaces. It expands upon the use of Intel MKL in C language
environments for function domains that provide only Fortran interfaces, as well as explains
usage of language-specific interfaces, specifically the Fortran 95 interfaces to LAPACK and
BLAS. The chapter also discusses compiler-dependent functions to explain why Fortran 90
modules are supplied as sources. A separate section guides you through the process of
running examples to invoke Intel MKL functions from Java*.

Using Language-Specific Interfaces with Intel® MKL

You can create the following interface libraries and modules using the respective makefiles
located in the interfaces directory.

Table 7-1 Interface Libraries and Modules

File name Contains
Libraries, in Intel MKL architecture-specific directories
mkl_blas95.lib1 Fortran 95 wrappers for BLAS (BLAS95) for IA-32
architecture.
mkl_blas95_ilp64.lib1 Fortran 95 wrappers for BLAS (BLAS95) supporting LP64
interface.

7-1
7 Intel® Math Kernel Library User’s Guide

Table 7-1 Interface Libraries and Modules (continued)

File name Contains
mkl_blas95_lp64.lib1 Fortran 95 wrappers for BLAS (BLAS95) supporting ILP64
interface.
mkl_lapack95.lib1 Fortran 95 wrappers for LAPACK (LAPACK95) for IA-32
architecture.
mkl_lapack95_lp64.lib1 Fortran 95 wrappers for LAPACK (LAPACK95) supporting
LP64 interface.
mkl_lapack95_ilp64.lib1 Fortran 95 wrappers for LAPACK (LAPACK95) supporting
ILP64 interface.
fftw2xc_intel.lib1 Interfaces for FFTW version 2.x (C interface for Intel®
compiler) to call Intel MKL FFTs.
fftw2xc_ms.lib Contains interfaces for FFTW version 2.x (C interface for
Microsoft compiler) to call Intel MKL FFTs.
fftw2xf_intel.lib Interfaces for FFTW version 2.x (Fortran interface for
Intel compiler) to call Intel MKL FFTs.
fftw3xc_intel.lib2 Interfaces for FFTW version 3.x (C interface for Intel
compiler) to call Intel MKL FFTs.
fftw3xc_ms.lib Interfaces for FFTW version 3.x (C interface for Microsoft
compiler) to call Intel MKL FFTs.
fftw3xf_intel.lib2 Interfaces for FFTW version 3.x (Fortran interface for
Intel compiler) to call Intel MKL FFTs.
fftw2x_cdft_SINGLE.lib Single-precision interfaces for MPI FFTW version 2.x (C
interface) to call Intel MKL cluster FFTs.
fftw2x_cdft_DOUBLE.lib Double-precision interfaces for MPI FFTW version 2.x (C
interface) to call Intel MKL cluster FFTs.
Modules, in architecture- and interface-specific subdirectories of the Intel MKL include directory
blas95.mod1 Fortran 95 interface module for BLAS (BLAS95).

lapack95.mod 1 Fortran 95 interface module for LAPACK (LAPACK95).

f95_precision.mod 1 Fortran 95 definition of precision parameters for BLAS95

and LAPACK95.
mkl95_blas.mod1 Fortran 95 interface module for BLAS (BLAS95), identical
to blas95.mod. To be removed in one of the future
releases.
mkl95_lapack.mod1 Fortran 95 interface module for LAPACK (LAPACK95),
identical to lapack95.mod. To be removed in one of
the future releases.
mkl95_precision.mod1 Fortran 95 definition of precision parameters for BLAS95
and LAPACK95, identical to f95_precision.mod. To
be removed in one of the future releases.
1. Prebuilt for the Intel® Fortran compiler

7-2
 Language-specific Usage Options 7
2. FFTW3 interfaces are integrated with Intel MKL. Look into <mkl directory>\interfaces\fftw3x*\makefile for options
defining how to build and where to place the standalone library with the wrappers.

See Fortran 95 Interfaces to LAPACK and BLAS, which shows by example how to generate
these libraries and modules.
See Appendix G in the Intel MKL Reference Manual for details of FFTW to Intel MKL
wrappers.

Fortran 95 Interfaces to LAPACK and BLAS

Fortran 95 interfaces are compiler-dependent. Intel MKL provides the interface libraries
and modules precompiled with the Intel® Fortran compiler. Additionally, the Fortran 95
interfaces and wrappers are delivered as sources. (For more information, see
Compiler-dependent Functions and Fortran 90 Modules). If you are using a different
compiler, build the appropriate library and modules with your compiler and link the library
as a user's library:
1. Go to the respective directory <mkl_directory>\interfaces\blas95 or
<mkl_directory>\interfaces\lapack95
2. Type one of the following commands:

nmake lib32 install_dir=<user_dir> for the IA-32 architecture.

nmake libem64t [interface=lp64|ilp64] for the Intel® 64 architecture.
install_dir=<user_dir>
nmake lib64 [interface=lp64|ilp64] for the IA-64 architecture.
install_dir=<user_dir>

NOTE. Parameter install_dir is required.

As a result, the required library will be built and installed in the

<user dir>\<arch>\lib directory, and the .mod files will be built and installed in the
<user dir>\include\<arch>[\{lp64 | ilp64}] directory, where <arch> is one of
{ia32, em64t, ia64}.
By default, the ifort compiler is assumed. You may change the compiler command name
with an additional parameter of nmake: FC=<compiler>.
For example, command
nmake libem64t FC=f95 install_dir=<user_f95_dir> interface=lp64
builds the required library and .mod files and installs them in subdirectories of
<user_f95_dir>.
There is also a way to use the interfaces without building the libraries.

7-3
7 Intel® Math Kernel Library User’s Guide

To delete the library from the building directory, use the following commands:

nmake clean32 install_dir=<user_dir> for the IA-32 architecture.

nmake cleanem64t install_dir=<user_dir> for the Intel® 64 architecture.
nmake clean64 install_dir=<user_dir> for the IA-64 architecture.
nmake clean install_dir=<user_dir> for all the architectures.

NOTE. Setting install_dir=..\.. or

install_dir=<mkl_directory> in a build or clean command above will
replace or delete the Intel MKL prebuilt Fortran 95 library and modules.
Though this is possible only if you have administrative rights, you are
strongly discouraged from doing this.

Compiler-dependent Functions and Fortran 90 Modules

Compiler-dependent functions occur whenever the compiler places into the object code
function calls that are resolved in its run-time library (RTL). Linking of such code without
the appropriate RTL will result in undefined symbols. Intel MKL has been designed to
minimize RTL dependencies.
Where dependencies occur, a supporting RTL is shipped with Intel MKL. The only examples
of such RTLs, except those that are relevant to the Intel MKL cluster software, are libiomp
and libguide, which are the libraries for the OpenMP* code compiled with an Intel®
compiler. Both libiomp and libguide support the threaded code in Intel MKL.
In other cases where RTL dependencies might arise, the functions are delivered as source
code and you need to compile the code with whatever compiler you are using for your
application.
In particular, Fortran 90 modules result in the compiler-specific code generation requiring
RTL support, so, Intel MKL delivers these modules as source code.

Mixed-language Programming with Intel® MKL

Appendix A lists the programming languages supported for each Intel MKL function
domain. However, you can call Intel MKL routines from different language environments.
This section explains how to do this using mixed-language programming.

7-4
 Language-specific Usage Options 7
Calling LAPACK, BLAS, and CBLAS Routines from C Language
Environments
Not all Intel MKL function domains support both C and Fortran environments. To use Intel
MKL Fortran-style functions in C/C++ environments, you should observe certain
conventions, which are discussed for LAPACK and BLAS in the subsections below.

WARNING. Avoid calling BLAS95/LAPACK95 from C/C++. Such calls

require skills in manipulating the descriptor of a deferred-shape array,
which is the Fortran 90 type. Moreover, BLAS95/LAPACK95 routines
contain links to a Fortran RTL.

LAPACK
Because LAPACK routines are Fortran-style, when calling them from C-language programs,
make sure that you follow the Fortran-style calling conventions:
• Pass variables by address as opposed to pass by value.
Function calls in Example 7-2 and Example 7-3 illustrate this.
• Store your data in Fortran style, that is, column-major rather than row-major order.
With row-major order, adopted in C, the last array index changes most quickly and the
first one changes most slowly when traversing the memory segment where the array is
stored. With Fortran-style column-major order, the last index changes most slowly
whereas the first one changes most quickly (as illustrated by Figure 7-1 for a
two-dimensional array).

7-5
7 Intel® Math Kernel Library User’s Guide

Figure 7-1 Column-major Order versus Row-major Order

For example, if a two-dimensional matrix A of size m x n is stored densely in a

one-dimensional array B, you can access a matrix element like this:
A[i][j] = B[i*n+j] in C (i=0, ... , m-1, j=0, ... , n-1)
A(i,j) = B(j*m+i) in Fortran (i=1, ... , m, j=1, ... , n).
When calling LAPACK routines from C, be aware that because the Fortran language is
case-insensitive, the LAPACK routine names can be both upper-case or lower-case, with or
without the trailing underscore. For example, these names are equivalent: dgetrf,
DGETRF, dgetrf_, DGETRF_.

BLAS
BLAS routines are Fortran-style routines. If you call BLAS routines from a C-language
program, you must follow the Fortran-style calling conventions:
• Pass variables by address as opposed to passing by value
• Store data in Fortran style, that is, column-major rather than row-major order
Refer to the LAPACK section for details of these conventions. See Example 7-2 on how to
call BLAS routines from C.
When calling BLAS routines from C, be aware that because the Fortran language is
case-insensitive, the BLAS routine names can be both upper-case and lower-case, with or
without the trailing underscore. For example, these names are equivalent: dgemm, DGEMM,
dgemm_, DGEMM_.

7-6
 Language-specific Usage Options 7
CBLAS
Instead of calling BLAS routines from a C-language program, you can use the CBLAS
interface.
CBLAS is a C-style interface to the BLAS routines. You can call CBLAS routines using
regular C-style calls. When using the CBLAS interface, the header file mkl.h will simplify
the program development because it specifies enumerated values as well as prototypes of
all the functions. The header determines if the program is being compiled with a C++
compiler, and if it is, the included file will be correct for use with C++ compilation.
Example 7-3 illustrates the use of the CBLAS interface.

Using Complex Types in C/C++

As described in the Building Applications document for the Intel® Visual Fortran Compiler,
C/C++ does not directly implement the Fortran types COMPLEX(4) and COMPLEX(8).
However, you can write equivalent structures. The type COMPLEX(4)consists of two 4-byte
floating-point numbers. The first of them is the real-number component, and the second
one is the imaginary-number component. The type COMPLEX(8) is similar to COMPLEX(4)
except that it contains two 8-byte floating-point numbers.
Intel MKL provides complex types MKL_Complex8 and MKL_Complex16, which are
structures equivalent to the Fortran complex types COMPLEX(4) and COMPLEX(8),
respectively. These types are defined in the mkl_types.h header file. You can use these
types to define complex data. You can also redefine the types with your own types before
including the mkl_types.h header file. The only requirement is that the types must be
compatible with the Fortran complex layout, that is, the complex type must be a pair of
real numbers for the values of real and imaginary parts.
For example, you can use the following definitions in your C++ code:
#define MKL_Complex8 std::complex<float>
and
#define MKL_Complex16 std::complex<double> .
See Example 7-2 for details. You can also define these types in the command line:
-DMKL_Complex8="std::complex<float>"
-DMKL_Complex16="std::complex<double>".

7-7
7 Intel® Math Kernel Library User’s Guide

Calling BLAS Functions that Return the Complex Values in

C/C++ Code
You must be careful when handling a call from C to a BLAS function that returns complex
values. The problem arises because BLAS comprises Fortran functions, and complex values
they return are handled quite differently for C and Fortran. However, in addition to normal
function calls, Fortran enables calling functions as though they were subroutines, which
provides a mechanism for returning the complex value correctly when the function is called
from a C program. When a Fortran function is called as a subroutine, the return value is the
first parameter in the calling sequence. This feature can be used by the C programmer.
The following example shows how this works.
Normal Fortran function call: result = cdotc( n, x, 1, y, 1 ).
A call to the function as a
subroutine: call cdotc( result, n, x, 1, y, 1).
A call to the function from C
(notice that the hidden
parameter gets exposed): cdotc( &result, &n, x, &one, y, &one ).

NOTE. Intel MKL has both upper-case and lower-case entry points in the
Fortran-style (case-insensitive) BLAS, with or without the trailing
underscore. So, all these names are equivalent and acceptable: cdotc,
CDOTC, cdotc_, CDOTC_.

The above example shows one of the ways to call several level 1 BLAS functions that return
complex values from your C and C++ applications. An easier way is to use the CBLAS
interface. For instance, you can call the same function using the CBLAS interface as
follows:
cblas_cdotu( n, x, 1, y, 1, &result )

NOTE. The complex value comes last on the argument list in this case.

The following examples show use of the Fortran-style BLAS interface from C and C++, as
well as the CBLAS (C language) interface.

7-8
 Language-specific Usage Options 7
The example below illustrates a call from a C program to the complex BLAS Level 1
function zdotc(). This function computes the dot product of two double-precision complex
vectors.
In this example, the complex dot product is returned in the structure c.

Example 7-1 Calling a Complex BLAS Level 1 Function from C

#include "mkl.h"
#define N 5
void main()
{
MKL_int n = N, inca = 1, incb = 1, i;
MKL_Complex16 a[N], b[N], c;
for( i = 0; i < n; i++ ){
a[i].re = (double)i; a[i].im = (double)i * 2.0;
b[i].re = (double)(n - i); b[i].im = (double)i * 2.0;
}
zdotc( &c, &n, a, &inca, b, &incb );
printf( "The complex dot product is: ( %6.2f, %6.2f)\n", c.re, c.im );
}

7-9
7 Intel® Math Kernel Library User’s Guide

Below is the C++ implementation:

Example 7-2 Calling a Complex BLAS Level 1 Function from C++

#include <complex>
#include <iostream>
#define MKL_Complex16 std::complex<double>
#include "mkl.h"

#define N 5

int main()
{
int n, inca = 1, incb = 1, i;
std::complex<double> a[N], b[N], c;
n = N;
for( i = 0; i < n; i++ ){
a[i] = std::complex<double>(i,i*2.0);
b[i] = std::complex<double>(n-i,i*2.0);
}
zdotc(&c, &n, a, &inca, b, &incb );
std::cout << "The complex dot product is: " << c << std::endl;
return 0;
}

7-10
 Language-specific Usage Options 7
The example below uses CBLAS:

Example 7-3 Using CBLAS Interface Instead of Calling BLAS Directly from C
#include "mkl.h"
typedef struct{ double re; double im; } complex16;

extern "C" void cblas_zdotc_sub ( const int , const complex16 *,

const int , const complex16 *, const int, const complex16*);

#define N 5

void main()
{

int n, inca = 1, incb = 1, i;

complex16 a[N], b[N], c;

n = N;
for( i = 0; i < n; i++ ){

a[i].re = (double)i; a[i].im = (double)i * 2.0;

b[i].re = (double)(n - i); b[i].im = (double)i * 2.0;
}
cblas_zdotc_sub(n, a, inca, b, incb,&c );
printf( "The complex dot product is: ( %6.2f, %6.2f)\n", c.re, c.im );
}

Support for Boost uBLAS Matrix-matrix Multiplication

If you are used to uBLAS, you can perform BLAS matrix-matrix multiplication in C++ using
Intel MKL substitution of Boost uBLAS functions. uBLAS pertains to the Boost C++
open-source libraries and provides BLAS functionality for dense, packed, and sparse

7-11
7 Intel® Math Kernel Library User’s Guide

matrices. The library uses an expression template technique for passing expressions as
function arguments, which enables evaluating vector and matrix expressions in one pass
without temporary matrices. uBLAS provides two modes:
• Debug (safe) mode, default.
Type and conformance checking is performed.
• Release (fast) mode.
Enabled by the NDEBUG preprocessor symbol.
The documentation for the Boost uBLAS is available at www. boost. org/.
Intel MKL provides overloaded prod() functions for substituting uBLAS dense
matrix-matrix multiplication with the Intel MKL gemm calls. Though these functions break
uBLAS expression templates and introduce temporary matrices, the performance
advantage can be considerable for matrix sizes that are not too small (roughly, over 50).
You do not need to change your source code to use the functions. To call them:
• Include the header file mkl_boost_ublas_matrix_prod.hpp in your code (from the
Intel MKL include directory).
• Add appropriate Intel MKL libraries to the link line (see Linking Your Application with
the Intel® Math Kernel Library).
Only the following expressions are substituted:
prod( m1, m2 )
prod( trans(m1), m2 )
prod( trans(conj(m1)), m2 )
prod( conj(trans(m1)), m2 )
prod( m1, trans(m2) )
prod( trans(m1), trans(m2) )
prod( trans(conj(m1)), trans(m2) )
prod( conj(trans(m1)), trans(m2) )
prod( m1, trans(conj(m2)) )
prod( trans(m1), trans(conj(m2)) )
prod( trans(conj(m1)), trans(conj(m2)) )
prod( conj(trans(m1)), trans(conj(m2)) )
prod( m1, conj(trans(m2)) )
prod( trans(m1), conj(trans(m2)) )
prod( trans(conj(m1)), conj(trans(m2)) )

7-12
 Language-specific Usage Options 7
prod( conj(trans(m1)), conj(trans(m2)) )
These expressions are substituted in the release mode only (with NDEBUG preprocessor
symbol defined). Supported uBLAS versions are Boost 1.34.1, 1.35.0, 1.36.0, and 1.37.0.
To get them, visit www .boost . org.
A code example provided in the
<mkl_directory>\examples\ublas\source\sylvester.cpp file illustrates usage of
the Intel MKL uBLAS header file for solving a special case of the Sylvester equation.
To run the Intel MKL ublas examples, specify the boost_root parameter in the nmake
command, for instance, when using Boost version 1.37.0:
nmake lib32 boost_root=<your_path>\boost_1_37_0
Intel MKL ublas examples on default Boost uBLAS configuration support only:
• Microsoft Visual C++* compiler versions 2003, 2005, and 2008
• Intel C++ compiler 10.1 and 11.x with Microsoft Visual Studio IDE 2003, 2005, and
2008

Invoking Intel® MKL Functions from Java* Applications

This section describes examples that are provided with the Intel MKL package and illustrate
calling the library functions from Java.

Intel MKL Java Examples

Java was positioned by its inventor, the Sun Microsystems Corporation as a "Write Once
Run Anywhere" (WORA) language. Intel MKL may help to speed-up Java applications, while
partially supporting the WORA philosophy, because Intel MKL editions are intended for a
wide variety of operating systems and processors including most kinds of laptops and
desktops, as well as many workstations and servers.
To demonstrate binding with Java, Intel MKL includes the set of Java examples found in the
following directory:
<mkl directory>\examples\java .
The examples are provided for the following MKL functions:
• ?gemm, ?gemv, and ?dot families from CBLAS
• The complete set of non-cluster FFT functions
• ESSL1-like functions for one-dimensional convolution and correlation
• VSL Random Number Generators (RNG), except user-defined ones and file subroutines

1. IBM Engineering Scientific Subroutine Library (ESSL*).

7-13
7 Intel® Math Kernel Library User’s Guide

• VML functions, except GetErrorCallBack, SetErrorCallBack, and

ClearErrorCallBack
You can see the example sources in the following directory:
<mkl directory>\examples\java\examples .
The examples are written in Java. They demonstrate usage of the MKL functions with the
following variety of data:
• 1- and 2-dimensional data sequences
• real and complex types of the data
• single and double precision
However, note that the wrappers, used in the examples, do not:
• Demonstrate the use of huge arrays (>2 billion elements)
• Demonstrate processing of arrays in native memory
• Check correctness of function parameters
• Demonstrate performance optimizations
To bind with Intel MKL, the examples use the Java Native Interface (JNI* developer
framework). The JNI documentation is available from
http://java.sun.com/javase/6/docs/technotes/guides/jni/ .
The Java example set includes JNI wrappers that perform the binding. The wrappers do not
depend on the examples and may be used in your Java applications. The wrappers for
CBLAS, FFT, VML, VSL RNG, and ESSL-like convolution and correlation functions do not
depend on each other.
To build the wrappers, just run the examples (see Running the Examples for details). The
makefile builds the wrapper binaries. After running the makefile, you can run the
examples, which will determine whether the wrappers were built correctly. As a result of
running the examples, the following directories will be created in
<mkl directory>\examples\java:
• docs
• include
• classes
• bin
• _results
The directories docs, include, classes, and bin will contain the wrapper binaries and
documentation; the directory _results will contain the testing results.
For a Java programmer, the wrappers look like the following Java classes:
• com.intel.mkl.CBLAS

7-14
 Language-specific Usage Options 7
• com.intel.mkl.DFTI
• com.intel.mkl.ESSL
• com.intel.mkl.VML
• com.intel.mkl.VSL
Documentation for the particular wrapper and example classes will be generated from the
Java sources while building and running the examples. To browse the documentation, open
the index file in the docs directory (created by the build script):
<mkl directory>\examples\java\docs\index.html .
The Java wrappers for CBLAS, VML, VSL RNG, and FFT establish the interface that directly
corresponds to the underlying native functions, so you can refer to the Intel MKL Reference
Manual for their functionality and parameters. Interfaces for the ESSL-like functions are
described in the generated documentation for the com.intel.mkl.ESSL class.
Each wrapper consists of the interface part for Java and JNI stub written in C. You can find
the sources in the following directory:
<mkl directory>\examples\java\wrappers .
Both Java and C parts of the wrapper for CBLAS and VML demonstrate the straightforward
approach, which you may use to cover additional CBLAS functions.
The wrapper for FFT is more complicated because it needs to support the lifecycle for FFT
descriptor objects. To compute a single Fourier transform, an application needs to call the
FFT software several times with the same copy of the native FFT descriptor. The wrapper
provides the handler class to hold the native descriptor, while the virtual machine runs Java
bytecode.
The wrapper for VSL RNG is similar to the one for FFT. The wrapper provides the handler
class to hold the native descriptor of the stream state.
The wrapper for the convolution and correlation functions mitigates the same difficulty of
the VSL interface, which assumes a similar lifecycle for "task descriptors". The wrapper
utilizes the ESSL-like interface for those functions, which is simpler for the case of
1-dimensional data. The JNI stub additionally encapsulates the MKL functions into the
ESSL-like wrappers written in C and so "packs" the lifecycle of a task descriptor into a
single call to the native method.
The wrappers meet the JNI Specification versions 1.1 and 5.0 and should work with
virtually every modern implementation of Java.
The examples and the Java part of the wrappers are written for the Java language
described in “The Java Language Specification (First Edition)” and extended with the
feature of "inner classes" (this refers to late 1990s). This level of language version is
supported by all versions of the Sun Java Development Kit* (JDK*) developer toolkit and
compatible implementations starting from version 1.1.5, or by all modern versions of Java.

7-15
7 Intel® Math Kernel Library User’s Guide

The level of C language is "Standard C" (that is, C89) with additional assumptions about
integer and floating-point data types required by the Intel MKL interfaces and the JNI
header files. That is, the native float and double data types must be the same as JNI
jfloat and jdouble data types, respectively, and the native int must be 4 bytes long.

Running the Examples

The Java examples support all the C and C++ compilers that the Intel MKL does. The
makefile intended to run the examples also needs the nmake utility, which is typically
provided with the C/C++ compiler package.
To run Java examples, the JDK* developer toolkit is required for compiling and running
Java code. A Java implementation must be installed on the computer or available via the
network. You may download the JDK from the vendor website.
The examples should work for all versions of JDK. However, they were tested only with the
following Java implementations:
• J2SE* SDK 1.4.2, JDK 5.0 and 6.0 from Sun Microsystems, Inc.
(http ://sun .com)
• JRockit* JDK 1.4.2 and 5.0 from BEA Systems, Inc. (http ://bea .com)

NOTE. The implementations from the Sun Microsystems Corporation

supports only processors using the IA-32 and Intel® 64 architectures.
JRockit* JDK from BEA Systems supports Intel® Itanium® 2 processors
as well.

Also note that the Java run-time environment* (JRE*) system, which may be pre-installed
on your computer, is not enough. You need the JDK* developer toolkit that supports the
following set of tools:
• java
• javac
• javah
• javadoc
To make these tools available for the examples makefile, set the JAVA_HOME environment
variable and add the JDK binaries directory to the system PATH, for example:
SET JAVA_HOME=C:\Program Files\Java\jdk1.5.0_09
SET PATH=%JAVA_HOME%\bin;%PATH%
You may also need to clear the JDK_HOME environment variable, if it is assigned a value:
SET JDK_HOME=

7-16
 Language-specific Usage Options 7
To start the examples, use the makefile found in the Intel MKL Java examples directory:
nmake {dll32|dllem64t|dll64|lib32|libem64t|lib64} [function=…]
[compiler=…]
If you type the make command and omit the target (dll32), the makefile prints the help
info, which explains the targets and parameters.
For the examples list, see the examples.lst file in the Java examples directory.

Known Limitations
There are three kinds of limitations:
• Functionality
• Performance
• Known bugs

Functionality. It is possible that some MKL functions will not work if called from the Java
environment by using a wrapper, like those provided with the Intel MKL Java examples.
Only those specific CBLAS, FFT, VML, VSL RNG, and the convolution/correlation functions
listed in the Intel MKL Java Examples section were tested with the Java environment. So,
you may use the Java wrappers for these CBLAS, FFT, VML, VSL RNG, and
convolution/correlation functions in your Java applications.

Performance. The functions from Intel MKL must work faster than similar functions written
in pure Java. However, the main goal of these wrappers is to provide code examples, not
maximum performance. So, an Intel MKL function called from a Java application will
probably work slower than the same function called from a program written in C/C++ or
Fortran.

Known bugs. There are a number of known bugs in Intel MKL (identified in the Release
Notes), as well as incompatibilities between different versions of JDK. The examples and
wrappers include workarounds for these problems, so most of the examples work anyway.
The source code in the examples and wrappers include comments that describe the
workarounds.

7-17
Coding Tips 8
This chapter discusses programming with the Intel® Math Kernel Library (Intel® MKL) to
provide coding tips that meet certain, specific needs, such as numerical stability. Similarly,
Chapter 7 focuses on general language-specific programming options, and Chapter 6
provides tips relevant to performance and memory management.

Aligning Data for Numerical Stability

If linear algebra routines (LAPACK, BLAS) are applied to input data that are bit-for-bit
identical but the arrays are aligned differently or the computations are performed either on
different platforms or with different numbers of threads, the output may not be bit-for-bit
identical, though they will deviate within the appropriate error bounds. The Intel MKL
version may also affect numerical stability of the output, as the routines may be
implemented differently in different versions. With a given Intel MKL version, the outputs
will be bit-for-bit identical provided all the following conditions are met:
• the outputs are obtained on the same platform
• the inputs are bit-for-bit identical
• the input arrays are aligned identically at 16-byte boundaries
• Intel MKL is run in the sequential mode
Unlike the first two conditions, which are under the users' control, the alignment of arrays,
by default, is not. For instance, arrays dynamically allocated using malloc are aligned at
8-byte boundaries, not 16-byte boundaries. If you need numerically identical outputs, use
mkl_malloc() to get the properly aligned workspace, as shown below:

8-1
8 Intel® Math Kernel Library User’s Guide

Example 8-1 Aligning Addresses at 16-byte Boundaries

// ******* C language *******
...
#include <stdlib.h>
...
void *darray;
int workspace;
...
// Allocate workspace aligned on 16-bit boundary
darray = mkl_malloc( sizeof(double)*workspace, 16 );
...
// call the program using MKL
mkl_app( darray );
...
// Free workspace
mkl_free( darray );

! ******* Fortran language *******

...
double precision darray
pointer (p_wrk,darray(1))
integer workspace
...
! Allocate workspace aligned on 16-bit boundary
p_wrk = mkl_malloc( 8*workspace, 16 )
...
! call the program using MKL
call mkl_app( darray )
...
! Free workspace
call mkl_free(p_wrk)

8-2
Working with the Intel®
Math Kernel Library Cluster
Software 9
This chapter discusses the usage of the Intel® Math Kernel Library (Intel® MKL)
ScaLAPACK and Cluster FFTs, mainly describing linking your application with these domains
and including C- and Fortran-specific linking examples. Information on the supported MPI
is also discussed.
The current version of ScaLAPACK and Cluster FFTs contains both static and dynamic
libraries.
See Chapter 3 for details about the Intel MKL directory structure, including the available
documentation in the doc directory.
For information on MP LINPACK Benchmark for Clusters, see Chapter 11.

MPI support
Intel MKL ScaLAPACK and Cluster FFTs support MPI implementations identified in the Intel
MKL Release Notes.
To link applications with ScaLAPACK and Cluster FFTs, additional configuring is needed
depending on your MPI implementation.
If you are using MPICH2, do the following:
1. Add mpich2\include to the include path.
2. Add mpich2\lib to the library path.
3. Add mpi.lib to user's link command.
4. Add fmpich2.lib to user's Fortran link command.
5. Add cxx.lib to user's Release target link command and cxxd.lib to user's Debug
target link command for C++ programs.
If you are using a Microsoft MPI, do the following:

9-1
9 Intel® Math Kernel Library User’s Guide

1. Make certain that Microsoft Compute Cluster Pack\include was added to the
include path.
2. Make certain that Microsoft Compute Cluster Pack\Lib\AMD64 was added to the
library path.
3. Add msmpi.lib to user's link command.
If you are using an Intel MPI, do the following:
1. Make certain that the following string was added to the include path:
%ProgramFiles%\Intel\MPI\<ver>\<arch>\include ,
where <ver> is the directory for a particular MPI version and <arch> is ia32 or
em64t, for example, %ProgramFiles%\Intel\MPI\3.1b\em64t\include.
2. Make certain that the following string was added to the library path:
%ProgramFiles%\Intel\MPI\<ver>\<arch>\lib, for example,
%ProgramFiles%\Intel\MPI\3.1b\em64t\lib.
3. Add impi.lib and impicxx.lib to your link command.
To link a program that calls ScaLAPACK and/or Cluster FFTs, you need to know how to link
an MPI application first. For this information, see manuals on MPICH2, Microsoft Windows
Compute Cluster Pack SDK, and Intel MPI.

Linking with ScaLAPACK and Cluster FFTs

To link with the Intel MKL ScaLAPACK and/or Cluster FFTs, basically use the following
general form:
set lib = <path to libraries from Cluster MKL>;<path to MPI
libraries>;%lib%
<linker> <files to link> <Cluster MKL Libraries> <BLACS> <MKL Core
Libraries> <MPI Libraries>
where:
<linker> is one of icl, ifort, xilink.
<MKL Cluster Library> is one of ScaLAPACK or Cluster FFT libraries for the
appropriate architecture, which are listed in Table 3-5, Table 3-6, and Table 3-7. For
example, for the IA-32 architecture, it is one of mkl_scalapack_core.lib or
mkl_cdft_core.lib.
<BLACS> is one of mkl_blacs_mpich2.lib, mkl_blacs_msmpi.lib,
mkl_blacs_intelmpi.lib in case of static linking or mkl_blacs_dll.lib in case of
dynamic linking.
<MKL Core Libraries> is any Intel MKL library other than ScaLAPACK or Cluster
FFTs.

9-2
 Working with the Intel® Math Kernel Library Cluster Software 9
Note that <BLACS> library should correspond to the MPI version. For instance, if it is
MPICH2 then you have to use mkl_blacs_mpich2.lib.
For information on linking with Intel® MKL libraries, see Linking Your Application with the
Intel® Math Kernel Library.

Setting the Number of Threads

The OpenMP* software responds to the environmental variable OMP_NUM_THREADS. Intel
MKL also has other mechanisms to set the number of threads, such as MKL_NUM_THREADS
or MKL_DOMAIN_NUM_THREADS (see Using Additional Threading Control). Make sure that
the relevant environment variables have the same and correct values on all the nodes. The
Intel MKL versions 10.0 and higher also no longer set the default number of threads to one,
but depend on the OpenMP libraries used with the compiler to set the default number. For
the threading layer based on the Intel® compiler (mkl_intel_thread.lib), this value is
the number of CPUs according to the OS. Be cautious to avoid over-prescribing the number
of threads, which may occur, for instance, when the number of MPI ranks per node and the
number of threads per node are both greater than one.
Considering, for example, the OMP_NUM_THREADS variable, it is recommended that
OMP_NUM_THREADS be set equal to the number of real processors or cores of node. If
Hyper-Threading technology is enabled on the node, it is recommended to use only half
number of the processors that are visible on Windows. The best way to set this variable for
MPICH2 or Intel MPI is to use –env , –genv , –genvlist keys of mpiexec. See the
following examples on how to do it for MPICH2:
mpiexec –genv OMP_NUM_THREADS 2 ….
mpiexec –genvlist OMP_NUM_THREADS ….
mpiexec –n 1 –host first –env OMP_NUM_THREADS 2 test.exe : -n 1 –host
second –env OMP_NUM_THREADS 3 test.exe ….
Using MPICH2, you may, however, have problems with getting global environment, such as
MKL_BLACS_MPI, by the –genvlist key. In this case, set up user or system environments
on each node. Go to the Start menu and select Settings > Control Panel > System >
Advanced > Environment Variables.
All the above ways of setting the number of threads are also applicable for the Microsoft
MPI if the Microsoft SMPD process managers are running in debug mode on all nodes in the
cluster. However, the best way for the Microsoft MPI users is to use the Job Scheduler with
the help of the Microsoft Management Console and/or the Command Line Interface (CLI)
for submitting a job and passing environment variables. For more information about
Microsoft Management Console (MMC) and Command Line Interface (CLI), see the
Microsoft Help and Support page at the Microsoft Web site (http ://www.microsoft.com/).

9-3
9 Intel® Math Kernel Library User’s Guide

Using DLLs
All needed DLLs must be visible on all the nodes at run time and the user should install
Intel MKL on each node of the cluster in this case. Instead of manually installing Intel MKL
on each node you can use Remote Installation Services (RIS) provided by Microsoft to
remotely install the library on each of the compute nodes that are part of your cluster. The
best way to make DLL libraries visible is to point these libraries in PATH environment
variable. You can set the PATH environment variable the same way as OMP_NUM_THREADS.
Note that the ScaLAPACK DLLs for IA-32 and Intel® 64 architectures (em64t\bin and
ia32\bin subdirectories, respectively) use MPI dispatching mechanism. The MPI
dispatching is based on the MKL_BLACS_MPI environment variable. BLACS DLL uses
MKL_BLACS_MPI for choosing the needed MPI libraries. Table 9-1 lists possible values of
the variable.

Table 9-1 Possible Values of MKL_BLACS_MPI

Value Comment
MPICH2 Default value. MPICH2 1.0.x for Microsoft Windows will be
used for message passing
MSMPI Microsoft MPI will be used for message passing
INTELMPI Intel MPI will be used for message passing

Note also that if non-default MPI is used, MKL_BLACS_MPI should be assigned the same
appropriate value on all nodes. You can set MKL_BLACS_MPI environment variable in the
same way as setting PATH, OMP_NUM_THREADS, and other environment variables but the
best and simplest way for passing the value of MKL_BLACS_MPI to all nodes is to use –env,
–genv, –genvlist keys for mpiexec. See the following examples on how to do it for
INTELMPI:
mpiexec –genv MKL_BLACS_MPI INTELMPI ….
mpiexec –genvlist MKL_BLACS_MPI ….
mpiexec –n 1 –host first –env MKL_BLACS_MPI INTELMPI test.exe : -n 1
–host second –env MKL_BLACS_MPI INTELMPI test.exe .

ScaLAPACK Tests
To build NetLib ScaLAPACK tests:
• For the IA-32 architecture, add mkl_scalapack_core.lib to your link command.
• For the IA-64 and Intel® 64 architectures, add mkl_scalapack_lp64.lib or
mkl_scalapack_ilp64.lib, depending upon the desired interface.

9-4
 Working with the Intel® Math Kernel Library Cluster Software 9
Examples for Linking with ScaLAPACK and Cluster FFT
For the detailed information on the structure of the Intel MKL architecture-specific
directories and the names of the cluster libraries to link, see Directory Structure in Detail.

Examples for Linking a C Application

Suppose the following conditions are met:
• MPICH2 1.0.x installed in c:\mpich2x64.
• You use the Intel® C++ Compiler 9.1 or higher and the main module is in C.

To link with ScaLAPACK using LP64 interface for a cluster of Intel® 64 architecture based
systems, use the following:
set lib=c:\mpich2x64\lib;<mkl_directory>\em64t\lib;%lib%
icl <user files to link> mkl_scalapack_lp64.lib mkl_blacs_mpich2_lp64.lib
mkl_intel_lp64.lib mkl_intel_thread.lib mkl_core.lib libiomp5md.lib
mpi.lib cxx.lib bufferoverflowu.lib

To link with Cluster FFT using LP64 interface for a cluster of Intel® 64 architecture based
systems, use the following:
set lib=c:\mpich2x64\lib;<mkl_directory>\em64t\lib;%lib%
icl <user files to link> mkl_cdft_core.lib mkl_blacs_mpich2_lp64.lib
mkl_intel_lp64.lib mkl_intel_thread.lib mkl_core.lib libiomp5md.lib
mpi.lib cxx.lib bufferoverflowu.lib

Examples for Linking a Fortran Application

Suppose the following conditions are met:
• Microsoft Windows Compute Cluster Pack SDK installed in c:\MS CCP SDK.
• You use the Intel® Fortran Compiler 9.1 or higher and the main module is in Fortran.

To link with ScaLAPACK using LP64 interface for a cluster of Intel® 64 architecture based
systems, use the following:
set lib="c:\MS CCP SDK\Lib\AMD64";<mkl_directory>\em64t\lib;%lib%
ifort <user files to link> mkl_scalapack_lp64.lib mkl_blacs_mpich2_lp64.lib
mkl_lapack.lib mkl_intel_lp64.lib mkl_intel_thread.lib mkl_core.lib
libiomp5md.lib msmpi.lib bufferoverflowu.lib

9-5
9 Intel® Math Kernel Library User’s Guide

To link with Cluster FFTs using LP64 interface for a cluster of Intel® 64 architecture based
systems, use the following:
set lib="c:\MS CCP SDK\Lib\AMD64";<mkl_directory>\em64t\lib;%lib%
ifort <user files to link> mkl_cdft_core.lib mkl_blacs_mpich2_lp64.lib
mkl_intel_lp64.lib mkl_intel_thread.lib mkl_core.lib libiomp5md.lib
msmpi.lib bufferoverflowu.lib
A binary linked with ScaLAPACK runs in the same way as any other MPI application (refer
to the documentation that comes with the MPI implementation).
For further linking examples, see the support website for Intel products at
http://www.intel.com/software/products/support/.

9-6
Getting Assistance for
Programming in the
Microsoft Visual Studio* IDE 10
This chapter discusses features of the Intel® Math Kernel Library (Intel® MKL) which
software engineers can benefit from when working in the Microsoft Visual Studio* IDE.
The following features assist programming in Visual Studio .Net 2003, Visual Studio (VS)
2005, and 2008 (Orcas):
• The Intel MKL documentation viewable from within the IDE
• Context-sensitive help in the VS Code Editor
• Microsoft Intellisense* capability

Viewing Intel® MKL Documentation in Visual Studio

To get access to the Intel MKL documentation in the Visual Studio IDE (VS),
1. Select Help>Contents from the menu.
This displays the list of VS Help collections.
2. Click Intel Math Kernel Library Help
3. In the Help tree that expands, click Intel MKL Reference Manual (see Figure 10-1).
The Intel MKL documentation index is incorporated in the VS Help index. To open the
index, select Help>Index from the menu. The library documentation is also included in
the VS Help search. To open the search page, select Help>Search from the menu.

10-1
10 Intel® Math Kernel Library User’s Guide

Figure 10-1 Intel® MKL Help in the Microsoft Visual Studio* IDE

You can filter Visual Studio Help collections so that only collections from Intel are shown. To
do this, select "Intel" from the Filtered by list (see Figure 10-2). This hides the contents
and index entries for all collections that do not refer to Intel.

10-2
 Getting Assistance for Programming in the Microsoft Visual Studio* IDE 10
Figure 10-2 Help Collections filtered by the "Intel" filter

Using Context-Sensitive Help

When typing your code in the VS Code Editor, you can get context-sensitive help using the
F1 Help and Dynamic Help features.

NOTE. To use F1 Help in Microsoft Visual Studio .Net 2003, install

Microsoft Visual Studio .Net 2003 Help files.

F1 Help
F1 Help opens the Help topic relevant to the current selection upon pressing F1.

10-3
10 Intel® Math Kernel Library User’s Guide

In particular, to open the Intel MKL Help topic describing the Intel MKL function called in
your code, select the function name and press F1. The topic with the function description
opens in the window that displays search results (see Figure 10-3).

Figure 10-3 Context-Sensitive Help for Intel® MKL in the Microsoft Visual Studio* IDE

Dynamic Help
Dynamic Help also provides access to topics relevant to the current selection or to the text
being typed. Links to all relevant topics are displayed in the Dynamic Help window.

10-4
 Getting Assistance for Programming in the Microsoft Visual Studio* IDE 10
To get the list of relevant topics each time you select the Intel MKL function name or as you
type it in your code, open the Dynamic Help window by selecting Help>Dynamic Help
from the menu.
To open a topic from the list, click the appropriate link in the Dynamic Help window (see
Figure 10-3). Typically only one link corresponds to each Intel MKL function.

NOTE. All the VS windows in Figure 10-3 are floating, whereas the
default windows layout is different.

Using the IntelliSense* Capability

IntelliSense is a set of native VS features that make language references easily accessible.
The user programming with Intel MKL in the VS Code Editor can employ two IntelliSense
features: Parameter Info and Complete Word.

NOTE. Both features use header files. Therefore, to benefit from

IntelliSense, make sure the path to the include files is specified in the VS
or solution settings. For example, see Configuring the Microsoft Visual
C/C++* Development System to Link with Intel® MKL on how to do this.

Parameter Info
The Parameter Info feature displays the parameter list for a function to give information on
the number and types of parameters.
Provided your C/C++ code contains the include statement with the appropriate Intel MKL
header file, to get the list of parameters of a function specified in the header file,
1. Type the function name.
2. Type the opening parenthesis.
This makes the tooltip with the list of the function parameters pop up (see Figure 10-4).

10-5
10 Intel® Math Kernel Library User’s Guide

Figure 10-4 IntelliSense* Parameter Info

Complete Word
For a software library, the Complete Word feature types or prompts for the rest of the
name defined in the header file once the first few characters of the name are typed in your
code.
Provided your C/C++ code contains the include statement with the appropriate Intel MKL
header file, to complete the name of the function or named constant specified in the
header file,

10-6
 Getting Assistance for Programming in the Microsoft Visual Studio* IDE 10
1. Type the first few characters of the name.
2. Press Alt+RIGHT ARROW or Ctrl+SPACEBAR.
If you have typed enough characters to disambiguate the name, the rest of the name
is typed automatically. Otherwise, the pop-up list of the names specified in the header
file opens (see Figure 10-5).
3. Select the name from the list, if needed.

Figure 10-5 IntelliSense* Complete Word

10-7
LINPACK and MP LINPACK
Benchmarks 11
This chapter describes the Intel® Optimized LINPACK Benchmark for the Windows* OS (for
shared memory systems) and Intel® Optimized MP LINPACK Benchmark for Clusters (for
distributed memory systems).

Intel® Optimized LINPACK Benchmark for Windows OS*

Intel® Optimized LINPACK Benchmark is a generalization of the LINPACK 1000 benchmark.
It solves a dense (real*8) system of linear equations (Ax=b), measures the amount of
time it takes to factor and solve the system, converts that time into a performance rate,
and tests the results for accuracy. The generalization is in the number of equations (N) it
can solve, which is not limited to 1000. It uses partial pivoting to assure the accuracy of
the results.
This benchmark should not be used to report LINPACK 100 performance, as that is a
compiled-code only benchmark. This is a shared memory (SMP) implementation which
runs on a single platform. Do not confuse this benchmark with:
• MP LINPACK, which is a distributed memory version of the same benchmark.
• LINPACK, the library, which has been expanded upon by the LAPACK library.
Intel provides optimized versions of the LINPACK benchmarks to help you obtain high
LINPACK benchmark results on your systems based on genuine Intel® processors more
easily than with the High Performance Linpack (HPL) benchmark. Use this package to
benchmark your SMP machine.
Additional information on this software as well as other Intel® software performance
products is available at http://www.intel.com/software/products/.

Contents
The Intel Optimized LINPACK Benchmark for Windows* contains the following files, located
in the .\benchmarks\linpack\ subdirectory in the Intel MKL directory (see Table 3-1):

11-1
11 Intel® Math Kernel Library User’s Guide

Table 11-1 Contents of the LINPACK Benchmark

.\benchmarks\linpack\
linpack_itanium.exe The 64-bit program executable for a system based on Intel®
Itanium® 2 processor.
linpack_xeon32.exe The 32-bit program executable for a system based on Intel®
Xeon® processor or Intel® Xeon® processor MP with or without
Streaming SIMD Extensions 3 (SSE3).
linpack_xeon64.exe The 64-bit program executable for a system with Intel® Xeon®
processor using Intel® 64 architecture.
runme_itanium.bat A sample shell script for executing a pre-determined problem set
for linpack_itanium.exe. OMP_NUM_THREADS set to 8
processors.
runme_xeon32.bat A sample shell script for executing a pre-determined problem set
for linpack_xeon32.exe. OMP_NUM_THREADS set to 2
processors.
runme_xeon64.bat A sample shell script for executing a pre-determined problem set
for linpack_xeon64.exe. OMP_NUM_THREADS set to 4
processors.
lininput_itanium Input file for pre-determined problem for the runme_itanium
script.
lininput_xeon32 Input file for pre-determined problem for the runme_xeon32
script.
lininput_xeon64 Input file for pre-determined problem for the runme_xeon64
script.
win_itanium.txt Result of the runme_itanium script execution.

win_xeon32.txt Result of the runme_xeon32 script execution.

win_xeon64.txt Result of the runme_xeon64 script execution.

help.lpk Simple help file.

xhelp.lpk Extended help file.

Running the Software

To obtain results for the pre-determined sample problem sizes on a given system, type one
of the following, as appropriate:
runme_itanium.bat
runme_xeon32.bat
runme_xeon64.bat

11-2
 LINPACK and MP LINPACK Benchmarks 11
To run the software for other problem sizes, please refer to the extended help included with
the program. Extended help can be viewed by running the program executable with the -e
option:
linpack_itanium.exe -e
linpack_xeon32.exe -e
linpack_xeon64.exe -e .
The pre-defined data input files lininput_itanium, lininput_xeon32, and
lininput_xeon64 are provided merely as examples. Different systems have different
number of processors or amount of memory and thus require new input files. The extended
help can be used for insight into proper ways to change the sample input files.
Each input file requires at least the following amount of memory:
lininput_itanium 16 GB
lininput_xeon32 2 GB
lininput_xeon64 16 GB
If the system has less memory than the above sample data input requires, you may need
to edit or create your own data input files, as explained in the extended help.
Each sample script, in particular, uses the OMP_NUM_THREADS environment variable to set
the number of processors it is targeting. To optimize performance on a different number of
physical processors, change that line appropriately. If you run the Intel Optimized LINPACK
Benchmark without setting the number of threads, it will default to the number of cores
according to the OS. You can find the settings for this environment variable in the runme_*
sample scripts. If the settings do not already match the situation for your machine, edit the
script.

Known Limitations
The following limitations are known for the Intel Optimized LINPACK Benchmark for
Windows*:
• Intel Optimized LINPACK Benchmark is threaded to effectively use multiple processors.
So, in multi-processor systems, best performance will be obtained with
Hyper-Threading technology turned off, which ensures that the operating system
assigns threads to physical processors only.
• If an incomplete data input file is given, the binaries may either hang or fault. See the
sample data input files and/or the extended help for insight into creating a correct data
input file.

11-3
11 Intel® Math Kernel Library User’s Guide

Intel® Optimized MP LINPACK Benchmark for Clusters

The Intel® Optimized MP LINPACK Benchmark for Clusters is based on modifications and
additions to HPL 2.0 from Innovative Computing Laboratories (ICL) at the University of
Tennessee, Knoxville (UTK). The Intel Optimized MP LINPACK Benchmark for Clusters can be
used for Top 500 runs (see http ://www .top500 .org). To use the benchmark you need be
intimately familiar with the HPL distribution and usage. The Intel Optimized MP LINPACK
Benchmark for Clusters provides some additional enhancements and bug fixes designed to
make the HPL usage more convenient, as well as explain Intel® MPI settings that may
enhance performance. The .\benchmarks\mp_linpack directory adds techniques to
minimize search times frequently associated with long runs.
The Intel® Optimized MP LINPACK Benchmark for Clusters is an implementation of the
Massively Parallel MP LINPACK benchmark by means of HPL code. It solves a random dense
(real*8) system of linear equations (Ax=b), measures the amount of time it takes to
factor and solve the system, converts that time into a performance rate, and tests the
results for accuracy. You can solve any size (N) system of equations that fit into memory.
The benchmark uses full row pivoting to ensure the accuracy of the results.
Use the Intel Optimized MP LINPACK Benchmark for Clusters on a distributed memory
machine. On a shared memory machine, use the Intel Optimized LINPACK Benchmark.
Intel provides optimized versions of the LINPACK benchmarks to help you obtain high
LINPACK benchmark results on your systems based on genuine Intel® processors more
easily than with the HPL benchmark. Use the Intel® Optimized MP LINPACK Benchmark to
benchmark your cluster. The prebuilt binaries require that you first install Intel® MPI 3.x
be installed on the cluster. The run-time version of Intel MPI is free and can be downloaded
from www.intel.com/software/products/cluster.
The Intel package includes software developed at the University of Tennessee, Knoxville,
Innovative Computing Laboratories and neither the University nor ICL endorse or promote
this product. Although HPL 2.0 is redistributable under certain conditions, this particular
package is subject to the Intel MKL license.
Intel MKL has introduced a new functionality into MP LINPACK, which is called a hybrid
build, while continuing to support the older version. The term “hybrid” refers to special
optimizations added to take advantage of mixed OpenMP*/MPI parallelism. If you want to
use one MPI process per node and to achieve further parallelism by means of OpenMP, use
the hybrid build. In general, the hybrid build is useful when the number of MPI processes
per core is less than one. If you want to rely exclusively on MPI for parallelism and use one
MPI per core, use the non-hybrid build.

NOTE. The hybrid version of MP LINPACK is available only for IA-32 and
Intel® 64 architectures.

11-4
 LINPACK and MP LINPACK Benchmarks 11
In addition to supplying certain hybrid prebuilt binaries, Intel MKL supplies some hybrid
prebuilt libraries to take advantage of the additional OpenMP* optimizations.

NOTE. You can use the prebuilt hybrid libraries only with Intel® MPI.

If you wish to use an MPI version other than Intel MPI, you can do so by using the MP
LINPACK source provided. You can use the source to build a non-hybrid version that may
be used in a hybrid mode, but it would be missing some of the optimizations added to the
hybrid version.
Non-hybrid builds are the default of the source code makefiles provided. In some cases,
the use of the hybrid mode is required for external reasons. If there is a choice, the
non-hybrid code may be faster, although that may change in future releases. To use the
non-hybrid code in a hybrid mode, use the threaded version of Intel MKL BLAS, link with a
thread-safe MPI, and call function MPI_init_thread() so as to indicate a need for MPI to
be thread-safe.

Contents
The Intel Optimized MP LINPACK Benchmark for Clusters (MP LINPACK Benchmark)
includes the HPL 2.0 distribution in its entirety as well as the modifications, delivered in the
files listed in Table 11-2 and located in the .\benchmarks\mp_linpack\ subdirectory in
the Intel MKL directory (see Table 3-1):

Table 11-2 Contents of the MP LINPACK Benchmark

.\benchmarks\mp_linpack\
testing\ptest\HPL_pdtest.c HPL 2.0 code modified to display captured DGEMM
information in ASYOUGO2_DISPLAY (see details in
the New Features section) if it was captured.
src\blas\HPL_dgemm.c HPL 2.0 code modified to capture DGEMM information
if desired from ASYOUGO2_DISPLAY.
src\grid\HPL_grid_init.c HPL 2.0 code modified to do additional grid
experiments originally not in HPL 2.0.
src\pgesv\HPL_pdgesvK2.c HPL 2.0 code modified to do ASYOUGO and
ENDEARLY modifications.
src\pgesv\HPL_pdgesv0.c HPL 2.0 code modified to do ASYOUGO, ASYOUGO2,
and ENDEARLY modifications.
testing\ptest\HPL.dat HPL 2.0 sample HPL.dat modified.

11-5
11 Intel® Math Kernel Library User’s Guide

Table 11-2 Contents of the MP LINPACK Benchmark

.\benchmarks\mp_linpack\
makes All the makefiles in this directory have been rebuilt in
the Windows distribution.
testing\ptimer\ Some files in here have been modified in the Windows
distribution.
testing\timer\ Some files in here have been modified in the Windows
distribution.
Make (New) Sample architecture makefile for nmake utility
to be used on processors based on the IA-32, Intel®
64, and IA-64 architectures and Windows.
bin_intel\ia32\xhpl_ia32.exe (New) Prebuilt binary for the IA-32 architecture,
Windows* OS, and Intel® MPI.
bin_intel\em64t\xhpl_em64t.exe (New) Prebuilt binary for the Intel® 64 architecture,
Windows* OS, and Intel MPI.
lib_hybrid\ia32\libhpl_ (New) Prebuilt library with the hybrid version of MP
hybrid.lib LINPACK for the IA-32 architecture and Intel MPI.

lib_hybrid\em64t\libhpl_ (New) Prebuilt library with the hybrid version of MP

hybrid.lib LINPACK for the Intel® 64 architecture and Intel MPI.

bin_intel\ia32\xhpl_hybrid_ (New) Prebuilt hybrid binary for the IA-32

ia32.exe architecture, Windows* OS, and Intel MPI.

bin_intel\em64t\xhpl_hybrid_ (New) Prebuilt hybrid binary for the Intel® 64

em64t.exe architecture, Windows* OS, and Intel MPI.

nodeperf.c (New) Sample utility that tests the DGEMM speed

across the cluster.

Building the MP LINPACK

There are a few included sample architecture makefiles. It is recommended that you edit
them to fit your specific configuration. In particular:
• Set TOPdir to the directory that MP LINPACK is being built in.
• Set MPI variables, that is, MPdir, MPinc, and MPlib.
• Specify the location of Intel MKL and of files to be used (LAdir, LAinc, LAlib).
• Adjust compiler and compiler/linker options.
• Specify the version of MP LINPACK you are going to build (hybrid or non-hybrid) by
setting the version parameter for the nmake command, for example:
nmake arch=em64t mpi=intelmpi version=hybrid install .

11-6
 LINPACK and MP LINPACK Benchmarks 11
For some sample cases, the makefiles contain values that must be common. However, you
need to be familiar with building an HPL and picking appropriate values for these variables.

New Features
The toolset is basically identical with the HPL 2.0 distribution. There are a few changes that
are optionally compiled in and disabled until you specifically request them. These new
features are:

ASYOUGO: Provides non-intrusive performance information while runs proceed. There are
only a few outputs and this information does not impact performance. This is especially
useful because many runs can go hours without any information.

ASYOUGO2: Provides slightly intrusive additional performance information because it

intercepts every DGEMM.

ASYOUGO2_DISPLAY: Displays the performance of all the significant DGEMMs inside the run.

ENDEARLY: Displays a few performance hints and then terminates the run early.

FASTSWAP: Inserts the LAPACK-optimized DLASWP into HPL's code. This may yield a benefit
for Itanium® 2 processor. You can experiment with this to determine best results.

HYBRID: Establishes the Hybrid OpenMP/MPI mode of MP LINPACK, providing the possibility
to use threaded Intel MKL and prebuilt MP LINPACK hybrid libraries.

WARNING. Use this option only with an Intel compiler and the Intel®
MPI library version 3.1 or higher. You are also recommended to use the
compiler version 10.0 or higher.

Benchmarking a Cluster
To benchmark a cluster, follow the sequence of steps below (some of them are optional).
Pay special attention to the iterative steps 3 and 4. They make a loop that searches for HPL
parameters (specified in HPL.dat) that enable you to reach the top performance of your
cluster.

11-7
11 Intel® Math Kernel Library User’s Guide

1. Install HPL and make sure HPL is functional on all the nodes.
2. You may run nodeperf.c (included in the distribution) to see the performance of
DGEMM on all the nodes.
Compile nodeperf.c with your MPI and Intel MKL. For example:
icl /Za /O3 /w /D_WIN_ /I"<Home directory of MPI>\include" "<Home
directory of MPI libraries>\<MPI librariy>"
"<mkl directory>\em64t\lib\mkl_core.lib"
"<mkl directory>\em64t\lib\libiomp5md.lib" nodeperf.c ,
where <MPI library> is msmpi.lib in the case of Microsoft* MPI and mpi.lib in the
case of MPICH.
Launching nodeperf.c on all the nodes is especially helpful in a very large cluster.
Indeed, there may be a stray job on a certain node, for example, 738, which is running
5% slower than the rest. MP LINPACK will then run as slow as the slowest node. In this
case, nodeperf enables quick identification of the potential problem spot without
numerous small MP LINPACK runs around the cluster in search of the bad node. It is
common that after a bunch of HPL runs, there may be zombie processes and
nodeperf facilitates finding the slow nodes. It goes through all the nodes, one at a
time, and reports the performance of DGEMM followed by some host identifier.
Therefore, the higher the DGEMM performance, the faster that node was performing.
3. Edit HPL.dat to fit your cluster needs.
Read through the HPL documentation for ideas on this. However, you should try on at
least 4 nodes.
4. Make an HPL run, using compile options such as ASYOUGO or ASYOUGO2 or ENDEARLY to
aid in your search (These options enable you to gain insight into the performance
sooner than HPL would normally give this insight.)
When doing so, follow these recommendations:
— Use MP LINPACK, which is a patched version of HPL, to save time in the search.
Using a patched version of HPL should not hinder your performance. That’s why
features that could be performance intrusive are compile-optional (and it is
explained below) in MP LINPACK. That is, if you do not use the new options
explained in section Options to Reduce Search Time, then these changes are
disabled. The primary purpose of the additions is to assist you in finding solutions.
HPL requires a long time to search for many different parameters. In the MP
LINPACK, the goal is to get the best possible number.
Given that the input is not fixed, there is a large parameter space you must search
over. In fact, an exhaustive search of all possible inputs is improbably large even
for a powerful cluster.
This patched version of HPL optionally prints information on performance as it
proceeds, or even terminates early depending on your desires.

11-8
 LINPACK and MP LINPACK Benchmarks 11
— Save time by compiling with -DENDEARLY -DASYOUGO2 (described in the Options
to Reduce Search Time section) and using a negative threshold (do not to use a
negative threshold on the final run that you intend to submit as a Top500 entry).
You can set the threshold in line 13 of the HPL 2.0 input file HPL.dat.
— If you are going to run a problem to completion, do it with -DASYOUGO (see
Options to Reduce Search Time).
5. Using the quick performance feedback, return to step 3 and iterate until you are sure
that the performance is as good as possible.

Options to Reduce Search Time

Running huge problems to completion on large numbers of nodes can take many hours.
The search space for MP LINPACK is also huge: not only can you run any size problem, but
over a number of block sizes, grid layouts, lookahead steps, using different factorization
methods, etc. It can be a large waste of time to run a huge problem to completion only to
discover it ran 0.01% slower than your previous best problem.
There are 3 options you might want to experiment with to reduce the search time:
• -DASYOUGO
• -DENDEARLY
• -DASYOUGO2
Use -DASYOUGO2 cautiously because it does have a marginal performance impact. To
see DGEMM internal performance, compile with -DASYOUGO2 and
-DASYOUGO2_DISPLAY. These options provide a lot of useful DGEMM performance
information at the cost of around 0.2% performance loss.
If you want to use the old HPL, simply omit these options and recompile from scratch (try
"nmake arch=<arch> clean_arch_all").

-DASYOUGO: Gives performance data as the run proceeds. The performance always starts
off higher and then drops because this actually happens in LU decomposition. The ASYOUGO
performance estimate is usually an overestimate (because LU slows down as it goes), but it
gets more accurate as the problem proceeds. The greater the lookahead step, the less
accurate the first number may be. ASYOUGO tries to estimate where one is in the LU
decomposition that MP LINPACK performs and this is always an overestimate as compared
to ASYOUGO2, which measures actually achieved DGEMM performance. Note that the
ASYOUGO output is a subset of the information that ASYOUGO2 provides. So, refer to the
description of the -DASYOUGO2 option below for the details of the output.

-DENDEARLY: Terminates the problem after a few steps, so that you can set up 10 or 20
HPL runs without monitoring them, see how they all do, and then only run the fastest ones
to completion. -DENDEARLY assumes -DASYOUGO. You do not need to define both, although
it doesn't hurt. Because the problem terminates early, it is recommended setting the

11-9
11 Intel® Math Kernel Library User’s Guide

"threshold" parameter in HPL.dat to a negative number when testing ENDEARLY. There is

no point in doing a residual check if the problem ended early. It also sometimes gives a
better picture to compile with -DASYOUGO2 when using -DENDEARLY.
Usage notes on -DENDEARLY follow:
— -DENDEARLY stops the problem after a few iterations of DGEMM on the blocksize
(the bigger the blocksize, the further it gets). It prints only 5 or 6 "updates",
whereas -DASYOUGO prints about 46 or so output elements before the problem
completes.
— Performance for -DASYOUGO and -DENDEARLY always starts off at one speed,
slowly increases, and then slows down toward the end (because that is what LU
does). -DENDEARLY is likely to terminate before it starts to slow down.
— -DENDEARLY terminates the problem early with an HPL Error exit. It means that
you need to ignore the missing residual results, which are wrong, as the problem
never completed. However, you can get an idea what the initial performance was,
and if it looks good, then run the problem to completion without -DENDEARLY. To
avoid the error check, you can set HPL's threshold parameter in HPL.dat to a
negative number.
— Though -DENDEARLY terminates early, HPL treats the problem as completed and
computes Gflop rating as though the problem ran to completion. Ignore this
erroneously high rating.
— The bigger the problem, the more accurately the last update that -DENDEARLY
returns will be close to what happens when the problem runs to completion.
-DENDEARLY is a poor approximation for small problems. It is for this reason that
you are suggested to use ENDEARLY in conjunction with ASYOUGO2, because
ASYOUGO2 reports actual DGEMM performance, which can be a closer
approximation to problems just starting.
The best known compile options for Itanium® 2 processor are with the Intel®
compiler:
-O2 -ipo -ipo_obj -ftz -IPF_fltacc -IPF_fma -unroll -w -tpp2

-DASYOUGO2: Gives detailed single-node DGEMM performance information. It captures all

DGEMM calls (if you use Fortran BLAS) and records their data. Because of this, the routine
has a marginal intrusive overhead. Unlike -DASYOUGO, which is quite non-intrusive,
-DASYOUGO2 is interrupting every DGEMM call to monitor its performance. You should
beware of this overhead, although for big problems, it is, for sure, less than 1/10th of a
percent.
Here is a sample ASYOUGO2 output (the first 3 non-intrusive numbers can be found in
ASYOUGO and ENDEARLY), so it suffices to describe these numbers here:
Col=001280 Fract=0.050 Mflops=42454.99 (DT= 9.5 DF= 34.1
DMF=38322.78).

11-10
 LINPACK and MP LINPACK Benchmarks 11
The problem size was N=16000 with a blocksize of 128. After 10 blocks, that is, 1280
columns, an output was sent to the screen. Here, the fraction of columns completed is
1280/16000=0.08. Only up to 40 outputs are printed, at various places through the
matrix decomposition: fractions
0.005 0.010 0.015 0.020 0.025 0.030 0.035 0.040 0.045 0.050 0.055 0.060 0.065
0.070 0.075 0.080 0.085 0.090 0.095 0.100 0.105 0.110 0.115 0.120 0.125 0.130
0.135 0.140 0.145 0.150 0.155 0.160 0.165 0.170 0.175 0.180 0.185 0.190 0.195
0.200 0.205 0.210 0.215 0.220 0.225 0.230 0.235 0.240 0.245 0.250 0.255 0.260
0.265 0.270 0.275 0.280 0.285 0.290 0.295 0.300 0.305 0.310 0.315 0.320 0.325
0.330 0.335 0.340 0.345 0.350 0.355 0.360 0.365 0.370 0.375 0.380 0.385 0.390
0.395 0.400 0.405 0.410 0.415 0.420 0.425 0.430 0.435 0.440 0.445 0.450 0.455
0.460 0.465 0.470 0.475 0.480 0.485 0.490 0.495 0.515 0.535 0.555 0.575 0.595
0.615 0.635 0.655 0.675 0.695 0.795 0.895.
However, this problem size is so small and the block size so big by comparison that as
soon as it prints the value for 0.045, it was already through 0.08 fraction of the
columns. On a really big problem, the fractional number will be more accurate. It
never prints more than the 112 numbers above. So, smaller problems will have fewer
than 112 updates, and the biggest problems will have precisely 112 updates.
The Mflops is an estimate based on 1280 columns of LU being completed. However,
with lookahead steps, sometimes that work is not actually completed when the output
is made. Nevertheless, this is a good estimate for comparing identical runs.
The 3 numbers in parenthesis are intrusive ASYOUGO2 addins. The DT is the total time
processor 0 has spent in DGEMM. The DF is the number of billion operations that have
been performed in DGEMM by one processor. Hence, the performance of processor 0 (in
Gflops) in DGEMM is always DF/DT. Using the number of DGEMM flops as a basis instead
of the number of LU flops, you get a lower bound on performance of our run by looking
at DMF, which can be compared to Mflops above (It uses the global LU time, but the
DGEMM flops are computed under the assumption that the problem is evenly distributed
amongst the nodes, as only HPL’s node (0,0) returns any output.)
Note that when using the above performance monitoring tools to compare different
HPL.dat input data sets, you should beware that the pattern of performance drop off that
LU experiences is sensitive to some input data. For instance, when you try very small
problems, the performance drop off from the initial values to end values is very rapid. The
larger the problem, the less the drop off, and it is probably safe to use the first few
performance values to estimate the difference between a problem size 700000 and
701000, for instance. Another factor that influences the performance drop off is the grid
dimensions (P and Q). For big problems, the performance tends to fall off less from the first
few steps when P and Q are roughly equal in value. You can make use of a large number of
parameters, such as broadcast types, and change them so that the final performance is
determined very closely by the first few steps.
Using these tools will greatly assist the amount of data you can test.

11-11
Intel® Math Kernel Library
Language Interfaces
Support A
Table A-1 shows language interfaces that Intel® Math Kernel Library (Intel® MKL)
provides for each function domain, and Table A-2 lists the respective header files. However,
Intel MKL routines can be called from other languages using mixed-language
programming. For example, see Mixed-language Programming with Intel® MKL on how to
call Fortran routines from C/C++.

Table A-1 Language Interfaces Support

FORTRAN 77 Fortran 90/95 C/C++
Function Domain interface interface interface
Basic Linear Algebra Subprograms (BLAS) Yes Yes via CBLAS
BLAS-like extension transposition routines Yes Yes
Sparse BLAS Level 1 Yes Yes via CBLAS
Sparse BLAS Level 2 and 3 Yes Yes Yes
LAPACK routines for solving systems of linear Yes Yes †
equations
LAPACK routines for solving least-squares Yes Yes †
problems, eigenvalue and singular value
problems, and Sylvester's equations
Auxiliary and utility LAPACK routines Yes †
Parallel Basic Linear Algebra Subprograms Yes
(PBLAS)
ScaLAPACK routines Yes †
DSS/PARDISO* solvers Yes Yes Yes
Other Direct and Iterative Sparse Solver Yes Yes Yes
routines
Vector Mathematical Library (VML) functions Yes Yes Yes
Vector Statistical Library (VSL) functions Yes Yes Yes
Fourier Transform functions (FFT) Yes Yes
Cluster FFT functions Yes Yes

A-1
A Intel® Math Kernel Library User’s Guide

Table A-1 Language Interfaces Support (continued)

FORTRAN 77 Fortran 90/95 C/C++
Function Domain interface interface interface
Trigonometric Transform routines Yes Yes
Fast Poisson, Laplace, and Helmholtz Solver Yes Yes
(Poisson Library) routines
Optimization (Trust-Region) Solver routines Yes Yes Yes
GMP* arithmetic functions Yes
Service routines (including memory Yes
allocation)

† Supported using a mixed language programming call. See Table A-2 for the respective header file.

Table A-2 lists available header files for all Intel MKL function domains.

Table A-2 Include Files

Function domain Include files
Fortran C or C++
All function domains mkl.fi mkl.h
BLAS Routines blas.f90 mkl_blas.h
mkl_blas.fi
BLAS-like Extension mkl_trans.fi mkl_trans.h
Transposition Routines
CBLAS Interface to BLAS mkl_cblas.h
Sparse BLAS Routines mkl_spblas.fi mkl_spblas.h
LAPACK Routines lapack.f90 mkl_lapack.h
mkl_lapack.fi
ScaLAPACK Routines mkl_scalapack.h
All Sparse Solver Routines mkl_solver.f90 mkl_solver.h
• PARDISO mkl_pardiso.f77 mkl_pardiso.h
mkl_pardiso.f90
• DSS Interface mkl_dss.f77 mkl_dss.h
mkl_dss.f90
• RCI Iterative Solvers mkl_rci.fi mkl_rci.h
• ILU Factorization
Optimization Solver Routines mkl_rci.fi mkl_rci.h
Vector Mathematical Functions mkl_vml.f77 mkl_vml.h
mkl_vml.fi

A-2
 Intel® Math Kernel Library Language Interfaces Support A
Table A-2 Include Files (continued)
Function domain Include files
Fortran C or C++
Vector Statistical Functions mkl_vml.f77 mkl_vsl.h
mkl_vsl.fi
Fourier Transform Functions mkl_dfti.f90 mkl_dfti.h
Cluster Fourier Transform mkl_cdft.f90 mkl_cdft.h
Functions
Partial Differential Equations
Support Routines
• Trigonometric Transforms mkl_trig_transforms.f90 mkl_trig_transforms.h
• Poisson Solvers mkl_poisson.f90 mkl_poisson.h
GMP interface mkl_gmp.h
Service routines mkl_service.h
Memory allocation routines i_malloc.h
MKL examples interface mkl_example.h

A-3
Support for Third-Party
Interfaces B
This appendix describes in brief certain interfaces that Intel® Math Kernel Library (Intel®
MKL) supports.

GMP* Functions
Intel MKL implementation of GMP* arithmetic functions includes arbitrary precision
arithmetic operations on integer numbers. The interfaces of such functions fully match the
GNU Multiple Precision* (GMP) Arithmetic Library. For specifications of these functions,
please see http://www.intel.com/software/products/mkl/docs/gnump/WebHelp/.
If you currently use the GMP* library, you need to modify INCLUDE statements in your
programs to mkl_gmp.h.

FFTW Interface Support

Intel MKL offers two collections of wrappers being the FFTW interface (www.fftw.org)
superstructure to be used for calling the Intel MKL Fourier transform functions. These
collections correspond to the FFTW versions 2.x and 3.x and the Intel MKL versions 7.0 and
later.
These wrappers enable developers whose programs currently use FFTW to gain
performance with the Intel MKL Fourier transforms without changing the program source
code. See the "FFTW to Intel® MKL Wrappers" appendix in the Intel MKL Reference Manual
for details on the use of the wrappers.

B-1
Index
A mixed-language calls, 7-8
Advanced Vector Extensions, dispatching the techniques to improve performance, 6-14
instructions, 6-13 Compaq* Visual Fortran, see CVF interface
affinity mask, 6-16 Compatibility OpenMP* run-time library, 3-4
aligning data, 8-2 compiler support, 2-6
compiler support run-time libraries, 3-4
compiler-dependent function, 7-4
B
computational layer, 3-4
benchmark, 11-1 configuration file, for OOC DSS/PARDISO*, 4-11
BLAS
configuring development environment, 4-1
calling routines from C, 7-6 automatically configuring Microsoft Visual
Fortran-95 interfaces to, 7-3 Studio* project, 4-2
Intel(R) Visual Fortran, 4-2
C Microsoft Visual* C/C++, 4-1
C, calling LAPACK, BLAS, CBLAS from, 7-5 Visual Studio* project running Intel(R) MKL
calling examples, 4-4
BLAS functions in C, 7-8 configuring, Visual Studio* project with Intel(R)
complex BLAS Level 1 function from C, 7-9 MKL, automatically, 4-2
context-sensitive Help, for Intel(R) MKL, in Visual
complex BLAS Level 1 function from C++, 7-10
Studio* IDE, 10-3
Fortran-style routines from C, 7-5 custom DLL, 5-9
calling convention, cdecl and stdcall, 2-4 building, 5-9, 5-10
CBLAS, 7-7 specifying list of functions, 5-11
CBLAS, code example, 7-11
specifying makefile parameters, 5-9
cdecl calling convention, 2-4
Cluster FFT, linking with, 9-2 custom dynamically linked shared library
building, 5-9
cluster software, 9-1
linking examples, 9-5 CVF interface, support, 2-3
linking syntax, 9-2
MPI support, 9-1 D
coding data alignment, 8-2
data alignment, 8-1 denormal number, performance, 6-17

Index-1
Intel® Math Kernel Library User’s Guide

denormal, performance, 6-17 J

development environment, configuring, 4-1
Java* examples, 7-13
directory structure
documentation, 3-21
high-level, 3-1 L
in-detail, 3-8 language interfaces support, A-1
language-specific interfaces, 7-1
dispatching, of AVX instructions, 6-13
documentation, 3-21 LAPACK
for Intel(R) MKL, viewing in Visual Studio* IDE, calling routines from C, 7-5
10-1 Fortran-95 interfaces to, 7-3
packed routines performance, 6-14
E layer
compiler support RTL, 3-4
environment variables, setting, 4-1
examples computational, 3-4
code, 2-6 interface, 3-4
linking, general, 5-6 threading, 3-4
ScaLAPACK, Cluster FFT, linking with, 9-5 layered model, 3-3
library
run-time, Compatibility OpenMP*, 3-4
F
run-time, Legacy OpenMP*, 3-4
FFT functions, data alignment, 6-15
library structure, 3-1
FFT interface
link libraries
optimized radices, 6-17
computational, 5-4
FFTW interface support, B-1
for Intel(R) 64 architecture, 5-4
Fortran-95, interfaces to LAPACK and BLAS, 7-3
threading, 5-3
linking, 5-1
G with Cluster FFT, 9-2
GNU* Multiple Precision Arithmetic Library, B-1 with ScaLAPACK, 9-2
LINPACK benchmark, 11-1
H
Help, for Intel(R) MKL in Visual Studio* IDE, 10-1 M
HT Technology, see Hyper-Threading technology
memory functions, redefining, 6-18
hybrid, version, of MP LINPACK, 11-4
memory management, 6-17
Hyper-Threading Technology, configuration tip, 6-15
memory renaming, 6-18
mixed-language programming, 7-4
I module, Fortran-95, 7-4
ILP64 programming, support for, 3-5 MP LINPACK benchmark, 11-4
instability, numerical, getting rid of, 8-1 hybrid version, 11-4
installation, checking, 2-4 multi-core performance, 6-16
IntelliSense*, with Intel(R) MKL, in Visual Studio*
IDE, 10-5
N
interface layer, 3-4
notational conventions, 1-3

Index-2
number of threads T
changing at run time, 6-5
technical support, 1-1
changing with OpenMP* environment variable, thread safety, of Intel(R) MKL, 6-2
6-5 threading
Intel(R) MKL choice, particular cases, 6-11 avoiding conflicts, 6-4
setting for cluster, 9-3 environment variables and functions, 6-8
techniques to set, 6-3 Intel(R) MKL behavior, particular cases, 6-11
numerical stability, 8-1 Intel(R) MKL controls, 6-8
see also number of threads
O threading layer, 3-4
OpenMP*
Compatibility run-time library, 3-4
U
Legacy run-time library, 3-4
uBLAS, matrix-matrix multiplication, substitution
OpenMP*, run-time library, 5-2 with Intel MKL functions, 7-11
unstable output, numerically, getting rid of, 8-1
usage information, 1-1
P
parallel performance, 6-4
parallelism, 6-1 V
PARDISO* OOC, configuration file, 4-11 Visual C/C++*, configuring, 4-1
performance, 6-1 Visual Fortran, configuring, 4-2
coding techniques to gain, 6-14 Visual Studio* IDE
hardware tips to gain, 6-15 IntelliSense*, with Intel(R) MKL, 10-5
multi-core, 6-16 project configuring with Intel(R) MKL,
automatically, 4-2
of LAPACK packed routines, 6-14
Visual Studio* IDE, Intel(R) MKL Help, 10-1
with denormals, 6-17
context-sensitive, 10-3
with subnormals, 6-17

R
RTL, 7-4
run-time library, 7-4
Compatibility OpenMP*, 3-4
Legacy OpenMP*, 3-4

S
ScaLAPACK, linking with, 9-2
stability, numerical, 8-1
stdcall calling convention, 2-4
subnormal number, performance, 6-17
support, technical, 1-1
syntax, linking, cluster software, 9-2

Index-3