BLAS and LAPACK at TACC
Last update: July 17, 2020

BLAS/LAPACK Implementations

BLAS (Basic Linear Algebra Subprograms) is a set of definitions of common operations on vectors and (dense) matrices. LAPACK is the Linear Algebra Package that builds on BLAS and that offers numerical algorithms such as linear system solving and eigenvalue calculations. The so-called "reference" implementations of BLAS/LAPACK are written in Fortran and can be found on http://netlib.org, but in practice you don't want to use them since they have low performance. Instead, TACC offers libraries that conform to the specification, but that achieve high performance. They are typically written in a combination of C and Assembly.

Updating your makefile

If your makefile contains "libblas.a" or "-lblas". Most Linux distributions indeed have a library by that name, but it will not be tuned for the TACC processor types. Instead, use one of the following libraries.

MKL

Intel's Math Kernal Library (MKL) is a high performance implementation of BLAS/LAPACK and several other packages. MKL is installed on TACC's Frontera, Stampede2 and Lonestar5 resources. See each resource's user guide for detailed information on linking the MKL into your code.

In general:

  • Under the Intel compiler, using BLAS/LAPACK is done through adding the flag "-mkl" both at compile and link time.

  • Under the GCC compiler, there is a module "mkl" that defines the TACC_MKL_DIR and TACC_MKL_LIB environment variables. What libraries you need depends on your application, but often the following works:

      -L${TACC_MKL_LIB} -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread
  • If you are installing PETSc, use the the following options:

      --with-blas-lapack-dir=${TACC_MKL_DIR}

    This variable is automatically defined when using the Intel compiler. If you are compiling with gcc, then you must first load the MKL module:

      module load mkl
  • If you are installing software using cmake, add this flag (exact flag may differ by package):

      -D LAPACK_LIBRARIES=${TACC_MKL_DIR}/lib/intel64_lin/libmkl_intel_lp64.so;${TACC_MKL_DIR}/lib/intel64_lin/libmkl_core.so

    If paths and library names are separately specified:

      -D LAPACK_LIBRARY_DIRS:PATH="${TACC_MKL_LIB}"
      -D LAPACK_LIBRARY_NAMES:STRING="mkl_intel_lp64;mkl_sequential;mkl_core;pthread"

BLIS

BLIS (BLAS-like Library Instantiation Software Framework) is an open source high performance implementation of BLAS/LAPACK. It can be accessed through a module: "module load blis". You will then find the library file in $TACC_BLIS_LIB. BLIS extends the BLAS specification; for documentation see https://github.com/flame/blis.

Reference BLAS/LAPACK

The reference implementation for BLAS/LAPACK is written in Fortran and is very low performance. However, for debugging purposes it can be useful. You can load it with:

module load referencelapack

This gives access to the compiled Fortran sources from netlib.org/lapack.

Goto Blas and OpenBlas

Older implementations such as Goto Blas (after former TACC employee Kazushige Goto), and its offshoot, OpenBlas, are no longer maintained and should not be used. Instead, use MKL or BLIS as described above.

BLAS/Lapack on Longhorn

Longhorn is an IBM system, which comes with IBM's own compilers and numerical libraries. The equivalent of MKL is ESSL (Engineering and Scientific Software Library). Link ESSL into your code with:

login1$ xlc -o myprogram myprogram.c -L/opt/ibmmath/essl/6.2/lib64/ -lessl