Basic Linear Algebra Subprograms - AbsoluteAstronomy.com

Basic Linear Algebra Subprograms (BLAS) is a de facto

De facto standard

A de facto standard is a custom, convention, product, or system that has achieved a dominant position by public acceptance or market forces...

application programming interface

Application programming interface

An application programming interface is a source code based specification intended to be used as an interface by software components to communicate with each other...

standard for publishing libraries to perform basic linear algebra

Linear algebra

Linear algebra is a branch of mathematics that studies vector spaces, also called linear spaces, along with linear functions that input one vector and output another. Such functions are called linear maps and can be represented by matrices if a basis is given. Thus matrix theory is often...

operations such as vector

Vector space

A vector space is a mathematical structure formed by a collection of vectors: objects that may be added together and multiplied by numbers, called scalars in this context. Scalars are often taken to be real numbers, but one may also consider vector spaces with scalar multiplication by complex...

and matrix multiplication

Matrix multiplication

In mathematics, matrix multiplication is a binary operation that takes a pair of matrices, and produces another matrix. If A is an n-by-m matrix and B is an m-by-p matrix, the result AB of their multiplication is an n-by-p matrix defined only if the number of columns m of the left matrix A is the...

. They were first published in 1979, and are used to build larger packages such as LAPACK

LAPACK

-External links:* : a modern replacement for PLAPACK and ScaLAPACK* on Netlib.org* * * : a modern replacement for LAPACK that is MultiGPU ready* on Sourceforge.net* * optimized LAPACK for Solaris OS on SPARC/x86/x64 and Linux* * *...

. Heavily used in high-performance computing

High-performance computing

High-performance computing uses supercomputers and computer clusters to solve advanced computation problems. Today, computer systems approaching the teraflops-region are counted as HPC-computers.-Overview:...

, highly optimized implementations of the BLAS interface have been developed by hardware vendors such as Intel and AMD

Advanced Micro Devices

Advanced Micro Devices, Inc. or AMD is an American multinational semiconductor company based in Sunnyvale, California, that develops computer processors and related technologies for commercial and consumer markets...

, as well as by other authors, e.g. Goto

Kazushige Goto

was a research associate at the Texas Advanced Computing Center at the University of Texas at Austin when he famously hand-optimized assembly routines for supercomputing and PC platforms that outperform the best compiler generated code. Several of the fastest supercomputers in the world still use...

BLAS and ATLAS

Automatically Tuned Linear Algebra Software

Automatically Tuned Linear Algebra Software is a software library for linear algebra. It provides a mature open source implementation of BLAS APIs for C and Fortran77....

(a portable self-optimizing BLAS). The LINPACK

LINPACK

LINPACK is a software library for performing numerical linear algebra on digital computers. It was written in Fortran by Jack Dongarra, Jim Bunch, Cleve Moler, and Gilbert Stewart, and was intended for use on supercomputers in the 1970s and early 1980s...

benchmark relies heavily on DGEMM

General Matrix Multiply

The General Matrix Multiply is a subroutine in the Basic Linear Algebra Subprograms which performs matrix multiplication, that is the multiplication of two matrices...

, a BLAS subroutine, for its performance.

Level 1

This level contains vector operations of the form

as well as scalar dot product

Dot product

In mathematics, the dot product or scalar product is an algebraic operation that takes two equal-length sequences of numbers and returns a single number obtained by multiplying corresponding entries and then summing those products...

s and vector norm

Norm (mathematics)

In linear algebra, functional analysis and related areas of mathematics, a norm is a function that assigns a strictly positive length or size to all vectors in a vector space, other than the zero vector...

s, among other things.

Level 2

This level contains matrix-vector operations of the form

as well as solving

for

with

being triangular, among other things.

Level 3

This level contains matrix-matrix operations of the form

as well as solving

for triangular matrices

, among other things. This level contains the widely used General Matrix Multiply

General Matrix Multiply

The General Matrix Multiply is a subroutine in the Basic Linear Algebra Subprograms which performs matrix multiplication, that is the multiplication of two matrices...

operation.

Implementations

Accelerate: Apple's framework for Mac OS X

Mac OS X

Mac OS X is a series of Unix-based operating systems and graphical user interfaces developed, marketed, and sold by Apple Inc. Since 2002, has been included with all new Macintosh computer systems...

and iOS, which includes tuned versions of BLAS and LAPACK.http://developer.apple.com/library/mac/#releasenotes/Performance/RN-vecLib/ http://developer.apple.com/library/ios/#documentation/Accelerate/Reference/AccelerateFWRef/
ACML: The AMD Core Math Library

AMD Core Math Library

AMD Core Math Library is a software development library released by AMD. This library provides useful mathematical routines optimized for AMD processors....

, supporting the AMD Athlon

Athlon

Athlon is the brand name applied to a series of x86-compatible microprocessors designed and manufactured by Advanced Micro Devices . The original Athlon was the first seventh-generation x86 processor and, in a first, retained the initial performance lead it had over Intel's competing processors...

and Opteron

Opteron

Opteron is AMD's x86 server and workstation processor line, and was the first processor which supported the AMD64 instruction set architecture . It was released on April 22, 2003 with the SledgeHammer core and was intended to compete in the server and workstation markets, particularly in the same...

CPUs under Linux

Linux

Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...

and Windows

Microsoft Windows

Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...

.http://developer.amd.com/acml.aspx
ATLAS: Automatically Tuned Linear Algebra Software

Automatically Tuned Linear Algebra Software

Automatically Tuned Linear Algebra Software is a software library for linear algebra. It provides a mature open source implementation of BLAS APIs for C and Fortran77....

, an open source

Open source

The term open source describes practices in production and development that promote access to the end product's source materials. Some consider open source a philosophy, others consider it a pragmatic methodology...

implementation of BLAS API

Application programming interface

An application programming interface is a source code based specification intended to be used as an interface by software components to communicate with each other...

s for C

C (programming language)

C is a general-purpose computer programming language developed between 1969 and 1973 by Dennis Ritchie at the Bell Telephone Laboratories for use with the Unix operating system....

and Fortran 77

Fortran

Fortran is a general-purpose, procedural, imperative programming language that is especially suited to numeric computation and scientific computing...

.http://math-atlas.sourceforge.net/
ESSL: IBM

IBM

International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...

's Engineering and Scientific Subroutine Library, supporting the PowerPC

PowerPC

PowerPC is a RISC architecture created by the 1991 Apple–IBM–Motorola alliance, known as AIM...

architecture under AIX

AIX operating system

AIX AIX AIX (Advanced Interactive eXecutive, pronounced "a i ex" is a series of proprietary Unix operating systems developed and sold by IBM for several of its computer platforms...

and Linux

Linux

.http://publib.boulder.ibm.com/infocenter/clresctr/index.jsp?topic=/com.ibm.cluster.essl.doc/esslbooks.html
Eigen BLAS: A Fortran 77

Fortran

Fortran is a general-purpose, procedural, imperative programming language that is especially suited to numeric computation and scientific computing...

and C

C (programming language)

C is a general-purpose computer programming language developed between 1969 and 1973 by Dennis Ritchie at the Bell Telephone Laboratories for use with the Unix operating system....

BLAS library implemented on top of the open source

Open source

Eigen library, supporting x86, x86 64, ARM (NEON)

ARM architecture

ARM is a 32-bit reduced instruction set computer instruction set architecture developed by ARM Holdings. It was named the Advanced RISC Machine, and before that, the Acorn RISC Machine. The ARM architecture is the most widely used 32-bit ISA in numbers produced...

, and PowerPC

PowerPC

PowerPC is a RISC architecture created by the 1991 Apple–IBM–Motorola alliance, known as AIM...

architectures.http://eigen.tuxfamily.org (Note: as of Eigen 3.0.3, the BLAS interface is not built by default and the documentation refers to it as "a work in progress which is far to be ready for use".)
Goto BLAS: Kazushige Goto

Kazushige Goto

's implementation of BLAS.http://www.tacc.utexas.edu/tacc-projects/#blas
HP MLIB: HP

Hewlett-Packard

Hewlett-Packard Company or HP is an American multinational information technology corporation headquartered in Palo Alto, California, USA that provides products, technologies, softwares, solutions and services to consumers, small- and medium-sized businesses and large enterprises, including...

's Math library supporting IA-64, PA-RISC

PA-RISC

PA-RISC is an instruction set architecture developed by Hewlett-Packard. As the name implies, it is a reduced instruction set computer architecture, where the PA stands for Precision Architecture...

, x86 and Opteron

Opteron

architecture under HPUX and Linux

Linux

.
Intel MKL: The Intel Math Kernel Library

Math Kernel Library

Intel's Math Kernel Library is a library of optimized, math routines for science, engineering, and financial applications. Core math functions include BLAS, LAPACK, ScaLAPACK, Sparse Solvers, Fast Fourier Transforms and Vector Math....

, supporting the old Intel Pentium (although there are some doubts about future support to the Pentium architecture), Core

Intel Core

Yonah was the code name for Intel's first generation of 65 nm process mobile microprocessors, based on the Banias/Dothan-core Pentium M microarchitecture. SIMD performance has been improved through the addition of SSE3 instructions and improvements to SSE and SSE2 implementations, while integer...

and Itanium

Itanium

Itanium is a family of 64-bit Intel microprocessors that implement the Intel Itanium architecture . Intel markets the processors for enterprise servers and high-performance computing systems...

CPUs under Linux

Linux

, Windows

Microsoft Windows

and Mac OS X

Mac OS X

Mac OS X is a series of Unix-based operating systems and graphical user interfaces developed, marketed, and sold by Apple Inc. Since 2002, has been included with all new Macintosh computer systems...

.http://www.intel.com/cd/software/products/asmo-na/eng/perflib/mkl/
MathKeisan: NEC's math library, supporting NEC SX architecture under SUPER-UX

SUPER-UX

SUPER-UX, sometimes also annotated SuperUx or Super-UX, is the version of the Unix operating system that is used on NEC SX architecture supercomputers...

, and Itanium

Itanium

Itanium is a family of 64-bit Intel microprocessors that implement the Intel Itanium architecture . Intel markets the processors for enterprise servers and high-performance computing systems...

under Linux

Linux

http://www.mathkeisan.com/
Netlib BLAS: The official reference implementation on Netlib

Netlib

Netlib is a repository of software for scientific computing maintained by AT&T, Bell Laboratories, the University of Tennessee and Oak Ridge National Laboratory. Netlib comprises a large number of separate programs and libraries...

, written in Fortran 77

Fortran

Fortran is a general-purpose, procedural, imperative programming language that is especially suited to numeric computation and scientific computing...

. http://www.netlib.org/blas/
Netlib CBLAS: Reference C

C (programming language)

C is a general-purpose computer programming language developed between 1969 and 1973 by Dennis Ritchie at the Bell Telephone Laboratories for use with the Unix operating system....

interface to the BLAS. It is also possible (and popular) to call the Fortran BLAS from C. http://www.netlib.org/blas
PDLIB/SX: NEC's Public Domain Mathematical Library for the NEC SX-4 system.http://www.nec.co.jp/hpc/mediator/sxm_e/software/61.html
SCSL: SGI

Silicon Graphics

Silicon Graphics, Inc. was a manufacturer of high-performance computing solutions, including computer hardware and software, founded in 1981 by Jim Clark...

's Scientific Computing Software Library contains BLAS and LAPACK implementations for SGI's Irix

IRIX

IRIX is a computer operating system developed by Silicon Graphics, Inc. to run natively on their 32- and 64-bit MIPS architecture workstations and servers. It was based on UNIX System V with BSD extensions. IRIX was the first operating system to include the XFS file system.The last major version...

workstations.http://www.sgi.com/products/software/scsl.html
Sun Performance Library: Optimized BLAS and LAPACK for SPARC

SPARC

SPARC is a RISC instruction set architecture developed by Sun Microsystems and introduced in mid-1987....

, Core

Intel Core

and AMD64 architectures under Solaris 8, 9, and 10 as well as Linux.http://www.oracle.com/technetwork/server-storage/solarisstudio/overview/index.html
SurviveGotoBLAS2: Optimized BLAS that is an attempt to continue the work of Kazushige Goto

Kazushige Goto

by Ei-ji Nakama. http://prs.ism.ac.jp/~nakama/SurviveGotoBLAS2/
OpenBLAS: Optimized BLAS based on Goto BLAS hosted at GitHub with emphasis on Loongson processors. http://xianyi.github.com/OpenBLAS/

Other libraries offering BLAS-like functionality

AMD APPML: AMD Accelerated Parallel Processing Math Libraries contains FFT and Level 3 BLAS functions written in OpenCL

OpenCL

OpenCL is a framework for writing programs that execute across heterogeneous platforms consisting of CPUs, GPUs, and other processors. OpenCL includes a language for writing kernels , plus APIs that are used to define and then control the platforms...

. Designed to run on AMD GPUs supporting OpenCL also work on CPUs to facilitate multicore programming and debugging. http://developer.amd.com/gpu/appmathlibs/Pages/default.aspx
Armadillo: Armadillo

Armadillo (C++ library)

The Armadillo C++ Library aims to provide an efficient and streamlined base for linear algebra operations , while at the same time having a straightforward and easy to use interface...

is a C++ linear algebra library aiming towards a good balance between speed and ease of use. It employs template classes, and has optional links to BLAS/ATLAS and LAPACK. It is sponsored by NICTA (in Australia) and is licensed under LGPL. http://arma.sourceforge.net/.
Eigen: The Eigen template library provides an easy to use highly generic C++ template interface to matrix/vector operations and related algorithms like solving algorithms, decompositions etc. It uses vector capabilities and is optimized for both fixed size and dynamic sized and sparse matrices.http://eigen.tuxfamily.org
CUDA SDK: The NVIDIA CUDA

CUDA

CUDA or Compute Unified Device Architecture is a parallel computing architecture developed by Nvidia. CUDA is the computing engine in Nvidia graphics processing units that is accessible to software developers through variants of industry standard programming languages...

SDK includes BLAS functionality for writing C programs that runs on GeForce 8 Series

GeForce 8 Series

The GeForce 8 Series, is the eighth generation of NVIDIA's GeForce line of graphics processing units. The third major GPU architecture developed at NVIDIA, the GeForce 8 represents the company's first unified shader architecture.-Naming:...

or newer graphics cards.
GSL: The GNU Scientific Library

GNU Scientific Library

In computing, the GNU Scientific Library is a software library written in the C programming language for numerical calculations in applied mathematics and science...

Contains a multi-platform implementation in C which is distributed under the GNU

GNU

GNU is a Unix-like computer operating system developed by the GNU project, ultimately aiming to be a "complete Unix-compatible software system"...

General Public License.
Libflame: FLAME project implementation of dense linear algebra library http://z.cs.utexas.edu/wiki/flame.wiki/FrontPage
MAGMA: Matrix Algebra on GPU and Multicore Architectures (MAGMA) project develops a dense linear algebra library similar to LAPACK but for heterogeneous and hybrid architectures including multicore systems accelerated with GPGPU

GPGPU

General-purpose computing on graphics processing units is the technique of using a GPU, which typically handles computation only for computer graphics, to perform computation in applications traditionally handled by the CPU...

graphics cards. http://icl.eecs.utk.edu/magma/
MTL4: The Matrix Template Library

Matrix Template Library

The Matrix Template Library is a linear algebra library for C++ programs.The MTL uses template programming, which considerably reduces the code length. All matrices and vectors are available in all classical numerical formats: float, double, complex or complex.Furthermore, generic programming...

version 4 is a generic C++

C++

C++ is a statically typed, free-form, multi-paradigm, compiled, general-purpose programming language. It is regarded as an intermediate-level language, as it comprises a combination of both high-level and low-level language features. It was developed by Bjarne Stroustrup starting in 1979 at Bell...

template library providing sparse and dense BLAS functionality. MTL4 establishes an intuitive interface (similar to MATLAB

MATLAB

MATLAB is a numerical computing environment and fourth-generation programming language. Developed by MathWorks, MATLAB allows matrix manipulations, plotting of functions and data, implementation of algorithms, creation of user interfaces, and interfacing with programs written in other languages,...

) and broad applicability thanks to Generic programming

Generic programming

In a broad definition, generic programming is a style of computer programming in which algorithms are written in terms of to-be-specified-later types that are then instantiated when needed for specific types provided as parameters...

.
PLASMA: The Parallel Linear Algebra for Scalable Multi-core Architectures (PLASMA) project is a modern replacement of LAPACK for multi-core architectures. PLASMA is a software framework for development of asynchronous operations and features out of order scheduling with a runtime scheduler called QUARK that may be used for any code that expresses its dependencies with a Directed acyclic graph

Directed acyclic graph

In mathematics and computer science, a directed acyclic graph , is a directed graph with no directed cycles. That is, it is formed by a collection of vertices and directed edges, each edge connecting one vertex to another, such that there is no way to start at some vertex v and follow a sequence of...

. http://icl.eecs.utk.edu/
uBLAS: A generic C++

C++

template class library providing BLAS functionality. Part of the Boost library

Boost library

Boost is a set of free software libraries that extend the functionality of C++.-Overview:Most of the Boost libraries are licensed under the Boost Software License, designed to allow Boost to be used with both free and proprietary software projects...

. It provides bindings to many hardware-accelerated libraries in a unifying notation. Moreover, uBLAS focuses on correctness of the algorithms using advanced C++ features. http://www.boost.org/doc/libs/release/libs/numeric/ublas/doc/index.htm

The Sparse BLAS

Sparse

Sparse matrix

In the subfield of numerical analysis, a sparse matrix is a matrix populated primarily with zeros . The term itself was coined by Harry M. Markowitz....

extensions to the previously dense BLAS exist such as in ACML

External links

BLAS homepage on Netlib.org
BLAS FAQ
BLAS operations from the GNU Scientific Library reference manual
BLAS Quick Reference Guide from LAPACK Users' Guide
CSBlas for C#. CSBlas is the translation of Fortran to C# of the BLAS subroutines.
Lawson Oral History One of the original authors of the BLAS discusses its creation in an oral history interview. Charles L. Lawson Oral history interview by Thomas Haigh, 6 and 7 November 2004, San Clemente, California. Society for Industrial and Applied Mathematics, Philadelphia, PA.
Dongarra Oral History In an oral history interview, Jack Dongarra explores the early relationship of BLAS to LINPACK, the creation of higher level BLAS versions for new architectures, and his later work on the ATLAS system to automatically optimize BLAS for particular machines. Jack Dongarra, Oral history interview by Thomas Haigh, 26 April 2005, University of Tennessee, Knoxville TN. Society for Industrial and Applied Mathematics, Philadelphia, PA

An Overview of the Sparse Basic Linear Algebra Subprograms: The New Standard from the BLAS Technical Forum

The source of this article is wikipedia, the free encyclopedia. The text of this article is licensed under the GFDL.

Level 1

Level 2

Level 3

Implementations

Other libraries offering BLAS-like functionality

The Sparse BLAS

See also

External links