Threads synchronization primitive

In parallel computing

Parallel computing

Parallel computing is a form of computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently . There are several different forms of parallel computing: bit-level,...

, a barrier is a type of synchronization

Synchronization (computer science)

In computer science, synchronization refers to one of two distinct but related concepts: synchronization of processes, and synchronization of data. Process synchronization refers to the idea that multiple processes are to join up or handshake at a certain point, so as to reach an agreement or...

 method. A barrier for a group of threads or processes in the source code means any thread/process must stop at this point and cannot proceed until all other threads/processes reach this barrier.

Many collective routines and directive-based parallel languages impose implicit barriers. For example, a parallel do loop in Fortran

Fortran

Fortran is a general-purpose, procedural, imperative programming language that is especially suited to numeric computation and scientific computing...

  with OpenMP

OpenMP

OpenMP is an API that supports multi-platform shared memory multiprocessing programming in C, C++, and Fortran, on most processor architectures and operating systems, including Linux, Unix, AIX, Solaris, Mac OS X, and Microsoft Windows platforms...

 will not be allowed to continue on any thread until the last iteration is completed. This is in case the program relies on the result of the loop immediately after its completion. In message passing

Message passing

Message passing in computer science is a form of communication used in parallel computing, object-oriented programming, and interprocess communication. In this model, processes or objects can send and receive messages to other processes...

, any global communication (such as reduction or scatter) may imply a barrier.

See also: Rendezvous (Plan 9)

Rendezvous (Plan 9)

Rendezvous is a data synchronization mechanism in Plan 9 from Bell Labs. It is a system call that allows two processes to exchange a single datum while synchronizing....

.

Dynamic Barriers

Classic barrier constructs define the set of participating processes/threads statically. This is usually done either at program startup or when a barrier like the Pthreads

POSIX Threads

POSIX Threads, usually referred to as Pthreads, is a POSIX standard for threads. The standard, POSIX.1c, Threads extensions , defines an API for creating and manipulating threads....

 barrier is instantiated. This restricts the possible applications for which barriers can be used.

To support more dynamic programming paradigms like fork/join parallelism, the sets of participants have to be dynamic. Thus, the set of processes/threads participating in a barrier operation needs to be able to change over time. X10

X10 (programming language)

X10 is a programming language being developed by IBM at the Thomas J. Watson Research Center as part of the Productive, Easy-to-use, Reliable Computing System project funded by DARPA's High Productivity Computing Systems program...

 introduced the concept of clocks for that purpose, which provide a dynamic barrier semantic. Building on clocks, phasers have been proposed to add even more flexibility to barrier synchronization. With phasers it is possible to express data dependencies between the participating processes explicit to avoid unnecessary over-synchronization.

Processor and compiler barriers

Memory barrier

Memory barrier

Memory barrier, also known as membar or memory fence or fence instruction, is a type of barrier and a class of instruction which causes a central processing unit or compiler to enforce an ordering constraint on memory operations issued before and after the barrier instruction.CPUs employ...

is a class of instructions which cause a processor to enforce an ordering constraint on memory operations issued before and after the barrier instruction.

A barrier can also be a high level programming language statement which prevent the compiler

Compiler

A compiler is a computer program that transforms source code written in a programming language into another computer language...

from reordering other operations over the barrier statement during optimization passes. Such statements can potentially generate processor barrier instructions. Different classes of barrier exist and may apply to a specific set of operations only.