Cluster (computing)
Encyclopedia
A computer cluster is a group of linked computer
s, working together closely thus in many respects forming a single computer. The components of a cluster are commonly, but not always, connected to each other through fast local area network
s. Clusters are usually deployed to improve performance and availability over that of a single computer, while typically being much more cost-effective than single computers of comparable speed or availability.
s (also known as failover cluster) are implemented primarily for the purpose of improving the availability of services that the cluster provides. They operate by having redundant nodes
, which are then used to provide service when system components fail. The most common size for an HA cluster is two nodes
, which is the minimum requirement to provide redundancy. HA cluster implementations attempt to use redundancy of cluster components to eliminate single points of failure
.
There are commercial implementations of High-Availability clusters for many operating systems. The Linux-HA
project is one commonly used free software
HA package for the Linux
operating system.
is when multiple computers are linked together to share computational workload or function as a single virtual computer. Logically, from the user side, they are multiple machines, but function as a single virtual machine. Requests initiated from the user are managed by, and distributed among, all the standalone computers to form a cluster. This results in balanced computational work among different machines, improving the performance of the cluster systems.
of weather or vehicle crashes. The primary distinction within computer clusters is how tightly-coupled the individual nodes are. For instance, a single computer job may require frequent communication among nodes: this implies that the cluster shares a dedicated network, is densely located, and probably has homogenous nodes. This cluster design is usually referred to as Beowulf Cluster. The other extreme is where a computer job uses one or few nodes, and needs little or no inter-node communication. This latter category is sometimes called "Grid" computing. Tightly-coupled compute clusters are designed for work that might traditionally have been called "supercomputing". Middleware such as MPI (Message Passing Interface)
or PVM (Parallel Virtual Machine) permits compute clustering programs to be portable to a wide variety of clusters.
organization's semiannual list of the 500 fastest computers usually includes many clusters. TOP500 is a collaboration between the University of Mannheim
, the University of Tennessee
, and the National Energy Research Scientific Computing Center at Lawrence Berkeley National Laboratory
. As of July 2011 the top supercomputer
is the K computer
in Kobe, Japan, with performance of 8.162 PFlops
measured with the LINPACK
benchmark.
Clustering can provide significant performance benefits versus price. The System X
supercomputer at Virginia Tech, the 28th most powerful supercomputer on Earth as of June 2006, is a 12.25 TFlops computer cluster of 1100 Apple XServe
G5
2.3 GHz dual-processor machines (4 GB
RAM, 80 GB SATA
HD
) running Mac OS X
and using InfiniBand
interconnect. The cluster initially consisted of Power Mac G5s; the rack-mountable XServes are denser than desktop Macs, reducing the aggregate size of the cluster. The total cost of the previous Power Mac system was $5.2 million, a tenth of the cost of slower mainframe computer
supercomputers. (The Power Mac G5s were sold off.)
The central concept of a Beowulf
cluster is the use of commercial off-the-shelf
(COTS) computers to produce a cost-effective alternative to a traditional supercomputer. One project that took this to an extreme was the Stone Soupercomputer
.
However it is worth noting that Flops (floating point operations per second), aren't always the best metric for supercomputer speed. Clusters can have very high Flops, but they cannot access all data in the cluster as a whole at once. Therefore clusters are excellent for parallel computation, but much poorer than traditional supercomputers at non-parallel computation.
JavaSpaces is a specification from Sun Microsystems
that enables clustering computers via a distributed shared memory
.
(HPC) clusters. Some examples of game console clusters are Sony PlayStation clusters
and Microsoft
Xbox clusters. It has been suggested on a news website that countries which are restricted from buying supercomputing technologies may be obtaining game systems to build computer clusters for military use.
The formal engineering basis of cluster computing as a means of doing parallel work of any sort was arguably invented by Gene Amdahl
of IBM
, who in 1967 published what has come to be regarded as the seminal paper on parallel processing: Amdahl's Law
. Amdahl's Law describes mathematically the speedup one can expect from parallelizing any given otherwise serially performed task on a parallel architecture. This article defined the engineering basis for both multiprocessor computing and cluster computing, where the primary differentiator is whether or not the interprocessor communications are supported "inside" the computer (on for example a customized internal communications bus or network) or "outside" the computer on a commodity network.
Consequently the history of early computer clusters is more or less directly tied into the history of early networks, as one of the primary motivations for the development of a network was to link computing resources, creating a de facto computer cluster. Packet switching
networks were conceptually invented by the RAND
corporation in 1962. Using the concept of a packet switched network, the ARPANET
project succeeded in creating in 1969 what was arguably the world's first commodity-network based computer cluster by linking four different computer centers (each of which was something of a "cluster" in its own right, but probably not a commodity cluster). The ARPANET project grew into the Internet
—which can be thought of as "the mother of all computer clusters" (as the union of nearly all of the compute resources, including clusters, that happen to be connected). It also established the paradigm in use by all computer clusters in the world today—the use of packet-switched networks to perform interprocessor communications between processor (sets) located in otherwise disconnected frames.
The development of customer-built and research clusters proceeded hand in hand with that of both networks and the Unix
operating system from the early 1970s, as both TCP/IP and the Xerox PARC
project created and formalized protocols for network-based communications. The Hydra operating system
was built for a cluster of DEC PDP-11
minicomputers called C.mmp
at Carnegie Mellon University
in 1971. However, it was not until circa 1983 that the protocols and tools for easily doing remote job distribution and file sharing were defined (largely within the context of BSD
Unix, as implemented by Sun Microsystems
) and hence became generally available commercially, along with a shared filesystem.
The first commercial clustering product was ARCnet
, developed by Datapoint
in 1977. ARCnet was not a commercial success and clustering per se did not really take off until Digital Equipment Corporation
released their VAXcluster product in 1984 for the VAX/VMS operating system. The ARCnet and VAXcluster products not only supported parallel computing, but also shared file system
s and peripheral
devices. The idea was to provide the advantages of parallel processing, while maintaining data reliability and uniqueness. VAXcluster, now VMScluster, is still available on OpenVMS
systems from HP
running on Alpha and Itanium systems.
Two other noteworthy early commercial clusters were the Tandem Himalaya
(a circa 1994 high-availability product) and the IBM S/390 Parallel Sysplex (also circa 1994, primarily for business use).
No history of commodity computer clusters would be complete without noting the pivotal role played by the development of Parallel Virtual Machine
(PVM) software in 1989. This open source software based on TCP/IP communications enabled the instant creation of a virtual supercomputer—a high performance compute cluster—made out of any TCP/IP connected systems. Free form heterogeneous clusters built on top of this model rapidly achieved total throughput in FLOPS
that greatly exceeded that available even with the most expensive "big iron
" supercomputers. PVM and the advent of inexpensive networked PCs led, in 1993, to a NASA
project to build supercomputers out of commodity clusters. In 1995 the Beowulf cluster—a cluster built on top of a commodity network for the specific purpose of "being a supercomputer" capable of performing tightly coupled parallel HPC computations—was invented, which spurred the independent development of grid computing
as a named entity, although Grid-style clustering had been around at least as long as the Unix
operating system and the Arpanet, whether or not it, or the clusters that used it, were named.
is a widely-available communications library that enables parallel programs to be written in C
, Fortran
, Python
, OCaml, and many other programming languages.
The GNU/Linux world supports various cluster software; for application clustering, there is Beowulf
, distcc
, and MPICH
. Linux Virtual Server
, Linux-HA
- director-based clusters that allow incoming requests for services to be distributed across multiple cluster nodes. MOSIX
, openMosix
, Kerrighed
, OpenSSI
are full-blown clusters integrated into the kernel that provide for automatic process migration among homogeneous nodes. OpenSSI, openMosix and Kerrighed are single-system image
implementations.
Microsoft Windows
Compute Cluster Server 2003 based on the Windows Server
platform provides pieces for High Performance Computing like the Job Scheduler, MSMPI library and management tools. NCSA
's recently installed Lincoln is a cluster of 450 Dell PowerEdge 1855 blade servers running Windows Compute Cluster Server 2003. This cluster debuted at #130 on the Top500
list in June 2006.
gridMathematica
provides distributed computations over clusters including data analysis, computer algebra and 3D visualization. It can make use of other technologies such as Altair PBS Professional, Microsoft Windows Compute Cluster Server, Platform LSF and Sun Grid Engine
.
gLite
is a set of middleware technologies created by the Enabling Grids for E-sciencE (EGEE) project.
Another example of consumer game products being added to high-performance computing is the Nvidia Tesla Personal Supercomputer
workstation, which gets its processing power by harnessing the power of multiple graphics accelerator processor chips.
Algorithmic skeleton
s are a high-level parallel programming model for parallel and distributed computing which take advantage of common programming patterns to hide the complexity of parallel and distributed applications. Starting from a basic set of patterns (skeletons), more complex patterns can be built by combining the basic ones.
Global Storage Architecture (GSA)—a highly scalable cloud based NAS solution—combines proprietary IBM HPC technology (storage and server hardware and IBM's high-performance shared-disk clustered file system, GPFS) with open source components like Linux, Samba and CTDB to deliver distributed storage solutions. GSA exports the clustered file system through industry standard protocols like CIFS, NFS, FTP and HTTP. All of the GSA nodes in the grid export all files of all file systems simultaneously.
Computer
A computer is a programmable machine designed to sequentially and automatically carry out a sequence of arithmetic or logical operations. The particular sequence of operations can be changed readily, allowing the computer to solve more than one kind of problem...
s, working together closely thus in many respects forming a single computer. The components of a cluster are commonly, but not always, connected to each other through fast local area network
Local area network
A local area network is a computer network that interconnects computers in a limited area such as a home, school, computer laboratory, or office building...
s. Clusters are usually deployed to improve performance and availability over that of a single computer, while typically being much more cost-effective than single computers of comparable speed or availability.
High-availability (HA) clusters
High-availability clusterHigh-availability cluster
High-availability clusters are groups of computers that support server applications that can be reliably utilized with a minimum of down-time. They operate by harnessing redundant computers in groups or clusters that provide continued service when system components fail...
s (also known as failover cluster) are implemented primarily for the purpose of improving the availability of services that the cluster provides. They operate by having redundant nodes
Node (networking)
In communication networks, a node is a connection point, either a redistribution point or a communication endpoint . The definition of a node depends on the network and protocol layer referred to...
, which are then used to provide service when system components fail. The most common size for an HA cluster is two nodes
Two-node cluster
A two-node cluster is the minimal high-availability cluster that can be built.Should one node fail , the other must acquire the resources being previously managed by the failed node, in order to re-enable access to these resources...
, which is the minimum requirement to provide redundancy. HA cluster implementations attempt to use redundancy of cluster components to eliminate single points of failure
Single point of failure
A single point of failure is a part of a system that, if it fails, will stop the entire system from working. They are undesirable in any system with a goal of high availability or reliability, be it a business practice, software application, or other industrial system.-Overview:Systems can be made...
.
There are commercial implementations of High-Availability clusters for many operating systems. The Linux-HA
Linux-HA
The Linux-HA project provides a high-availability solution for Linux, FreeBSD, OpenBSD, Solaris and Mac OS X which promotes reliability, availability, and serviceability ....
project is one commonly used free software
Free software
Free software, software libre or libre software is software that can be used, studied, and modified without restriction, and which can be copied and redistributed in modified or unmodified form either without restriction, or with restrictions that only ensure that further recipients can also do...
HA package for the Linux
Linux
Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
operating system.
Load-balancing clusters
Load-balancingLoad balancing (computing)
Load balancing is a computer networking methodology to distribute workload across multiple computers or a computer cluster, network links, central processing units, disk drives, or other resources, to achieve optimal resource utilization, maximize throughput, minimize response time, and avoid...
is when multiple computers are linked together to share computational workload or function as a single virtual computer. Logically, from the user side, they are multiple machines, but function as a single virtual machine. Requests initiated from the user are managed by, and distributed among, all the standalone computers to form a cluster. This results in balanced computational work among different machines, improving the performance of the cluster systems.
Compute clusters
Often clusters are used primarily for computational purposes, rather than handling IO-oriented operations such as web service or databases. For instance, a cluster might support computational simulationsComputer simulation
A computer simulation, a computer model, or a computational model is a computer program, or network of computers, that attempts to simulate an abstract model of a particular system...
of weather or vehicle crashes. The primary distinction within computer clusters is how tightly-coupled the individual nodes are. For instance, a single computer job may require frequent communication among nodes: this implies that the cluster shares a dedicated network, is densely located, and probably has homogenous nodes. This cluster design is usually referred to as Beowulf Cluster. The other extreme is where a computer job uses one or few nodes, and needs little or no inter-node communication. This latter category is sometimes called "Grid" computing. Tightly-coupled compute clusters are designed for work that might traditionally have been called "supercomputing". Middleware such as MPI (Message Passing Interface)
Message Passing Interface
Message Passing Interface is a standardized and portable message-passing system designed by a group of researchers from academia and industry to function on a wide variety of parallel computers...
or PVM (Parallel Virtual Machine) permits compute clustering programs to be portable to a wide variety of clusters.
Implementations
The TOP500TOP500
The TOP500 project ranks and details the 500 most powerful known computer systems in the world. The project was started in 1993 and publishes an updated list of the supercomputers twice a year...
organization's semiannual list of the 500 fastest computers usually includes many clusters. TOP500 is a collaboration between the University of Mannheim
University of Mannheim
The University of Mannheim is one of the younger German universities. It offers Bachelor, Master, and PhD degrees.The University is mainly located in Mannheim’s palace the largest baroque palace in Germany. The whole city center of Mannheim is aligned symmetrically to the palace.About 800 scholars...
, the University of Tennessee
University of Tennessee
The University of Tennessee is a public land-grant university headquartered at Knoxville, Tennessee, United States...
, and the National Energy Research Scientific Computing Center at Lawrence Berkeley National Laboratory
Lawrence Berkeley National Laboratory
The Lawrence Berkeley National Laboratory , is a U.S. Department of Energy national laboratory conducting unclassified scientific research. It is located on the grounds of the University of California, Berkeley, in the Berkeley Hills above the central campus...
. As of July 2011 the top supercomputer
Supercomputer
A supercomputer is a computer at the frontline of current processing capacity, particularly speed of calculation.Supercomputers are used for highly calculation-intensive tasks such as problems including quantum physics, weather forecasting, climate research, molecular modeling A supercomputer is a...
is the K computer
K computer
The K computer – named for the Japanese word , which stands for 10 quadrillion – is a supercomputer being produced by Fujitsu at the RIKEN Advanced Institute for Computational Science campus in Kobe, Japan. In June 2011, TOP500 ranked K the world's fastest supercomputer, with a rating...
in Kobe, Japan, with performance of 8.162 PFlops
FLOPS
In computing, FLOPS is a measure of a computer's performance, especially in fields of scientific calculations that make heavy use of floating-point calculations, similar to the older, simpler, instructions per second...
measured with the LINPACK
LINPACK
LINPACK is a software library for performing numerical linear algebra on digital computers. It was written in Fortran by Jack Dongarra, Jim Bunch, Cleve Moler, and Gilbert Stewart, and was intended for use on supercomputers in the 1970s and early 1980s...
benchmark.
Clustering can provide significant performance benefits versus price. The System X
System X (computing)
System X is a supercomputer assembled by Virginia Tech's Advanced Research Computing facility in the summer of 2003 that was originally composed of 1,100 Apple Power Mac G5 computers. System X ran at 12.25 Teraflops, , and was ranked #3 on November 16, 2003 and #280 in the July 2008 edition of...
supercomputer at Virginia Tech, the 28th most powerful supercomputer on Earth as of June 2006, is a 12.25 TFlops computer cluster of 1100 Apple XServe
Xserve
Xserve was a line of rack unit computers designed by Apple Inc. for use as servers. When the Xserve was introduced in 2002, it was Apple's first designated server hardware design since the Apple Network Server in 1996...
G5
PowerPC 970
The PowerPC 970, PowerPC 970FX, PowerPC 970GX, and PowerPC 970MP, are 64-bit Power Architecture processors from IBM introduced in 2002. When used in Apple Inc. machines, they were dubbed the PowerPC G5....
2.3 GHz dual-processor machines (4 GB
Gigabyte
The gigabyte is a multiple of the unit byte for digital information storage. The prefix giga means 109 in the International System of Units , therefore 1 gigabyte is...
RAM, 80 GB SATA
Serial ATA
Serial ATA is a computer bus interface for connecting host bus adapters to mass storage devices such as hard disk drives and optical drives...
HD
Hard disk
A hard disk drive is a non-volatile, random access digital magnetic data storage device. It features rotating rigid platters on a motor-driven spindle within a protective enclosure. Data is magnetically read from and written to the platter by read/write heads that float on a film of air above the...
) running Mac OS X
Mac OS X
Mac OS X is a series of Unix-based operating systems and graphical user interfaces developed, marketed, and sold by Apple Inc. Since 2002, has been included with all new Macintosh computer systems...
and using InfiniBand
InfiniBand
InfiniBand is a switched fabric communications link used in high-performance computing and enterprise data centers. Its features include high throughput, low latency, quality of service and failover, and it is designed to be scalable...
interconnect. The cluster initially consisted of Power Mac G5s; the rack-mountable XServes are denser than desktop Macs, reducing the aggregate size of the cluster. The total cost of the previous Power Mac system was $5.2 million, a tenth of the cost of slower mainframe computer
Mainframe computer
Mainframes are powerful computers used primarily by corporate and governmental organizations for critical applications, bulk data processing such as census, industry and consumer statistics, enterprise resource planning, and financial transaction processing.The term originally referred to the...
supercomputers. (The Power Mac G5s were sold off.)
The central concept of a Beowulf
Beowulf (computing)
A Beowulf cluster is a computer cluster of what are normally identical, commodity-grade computers networked into a small local area network with libraries and programs installed which allow processing to be shared among them...
cluster is the use of commercial off-the-shelf
Commercial off-the-shelf
In the United States, Commercially available Off-The-Shelf is a Federal Acquisition Regulation term defining a nondevelopmental item of supply that is both commercial and sold in substantial quantities in the commercial marketplace, and that can be procured or utilized under government contract...
(COTS) computers to produce a cost-effective alternative to a traditional supercomputer. One project that took this to an extreme was the Stone Soupercomputer
Stone Soupercomputer
The Stone Soupercomputer was a Beowulf-style computer cluster built at the US Oak Ridge National Laboratory in the late 1990s.-History:A group of lab employees including William W. Hargrove and Forrest M. Hoffman had applied for a grant to build a cluster in 1996, but it was rejected.Software was...
.
However it is worth noting that Flops (floating point operations per second), aren't always the best metric for supercomputer speed. Clusters can have very high Flops, but they cannot access all data in the cluster as a whole at once. Therefore clusters are excellent for parallel computation, but much poorer than traditional supercomputers at non-parallel computation.
JavaSpaces is a specification from Sun Microsystems
Sun Microsystems
Sun Microsystems, Inc. was a company that sold :computers, computer components, :computer software, and :information technology services. Sun was founded on February 24, 1982...
that enables clustering computers via a distributed shared memory
Distributed shared memory
Distributed Shared Memory , in Computer Architecture is a form of memory architecture where the memories can be addressed as one address space...
.
Consumer game consoles
Due to the increasing computing power of each generation of game consoles, a novel use has emerged where they are repurposed into High-performance computingHigh-performance computing
High-performance computing uses supercomputers and computer clusters to solve advanced computation problems. Today, computer systems approaching the teraflops-region are counted as HPC-computers.-Overview:...
(HPC) clusters. Some examples of game console clusters are Sony PlayStation clusters
Playstation 3 cluster
The considerable computing capability of the PlayStation 3's Cell microprocessors has raised interest in using multiple, networked PS3s for various tasks that require affordable high-performance computing.- PS3 Clusters :...
and Microsoft
Microsoft
Microsoft Corporation is an American public multinational corporation headquartered in Redmond, Washington, USA that develops, manufactures, licenses, and supports a wide range of products and services predominantly related to computing through its various product divisions...
Xbox clusters. It has been suggested on a news website that countries which are restricted from buying supercomputing technologies may be obtaining game systems to build computer clusters for military use.
History
The history of cluster computing is best captured by a footnote in Greg Pfister's In Search of Clusters: “Virtually every press release from DEC mentioning clusters says ‘DEC, who invented clusters...’. IBM did not invent them either. Customers invented clusters, as soon as they could not fit all their work on one computer, or needed a backup. The date of the first is unknown, but it would be surprising if it was not in the 1960s, or even late 1950s.”The formal engineering basis of cluster computing as a means of doing parallel work of any sort was arguably invented by Gene Amdahl
Gene Amdahl
Gene Myron Amdahl is a Norwegian-American computer architect and high-tech entrepreneur, chiefly known for his work on mainframe computers at IBM and later his own companies, especially Amdahl Corporation...
of IBM
IBM
International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
, who in 1967 published what has come to be regarded as the seminal paper on parallel processing: Amdahl's Law
Amdahl's law
Amdahl's law, also known as Amdahl's argument, is named after computer architect Gene Amdahl, and is used to find the maximum expected improvement to an overall system when only part of the system is improved...
. Amdahl's Law describes mathematically the speedup one can expect from parallelizing any given otherwise serially performed task on a parallel architecture. This article defined the engineering basis for both multiprocessor computing and cluster computing, where the primary differentiator is whether or not the interprocessor communications are supported "inside" the computer (on for example a customized internal communications bus or network) or "outside" the computer on a commodity network.
Consequently the history of early computer clusters is more or less directly tied into the history of early networks, as one of the primary motivations for the development of a network was to link computing resources, creating a de facto computer cluster. Packet switching
Packet switching
Packet switching is a digital networking communications method that groups all transmitted data – regardless of content, type, or structure – into suitably sized blocks, called packets. Packet switching features delivery of variable-bit-rate data streams over a shared network...
networks were conceptually invented by the RAND
RAND
RAND Corporation is a nonprofit global policy think tank first formed to offer research and analysis to the United States armed forces by Douglas Aircraft Company. It is currently financed by the U.S. government and private endowment, corporations including the healthcare industry, universities...
corporation in 1962. Using the concept of a packet switched network, the ARPANET
ARPANET
The Advanced Research Projects Agency Network , was the world's first operational packet switching network and the core network of a set that came to compose the global Internet...
project succeeded in creating in 1969 what was arguably the world's first commodity-network based computer cluster by linking four different computer centers (each of which was something of a "cluster" in its own right, but probably not a commodity cluster). The ARPANET project grew into the Internet
Internet
The Internet is a global system of interconnected computer networks that use the standard Internet protocol suite to serve billions of users worldwide...
—which can be thought of as "the mother of all computer clusters" (as the union of nearly all of the compute resources, including clusters, that happen to be connected). It also established the paradigm in use by all computer clusters in the world today—the use of packet-switched networks to perform interprocessor communications between processor (sets) located in otherwise disconnected frames.
The development of customer-built and research clusters proceeded hand in hand with that of both networks and the Unix
Unix
Unix is a multitasking, multi-user computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs, including Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy, and Joe Ossanna...
operating system from the early 1970s, as both TCP/IP and the Xerox PARC
Xerox PARC
PARC , formerly Xerox PARC, is a research and co-development company in Palo Alto, California, with a distinguished reputation for its contributions to information technology and hardware systems....
project created and formalized protocols for network-based communications. The Hydra operating system
Hydra (operating system)
HYDRA was an early capability-based, object-oriented, microkernel designed to support a wide range of possible operating systems to run on top of it...
was built for a cluster of DEC PDP-11
PDP-11
The PDP-11 was a series of 16-bit minicomputers sold by Digital Equipment Corporation from 1970 into the 1990s, one of a succession of products in the PDP series. The PDP-11 replaced the PDP-8 in many real-time applications, although both product lines lived in parallel for more than 10 years...
minicomputers called C.mmp
C.mmp
The C.mmp was an early MIMD multiprocessor system developed at Carnegie Mellon University by William Wulf . The notation C.mmp came from the PMS notation of Bell and Newell, where a CPU was designated as C and a variant was noted by the dot notation; mmp stood for Multi-Mini-ProcessorSixteen...
at Carnegie Mellon University
Carnegie Mellon University
Carnegie Mellon University is a private research university in Pittsburgh, Pennsylvania, United States....
in 1971. However, it was not until circa 1983 that the protocols and tools for easily doing remote job distribution and file sharing were defined (largely within the context of BSD
Berkeley Software Distribution
Berkeley Software Distribution is a Unix operating system derivative developed and distributed by the Computer Systems Research Group of the University of California, Berkeley, from 1977 to 1995...
Unix, as implemented by Sun Microsystems
Sun Microsystems
Sun Microsystems, Inc. was a company that sold :computers, computer components, :computer software, and :information technology services. Sun was founded on February 24, 1982...
) and hence became generally available commercially, along with a shared filesystem.
The first commercial clustering product was ARCnet
ARCNET
ARCNET is a local area network protocol, similar in purpose to Ethernet or Token Ring. ARCNET was the first widely available networking system for microcomputers and became popular in the 1980s for office automation tasks...
, developed by Datapoint
Datapoint
Datapoint Corporation, originally known as Computer Terminal Corporation , was a computer company based in San Antonio, Texas, United States. Founded in 1967 by Phil Ray and Gus Roche, its first products were, as the company's initial name suggests, computer terminals...
in 1977. ARCnet was not a commercial success and clustering per se did not really take off until Digital Equipment Corporation
Digital Equipment Corporation
Digital Equipment Corporation was a major American company in the computer industry and a leading vendor of computer systems, software and peripherals from the 1960s to the 1990s...
released their VAXcluster product in 1984 for the VAX/VMS operating system. The ARCnet and VAXcluster products not only supported parallel computing, but also shared file system
File system
A file system is a means to organize data expected to be retained after a program terminates by providing procedures to store, retrieve and update data, as well as manage the available space on the device which contain it. A file system organizes data in an efficient manner and is tuned to the...
s and peripheral
Peripheral
A peripheral is a device attached to a host computer, but not part of it, and is more or less dependent on the host. It expands the host's capabilities, but does not form part of the core computer architecture....
devices. The idea was to provide the advantages of parallel processing, while maintaining data reliability and uniqueness. VAXcluster, now VMScluster, is still available on OpenVMS
OpenVMS
OpenVMS , previously known as VAX-11/VMS, VAX/VMS or VMS, is a computer server operating system that runs on VAX, Alpha and Itanium-based families of computers. Contrary to what its name suggests, OpenVMS is not open source software; however, the source listings are available for purchase...
systems from HP
Hewlett-Packard
Hewlett-Packard Company or HP is an American multinational information technology corporation headquartered in Palo Alto, California, USA that provides products, technologies, softwares, solutions and services to consumers, small- and medium-sized businesses and large enterprises, including...
running on Alpha and Itanium systems.
Two other noteworthy early commercial clusters were the Tandem Himalaya
Tandem Computers
Tandem Computers, Inc. was the dominant manufacturer of fault-tolerant computer systems for ATM networks, banks, stock exchanges, telephone switching centers, and other similar commercial transaction processing applications requiring maximum uptime and zero data loss. The company was founded in...
(a circa 1994 high-availability product) and the IBM S/390 Parallel Sysplex (also circa 1994, primarily for business use).
No history of commodity computer clusters would be complete without noting the pivotal role played by the development of Parallel Virtual Machine
Parallel Virtual Machine
The Parallel Virtual Machine is a software tool for parallel networking of computers. It is designed to allow a network of heterogeneous Unix and/or Windows machines to be used as a single distributed parallel processor. Thus large computational problems can be solved more cost effectively by...
(PVM) software in 1989. This open source software based on TCP/IP communications enabled the instant creation of a virtual supercomputer—a high performance compute cluster—made out of any TCP/IP connected systems. Free form heterogeneous clusters built on top of this model rapidly achieved total throughput in FLOPS
FLOPS
In computing, FLOPS is a measure of a computer's performance, especially in fields of scientific calculations that make heavy use of floating-point calculations, similar to the older, simpler, instructions per second...
that greatly exceeded that available even with the most expensive "big iron
Big iron
Big iron, as the hacker's dictionary the Jargon File defines it, "refers to large, expensive, ultra-fast computers. It is used generally for number crunching supercomputers such as Crays, but can include more conventional big commercial IBM mainframes"....
" supercomputers. PVM and the advent of inexpensive networked PCs led, in 1993, to a NASA
NASA
The National Aeronautics and Space Administration is the agency of the United States government that is responsible for the nation's civilian space program and for aeronautics and aerospace research...
project to build supercomputers out of commodity clusters. In 1995 the Beowulf cluster—a cluster built on top of a commodity network for the specific purpose of "being a supercomputer" capable of performing tightly coupled parallel HPC computations—was invented, which spurred the independent development of grid computing
Grid computing
Grid computing is a term referring to the combination of computer resources from multiple administrative domains to reach a common goal. The grid can be thought of as a distributed system with non-interactive workloads that involve a large number of files...
as a named entity, although Grid-style clustering had been around at least as long as the Unix
Unix
Unix is a multitasking, multi-user computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs, including Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy, and Joe Ossanna...
operating system and the Arpanet, whether or not it, or the clusters that used it, were named.
Technologies
MPIMessage Passing Interface
Message Passing Interface is a standardized and portable message-passing system designed by a group of researchers from academia and industry to function on a wide variety of parallel computers...
is a widely-available communications library that enables parallel programs to be written in C
C (programming language)
C is a general-purpose computer programming language developed between 1969 and 1973 by Dennis Ritchie at the Bell Telephone Laboratories for use with the Unix operating system....
, Fortran
Fortran
Fortran is a general-purpose, procedural, imperative programming language that is especially suited to numeric computation and scientific computing...
, Python
Python (programming language)
Python is a general-purpose, high-level programming language whose design philosophy emphasizes code readability. Python claims to "[combine] remarkable power with very clear syntax", and its standard library is large and comprehensive...
, OCaml, and many other programming languages.
The GNU/Linux world supports various cluster software; for application clustering, there is Beowulf
Beowulf (computing)
A Beowulf cluster is a computer cluster of what are normally identical, commodity-grade computers networked into a small local area network with libraries and programs installed which allow processing to be shared among them...
, distcc
Distcc
In software development, distcc is a tool for speeding up compilation of source code by using distributed computing over a computer network. With the right configuration, distcc can dramatically reduce a project's compilation time ....
, and MPICH
MPICH
MPICH is a freely available, portable implementation of MPI, a standard for message-passing for distributed-memory applications used in parallel computing...
. Linux Virtual Server
Linux Virtual Server
Linux Virtual Server is an advanced load balancing solution for Linux systems. It is an open source project started by Wensong Zhang in May 1998...
, Linux-HA
Linux-HA
The Linux-HA project provides a high-availability solution for Linux, FreeBSD, OpenBSD, Solaris and Mac OS X which promotes reliability, availability, and serviceability ....
- director-based clusters that allow incoming requests for services to be distributed across multiple cluster nodes. MOSIX
MOSIX
MOSIX is a distributed operating system. Although early versions were based on older UNIX systems, since 1999 it focuses on Linux clusters and grids...
, openMosix
OpenMosix
openMosix was a free cluster management system that provided single-system image capabilities, e.g. automatic work distribution among nodes. It allowed program processes to migrate to machines in the node's network that would be able to run that process faster...
, Kerrighed
Kerrighed
Kerrighed is an open source single-system image cluster software project. The project started in October 1998 at the Paris research group The French National Institute for Research in Computer Science and Control.-Background:...
, OpenSSI
OpenSSI
OpenSSI is an open source single-system image clustering system. It allows a collection of computers to be treated as one large system, allowing applications running on any one machine access to the resources of all the machines in the cluster....
are full-blown clusters integrated into the kernel that provide for automatic process migration among homogeneous nodes. OpenSSI, openMosix and Kerrighed are single-system image
Single-system image
In distributed computing, a single system image cluster is a cluster of machines that appears to be one single system. The concept is often considered synonymous with that of a distributed operating system, but a single image may be presented for more limited purposes, just job scheduling for...
implementations.
Microsoft Windows
Microsoft Windows
Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
Compute Cluster Server 2003 based on the Windows Server
Windows Server
Windows Server is a brand name for a group of server operating systems released by Microsoft Corporation. All are part of Microsoft Servers.- Members :This brand includes the following software:* Windows 2000 Server* Windows Server 2003...
platform provides pieces for High Performance Computing like the Job Scheduler, MSMPI library and management tools. NCSA
National Center for Supercomputing Applications
The National Center for Supercomputing Applications is an American state-federal partnership to develop and deploy national-scale cyberinfrastructure that advances science and engineering. NCSA operates as a unit of the University of Illinois at Urbana-Champaign but it provides high-performance...
's recently installed Lincoln is a cluster of 450 Dell PowerEdge 1855 blade servers running Windows Compute Cluster Server 2003. This cluster debuted at #130 on the Top500
TOP500
The TOP500 project ranks and details the 500 most powerful known computer systems in the world. The project was started in 1993 and publishes an updated list of the supercomputers twice a year...
list in June 2006.
gridMathematica
GridMathematica
gridMathematica is a software product sold by Wolfram Research which extends the parallel processing capabilities of its main product Mathematica.- Features :...
provides distributed computations over clusters including data analysis, computer algebra and 3D visualization. It can make use of other technologies such as Altair PBS Professional, Microsoft Windows Compute Cluster Server, Platform LSF and Sun Grid Engine
Sun Grid Engine
Oracle Grid Engine, previously known as Sun Grid Engine , previously known as CODINE or GRD , is an open source batch-queuing system, developed and supported by Sun Microsystems...
.
gLite
GLite
gLite is a middleware computer software project for grid computing used by the CERN LHC experiments and other scientific domains. It was implemented by collaborative efforts of more than 80 people in 12 different academic and industrial research centers in Europe...
is a set of middleware technologies created by the Enabling Grids for E-sciencE (EGEE) project.
Another example of consumer game products being added to high-performance computing is the Nvidia Tesla Personal Supercomputer
Nvidia Tesla Personal Supercomputer
The Tesla Personal Supercomputer is a desktop computer that is backed by Nvidia and built by Dell, Lenovo and other companies...
workstation, which gets its processing power by harnessing the power of multiple graphics accelerator processor chips.
Algorithmic skeleton
Algorithmic skeleton
In computing, algorithmic skeletons are a high-level parallel programming model for parallel and distributed computing....
s are a high-level parallel programming model for parallel and distributed computing which take advantage of common programming patterns to hide the complexity of parallel and distributed applications. Starting from a basic set of patterns (skeletons), more complex patterns can be built by combining the basic ones.
Global Storage Architecture (GSA)—a highly scalable cloud based NAS solution—combines proprietary IBM HPC technology (storage and server hardware and IBM's high-performance shared-disk clustered file system, GPFS) with open source components like Linux, Samba and CTDB to deliver distributed storage solutions. GSA exports the clustered file system through industry standard protocols like CIFS, NFS, FTP and HTTP. All of the GSA nodes in the grid export all files of all file systems simultaneously.
See also
- Grid computingGrid computingGrid computing is a term referring to the combination of computer resources from multiple administrative domains to reach a common goal. The grid can be thought of as a distributed system with non-interactive workloads that involve a large number of files...
- BotnetBotnetA botnet is a collection of compromised computers connected to the Internet. Termed "bots," they are generally used for malicious purposes. When a computer becomes compromised, it becomes a part of a botnet...
- DEGIMA (computer cluster)DEGIMA (computer cluster)The DEGIMA is a high performance computer cluster used for hierarchical N-body simulations at the Nagasaki Advanced Computing Center, Nagasaki University....
- Computer cluster in virtual machinesComputer cluster in virtual machinesComputer clusters run usually on physical computers. With the virtualization approach there are new possibilities of setting up different kinds of clusters.There are different categorizations of such clusters, depending...
- Clustered file systemClustered file systemA clustered file system is a file system which is shared by being simultaneously mounted on multiple servers. There are several approaches to clustering, most of which do not employ a clustered file system...
- Distributed data storeDistributed data storeA distributed data store is a blurred concept and means either a distributed database where users store their information on a number of nodes, or a network in which a user stores their information on a number of peer network nodes ....
- Flash mob computingFlash mob computingFlash mob computing is a temporary ad-hoc computer cluster running specific software to coordinate the individual computers into one single supercomputer...
- GPU clusterGpu clusterA GPU cluster is a computer cluster in which each node is equipped with a Graphics Processing Unit . By harnessing the computational power of modern GPUs via General-Purpose Computing on Graphics Processing Units , very fast calculations can be performed with a GPU cluster.- Hardware :The hardware...
- HP ServiceGuard
- Peer-to-peerPeer-to-peerPeer-to-peer computing or networking is a distributed application architecture that partitions tasks or workloads among peers. Peers are equally privileged, equipotent participants in the application...
- Red Hat Cluster SuiteRed Hat Cluster SuiteThe Red Hat cluster suite includes software to create a high availability and load balancing cluster, it currently does not contain functionality for distributed computing. The cluster suite itself contains two products. Both can be used on the same system although this use case is unlikely. Both...
- RoS (computing)RoS (computing)RoS is the abbreviation for the computing term Request of Service . It is used to refer to a request for a specific service or response from a dormant application running on a computer cluster....
- Server farmServer farmA server farm or server cluster is a collection of computer servers usually maintained by an enterprise to accomplish server needs far beyond the capability of one machine. Server farms often have backup servers, which can take over the function of primary servers in the event of a primary server...
- Compile farmCompile farmA compile farm is a server farm, a collection of one or more servers, which has been set up to compile computer programs remotely for various reasons...
- Single system image
- Solaris Cluster
- Symmetric multiprocessingSymmetric multiprocessingIn computing, symmetric multiprocessing involves a multiprocessor computer hardware architecture where two or more identical processors are connected to a single shared main memory and are controlled by a single OS instance. Most common multiprocessor systems today use an SMP architecture...
- Two-node clusterTwo-node clusterA two-node cluster is the minimal high-availability cluster that can be built.Should one node fail , the other must acquire the resources being previously managed by the failed node, in order to re-enable access to these resources...
- Veritas Cluster ServerVeritas Cluster ServerVeritas Cluster Server is a High-availability cluster software, for Unix, Linux and Microsoft Windows computer systems, created by Veritas Software...
Further reading
- Mark Baker, et al., Cluster Computing White Paper http://arxiv.org/abs/cs/0004014, 11 Jan 2001.
- Karl Kopper: The Linux Enterprise Cluster: Build a Highly Available Cluster with Commodity Hardware and Free Software, No Starch Press, ISBN 1-59327-036-4
- Robert W. Lucke: Building Clustered Linux Systems, Prentice Hall, 2005, ISBN 0-13-144853-6
- Evan Marcus, Hal Stern: Blueprints for High Availability: Designing Resilient Distributed Systems, John Wiley & Sons, ISBN 0-471-35601-8
- Greg Pfister: In Search of Clusters, Prentice Hall, ISBN 0-13-899709-8
- Rajkumar Buyya (editor): High Performance Cluster Computing: Architectures and Systems, Volume 1, ISBN 0-13-013784-7, Prentice Hall, NJ, USA, 1999.
- Rajkumar Buyya (editor): High Performance Cluster Computing: Programming and Applications, Volume 2, ISBN 0-13-013785-5, Prentice Hall, NJ, USA, 1999.