Gearman
Encyclopedia
Gearman is an open source
application framework originally written in Perl
by Brad Fitzpatrick
. Brian Aker
and Eric Day rewrote the framework in C
. Gearman is designed to distribute appropriate computer tasks to multiple computers, so large tasks can be done more quickly. In some cases, load balancing
rather than raw speed may be the main goal; a Web server, for instance, could use Gearman to send tasks for which it is not optimized to another computer (which may be running on a different architecture
, using another operating system
, or loaded with a computer language better suited to a particular operation).
, job server
, or worker. A worker machine can be assigned multiple instances of the worker role, which allows more powerful computers to complete more portions of a given task. Tasks originate on a client, are transmitted from the client to the job server, and performed on one or more workers. The completed task's output is then returned, again by way of the job server, to the client where the task originated. Gearman is conceptually related to MapReduce
; Gearman handles MapReduce by allowing worker nodes to map out work to other workers, with the original worker acting as the reducer.
To mitigate the damage that would be done if a job server (or its network connection) were to fail, clients can be configured with more than one assigned job server; if the first assigned job server fails, another can be transparently substituted.
Gearman implements a protocol that consists of binary packets containing requests and responses; this protocol defines the structure of messages passing between the three parts of a Gearman implementation. By default, the Gearman protocol uses TCP
port 4730.
It previously operated on port 7003, but this conflicted with the AFS
port range and the new port (4730) was assigned by IANA
.
The name "Gearman" was chosen as an anagram
for "Manager," "since it dispatches jobs to be done, but does not do anything useful itself."
Open source
The term open source describes practices in production and development that promote access to the end product's source materials. Some consider open source a philosophy, others consider it a pragmatic methodology...
application framework originally written in Perl
Perl
Perl is a high-level, general-purpose, interpreted, dynamic programming language. Perl was originally developed by Larry Wall in 1987 as a general-purpose Unix scripting language to make report processing easier. Since then, it has undergone many changes and revisions and become widely popular...
by Brad Fitzpatrick
Brad Fitzpatrick
Bradley Joseph "Brad" Fitzpatrick , is an American programmer. He is best known as the creator of LiveJournal and is the author of a variety of free software projects such as memcached....
. Brian Aker
Brian Aker
Brian Aker, born August 4, 1972 in Lexington, Kentucky, USA is an open-source hacker who has worked on various Apache modules, the Slash system, and numerous storage engines for the MySQL database. Aker was Director of Architecture at MySQL AB until it was acquired by Sun Microsystems. He led...
and Eric Day rewrote the framework in C
C (programming language)
C is a general-purpose computer programming language developed between 1969 and 1973 by Dennis Ritchie at the Bell Telephone Laboratories for use with the Unix operating system....
. Gearman is designed to distribute appropriate computer tasks to multiple computers, so large tasks can be done more quickly. In some cases, load balancing
Load balancing (computing)
Load balancing is a computer networking methodology to distribute workload across multiple computers or a computer cluster, network links, central processing units, disk drives, or other resources, to achieve optimal resource utilization, maximize throughput, minimize response time, and avoid...
rather than raw speed may be the main goal; a Web server, for instance, could use Gearman to send tasks for which it is not optimized to another computer (which may be running on a different architecture
Computer architecture
In computer science and engineering, computer architecture is the practical art of selecting and interconnecting hardware components to create computers that meet functional, performance and cost goals and the formal modelling of those systems....
, using another operating system
Operating system
An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
, or loaded with a computer language better suited to a particular operation).
How Gearman Works
Gearman assigns each involved computer a role as clientClient (computing)
A client is an application or system that accesses a service made available by a server. The server is often on another computer system, in which case the client accesses the service by way of a network....
, job server
Server (computing)
In the context of client-server architecture, a server is a computer program running to serve the requests of other programs, the "clients". Thus, the "server" performs some computational task on behalf of "clients"...
, or worker. A worker machine can be assigned multiple instances of the worker role, which allows more powerful computers to complete more portions of a given task. Tasks originate on a client, are transmitted from the client to the job server, and performed on one or more workers. The completed task's output is then returned, again by way of the job server, to the client where the task originated. Gearman is conceptually related to MapReduce
MapReduce
MapReduce is a software framework introduced by Google in 2004 to support distributed computing on large data sets on clusters of computers. Parts of the framework are patented in some countries....
; Gearman handles MapReduce by allowing worker nodes to map out work to other workers, with the original worker acting as the reducer.
To mitigate the damage that would be done if a job server (or its network connection) were to fail, clients can be configured with more than one assigned job server; if the first assigned job server fails, another can be transparently substituted.
Gearman implements a protocol that consists of binary packets containing requests and responses; this protocol defines the structure of messages passing between the three parts of a Gearman implementation. By default, the Gearman protocol uses TCP
Transmission Control Protocol
The Transmission Control Protocol is one of the core protocols of the Internet Protocol Suite. TCP is one of the two original components of the suite, complementing the Internet Protocol , and therefore the entire suite is commonly referred to as TCP/IP...
port 4730.
It previously operated on port 7003, but this conflicted with the AFS
Andrew file system
The Andrew File System is a distributed networked file system which uses a set of trusted servers to present a homogeneous, location-transparent file name space to all the client workstations. It was developed by Carnegie Mellon University as part of the Andrew Project. It is named after Andrew...
port range and the new port (4730) was assigned by IANA
Internet Assigned Numbers Authority
The Internet Assigned Numbers Authority is the entity that oversees global IP address allocation, autonomous system number allocation, root zone management in the Domain Name System , media types, and other Internet Protocol-related symbols and numbers...
.
The name "Gearman" was chosen as an anagram
Anagram
An anagram is a type of word play, the result of rearranging the letters of a word or phrase to produce a new word or phrase, using all the original letters exactly once; e.g., orchestra = carthorse, A decimal point = I'm a dot in place, Tom Marvolo Riddle = I am Lord Voldemort. Someone who...
for "Manager," "since it dispatches jobs to be done, but does not do anything useful itself."
External links
- http://danga.com/words/2007_06_usenix/usenix.pdf
- http://www.gearman.org/doku.php
- http://lists.danga.com/pipermail/gearman/2008-April/000076.html
- http://krow.livejournal.com/628025.html