DiaGrid (distributed computing network)
Encyclopedia
DiaGrid is a large high-throughput distributed research computing network utilizing the Condor system and centered at Purdue University
in West Lafayette, Indiana
. DiaGrid received a 2009 Campus Technology Innovators award from Campus Technology magazine and was employed at the SC09 supercomputing conference in Portland, Ore., to capture nearly 150 days of compute time for science jobs.
The grid, a partnership with Indiana University
, Indiana State University
, the University of Notre Dame
, the University of Louisville
, the University of Wisconsin, Purdue's Calumet
and North Central campuses, and Indiana University-Purdue University Fort Wayne
, is designed to accommodate computers at other campuses across the nation as new members join. In 2009, it included nearly 30,000 processors. The Purdue portion of the pool, named BoilerGrid, is the largest academic system of its kind. In 2008, users tapped DiaGrid’s 177 teraflops of computing power for 16 million compute hours, up from 10 million in 2007. Also in 2008, DiaGrid ran nearly 12 million jobs, 2 million more than in 2007. Since 2006, the system has provided nearly 50 million compute hours to more than 300 researchers.
Through Condor, developed at the University of Wisconsin, DiaGrid harvests and manages computing cycles from idle or underused high performance computing cluster nodes, machines in campus computer and other labs and office computers. Whenever a local user or scheduled job needs a given machine, the Condor job is stopped and sent to another Condor node as soon as possible. While this "opportunistic" model limits the ability to do parallel processing and communications, a Condor pool can provide smaller, serial jobs vast numbers of cycles in a very short amount of time. Condor—and by extension, DiaGrid—is designed for high-throughput computing and is excellent for parameter sweeps, Monte Carlo simulation, or nearly any serial application. Some classes of parallel jobs (master-worker) may be run effectively via Condor as well.
DiaGrid is managed by Information Technology at Purdue (ITaP), the central information technology organization at Purdue's West Lafayette campus, which also operates the Steele
and Coates
supercomputers. To pool computational resources spread around the state, the grid takes advantage of I-Light, the high-speed fiber-optic state network connecting Indiana campuses to each other, the Internet and national research networks such as the Internet2
and National LambdaRail.
DiaGrid and BoilerGrid have been used by researchers at Purdue and elsewhere for a variety of purposes, such as imaging the structure of viruses at near-atomic resolutions, simulating the early stages of the Solar System's formation, projecting the reliability of Indiana's electrical supply, modeling the spread of water pollutants, and identifying millions of potential new forms of zeolites, silicate minerals
widely used to catalyze chemical reactions on an industrial scale. DiaGrid provides computational resources to researchers on both the Open Science Grid
and the TeraGrid
.
Purdue University
Purdue University, located in West Lafayette, Indiana, U.S., is the flagship university of the six-campus Purdue University system. Purdue was founded on May 6, 1869, as a land-grant university when the Indiana General Assembly, taking advantage of the Morrill Act, accepted a donation of land and...
in West Lafayette, Indiana
West Lafayette, Indiana
As of the census of 2010, there were 29,596 people, 12,591 households, and 3,588 families residing in the city. The population density was 5,381.1 people per square mile . The racial makeup of the city was 74.3% White, 17.3% Asian, 2.7% African American, 0.16% Native American, 0.03% Pacific...
. DiaGrid received a 2009 Campus Technology Innovators award from Campus Technology magazine and was employed at the SC09 supercomputing conference in Portland, Ore., to capture nearly 150 days of compute time for science jobs.
The grid, a partnership with Indiana University
Indiana University
Indiana University is a multi-campus public university system in the state of Indiana, United States. Indiana University has a combined student body of more than 100,000 students, including approximately 42,000 students enrolled at the Indiana University Bloomington campus and approximately 37,000...
, Indiana State University
Indiana State University
Indiana State University is a public university located in Terre Haute, Indiana, United States.The Princeton Review has named Indiana State as one of the "Best in the Midwest" seven years running, and the College of Education's Graduate Program was recently named as a 'Top 100' by U.S...
, the University of Notre Dame
University of Notre Dame
The University of Notre Dame du Lac is a Catholic research university located in Notre Dame, an unincorporated community north of the city of South Bend, in St. Joseph County, Indiana, United States...
, the University of Louisville
University of Louisville
The University of Louisville is a public university in Louisville, Kentucky. When founded in 1798, it was the first city-owned public university in the United States and one of the first universities chartered west of the Allegheny Mountains. The university is mandated by the Kentucky General...
, the University of Wisconsin, Purdue's Calumet
Purdue University Calumet
Purdue University Calumet is a school within the Purdue University system that is located in Hammond, Indiana in the Northwest Indiana portion of the Chicago metropolitan area.-Schools:Purdue University Calumet is academically organized into six schools:...
and North Central campuses, and Indiana University-Purdue University Fort Wayne
Indiana University-Purdue University Fort Wayne
Indiana University – Purdue University Fort Wayne is the largest university in northern Indiana, United States, offering more than 200 Indiana University and Purdue University degrees and certificates. Since 1968, IPFW has conferred nearly 8,800 master’s degrees, 27,000 bachelor’s degrees, more...
, is designed to accommodate computers at other campuses across the nation as new members join. In 2009, it included nearly 30,000 processors. The Purdue portion of the pool, named BoilerGrid, is the largest academic system of its kind. In 2008, users tapped DiaGrid’s 177 teraflops of computing power for 16 million compute hours, up from 10 million in 2007. Also in 2008, DiaGrid ran nearly 12 million jobs, 2 million more than in 2007. Since 2006, the system has provided nearly 50 million compute hours to more than 300 researchers.
Through Condor, developed at the University of Wisconsin, DiaGrid harvests and manages computing cycles from idle or underused high performance computing cluster nodes, machines in campus computer and other labs and office computers. Whenever a local user or scheduled job needs a given machine, the Condor job is stopped and sent to another Condor node as soon as possible. While this "opportunistic" model limits the ability to do parallel processing and communications, a Condor pool can provide smaller, serial jobs vast numbers of cycles in a very short amount of time. Condor—and by extension, DiaGrid—is designed for high-throughput computing and is excellent for parameter sweeps, Monte Carlo simulation, or nearly any serial application. Some classes of parallel jobs (master-worker) may be run effectively via Condor as well.
DiaGrid is managed by Information Technology at Purdue (ITaP), the central information technology organization at Purdue's West Lafayette campus, which also operates the Steele
Steele (supercomputer)
Steele is a supercomputer at Purdue University.The cluster is the second largest campus supercomputer in the Big Ten not a part of a national center and was the largest when built. Steele is made up of 893 Dell dual quad-core computer nodes and has a theoretical peak performance of more than 60...
and Coates
Coates (supercomputer)
Coates is a supercomputer installed at Purdue University on July 21, 2009. The high-performance computing cluster is operated by Information Technology at Purdue , the university's central information technology organization...
supercomputers. To pool computational resources spread around the state, the grid takes advantage of I-Light, the high-speed fiber-optic state network connecting Indiana campuses to each other, the Internet and national research networks such as the Internet2
Internet2
Internet2 is an advanced not-for-profit US networking consortium led by members from the research and education communities, industry, and government....
and National LambdaRail.
DiaGrid and BoilerGrid have been used by researchers at Purdue and elsewhere for a variety of purposes, such as imaging the structure of viruses at near-atomic resolutions, simulating the early stages of the Solar System's formation, projecting the reliability of Indiana's electrical supply, modeling the spread of water pollutants, and identifying millions of potential new forms of zeolites, silicate minerals
Silicate minerals
The silicate minerals make up the largest and most important class of rock-forming minerals, constituting approximately 90 percent of the crust of the Earth. They are classified based on the structure of their silicate group...
widely used to catalyze chemical reactions on an industrial scale. DiaGrid provides computational resources to researchers on both the Open Science Grid
Open Science Grid Consortium
The Open Science Grid Consortium is an organization that administers a worldwide grid of technological resources called the Open Science Grid, which facilitates distributed computing for scientific research...
and the TeraGrid
TeraGrid
TeraGrid is an e-Science grid computing infrastructure combining resources at eleven partner sites. The project started in 2001 and operated from 2004 through 2011....
.
External links
- http://www.dia-grid.org
- http://campustechnology.com/articles/2009/07/22/campus-technology-innovators-awards-2009-high-performance-computing.aspx
- http://markets.hpcwire.com/taborcomm.hpcwire/?GUID=10770002&Page=MediaViewer&ChannelID=3198
- http://cloudcomputing.sys-con.com/node/1185883
- http://www.itap.purdue.edu/newsroom/detail.cfm?NewsID=2054
- http://news.uns.purdue.edu/x/2008b/081118McCartneyPool.html
- http://www.rcac.purdue.edu/boilergrid/index.cfm
- http://www.cs.wisc.edu/condor/
- http://www.itap.purdue.edu/