SciNet Consortium
Encyclopedia
University of Toronto
The University of Toronto is a public research university in Toronto, Ontario, Canada, situated on the grounds that surround Queen's Park. It was founded by royal charter in 1827 as King's College, the first institution of higher learning in Upper Canada...
and affiliated Ontario hospitals. It has received funding from both the federal and provincial government, Faculties at the University of Toronto, and affiliated hospitals.
It is one of seven regional High Performance Computing consortia across Canada and is the most powerful university HPC system outside of the US. As of November 2008, the partially constructed systems were already ranked at #53 on the Top 500 List. It is also the only Canadian HPC in top one hundred of the list. The parallel systems were anticipated to rank around #50 and #25 upon completion in June 2009. The TOP500 list for June 2009 ranked the GPC iDataplex system at #16, while the TCS dropped to #80.
The SciNet offices are based on the St. George street campus, however, to accommodate the large floor space and power needs, the datacentre facility is housed in a warehouse about 30 km north of campus in Vaughan
Vaughan
Vaughan is a city in York Region north of Toronto, Ontario, Canada. Vaughan is the fastest growing municipality in Canada achieving a population growth rate of 80.2% between 1996–2006, according to Statistics Canada having nearly doubled in population since 1991. Vaughan is located in Southern...
.
At the core of SciNet research are six key areas of study: Astronomy and Astrophysics, Aerospace and Biomedical Engineering, High Energy Particle Physics, Integrative Computational Biology, Planetary Physics, and Theoretical Chemical Physics.
History
SciNet was initially formed in the fall of 2004 following an agreement between the Canadian high-performance computingHigh-performance computing
High-performance computing uses supercomputers and computer clusters to solve advanced computation problems. Today, computer systems approaching the teraflops-region are counted as HPC-computers.-Overview:...
community to develop a response to the newly created National Platform Fund. The community felt that funding from the NPF would enable the development of a collective national capability in HPC. The Canadian HPC community was successful in its NPF proposal and SciNet was awarded a portion of that funding.
SciNet finalized its contract with IBM
IBM
International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
to build the system in July 2008 and the formal announcement was August 14th, 2008.
Specifications
The SciNet has two compute clusters that are optimized for different types of computing:- One is a Tighly-coupled Capability SystemMultiprocessingMultiprocessing is the use of two or more central processing units within a single computer system. The term also refers to the ability of a system to support more than one processor and/or the ability to allocate tasks between them...
(TCS) which has 104 POWER6POWER6The POWER6 is a microprocessor developed by IBM that implemented the Power ISA v.2.03. When it became available in systems in 2007, it succeeded the POWER5+ as IBM's flagship Power microprocessor...
nodes, wherein each node contains 32 cores (4.7 GHz) and 128 GiB RAM. Theoretical peak is 60 TFlops and 14 TiB of RAM. - The second is the General Purpose Cluster (GPC) which has 30,240 cores of an Intel Nehalem-based processor, each with 2GiB RAM. Theoretical peak is 306 TFlops and 60 TiB of RAM.
General Purpose Cluster
The General Purpose Cluster consists of 3,780 IBM System xIBM System x
The IBM System x computers form a sub-brand of International Business Machines System brand servers...
iDataPlex dx360 M3 nodes, each with 2 quad-core Intel Nehalem (Xeon
Xeon
The Xeon is a brand of multiprocessing- or multi-socket-capable x86 microprocessors from Intel Corporation targeted at the non-consumer server, workstation and embedded system markets.-Overview:...
5540) processor running at 2.53 GHz, totaling 30,240 cores in 45 racks. (An iDataPlex rack cabinet provides 84 rack unit
Rack unit
A rack unit or U is a unit of measure used to describe the height of equipment intended for mounting in a 19-inch rack or a 23-inch rack...
s of space.) All nodes are connected with Gigabit Ethernet
Gigabit Ethernet
Gigabit Ethernet is a term describing various technologies for transmitting Ethernet frames at a rate of a gigabit per second , as defined by the IEEE 802.3-2008 standard. It came into use beginning in 1999, gradually supplanting Fast Ethernet in wired local networks where it performed...
, and DDR InfiniBand
InfiniBand
InfiniBand is a switched fabric communications link used in high-performance computing and enterprise data centers. Its features include high throughput, low latency, quality of service and failover, and it is designed to be scalable...
is used additionally in 864 nodes to provide high-speed and low-latency communication for message passing applications. The computer will use the same amount of energy which could be used to power four thousand homes, and is water-cooled. To utilize the cold Canadian climate, the system is notified when external air goes below a certain temperature, at which time the chiller switches over to use the "free-air" cooling available. SciNet, IBM
IBM
International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
Corp and Compute Canada collaborated on the supercomputer venture.
The new computer system at U of T's SciNet is the largest Intel processor based IBM installation globally.
Data center
The computer room itself is 3000 square feet (278.7 m²) on a raised floor. IT has a 735 ton chiller and cooling towers for “free-air” cooling. A significant research area that will be addressed using the SciNet machines is that of climate change and global warming, which is why creating one of the greenest datacenters in the world was of key importance in this project. A traditional datacenter generally uses 33% of the energy going into its centre for cooling and other non-computing power consumption; however, SciNet and IBM have successfully created a centre that uses less than 20% towards these areas.Partners
Founding Institution- University of TorontoUniversity of TorontoThe University of Toronto is a public research university in Toronto, Ontario, Canada, situated on the grounds that surround Queen's Park. It was founded by royal charter in 1827 as King's College, the first institution of higher learning in Upper Canada...
Affiliated Hospitals
- Hospital for Sick KidsHospital for Sick ChildrenThe Hospital for Sick Children – is a major paediatric centre for the Greater Toronto Area, serving patients up to age 18. Located on University Avenue in Downtown Toronto, SickKids is part of the city’s Discovery District, a critical mass of scientists and entrepreneurs who are focused on...
- Mount Sinai HospitalMount Sinai Hospital (Toronto)Mount Sinai Hospital is a hospital in Toronto, Ontario, Canada. Although it is physically linked by bridges and tunnels to two University Health Network hospitals , Mount Sinai is an independently operated facility...
- Ontario Institute for Cancer ResearchOntario Institute for Cancer ResearchThe Ontario Institute for Cancer Research is a not-for-profit organization based in Toronto, Ontario, Canada that focuses on the prevention, early detection, diagnosis and treatment of cancer. OICR intends to make Ontario more effective in knowledge transfer and commercialization while maximizing...
- Baycrest Centre for Geriatric CareBaycrest Centre for Geriatric CareBaycrest is a research and education hospital on Bathurst Street in Toronto, Ontario, Canada.It was founded in 1918 as the Toronto Jewish Old Folks Home in a semi-detached house on Cecil Street in downtown Toronto. In 1968, the new Jewish Home for the Aged opened at Baycrest's present location in...
Common uses
The U of T supercomputer which can perform 300 trillion calculations per second will be used for highly calculation-intensive tasks such as problems involving quantum mechanical physics, weather forecastingWeather forecasting
Weather forecasting is the application of science and technology to predict the state of the atmosphere for a given location. Human beings have attempted to predict the weather informally for millennia, and formally since the nineteenth century...
, climate research, climate change models, molecular modeling
Computational chemistry
Computational chemistry is a branch of chemistry that uses principles of computer science to assist in solving chemical problems. It uses the results of theoretical chemistry, incorporated into efficient computer programs, to calculate the structures and properties of molecules and solids...
(computing the structures and properties of chemical compounds, biological macromolecules, polymers, and crystals), physical simulations (such as simulation of the big bang
Big Bang
The Big Bang theory is the prevailing cosmological model that explains the early development of the Universe. According to the Big Bang theory, the Universe was once in an extremely hot and dense state which expanded rapidly. This rapid expansion caused the young Universe to cool and resulted in...
theory in conjunction with the Large Hadron Collider
Large Hadron Collider
The Large Hadron Collider is the world's largest and highest-energy particle accelerator. It is expected to address some of the most fundamental questions of physics, advancing the understanding of the deepest laws of nature....
(LHC) in CERN
CERN
The European Organization for Nuclear Research , known as CERN , is an international organization whose purpose is to operate the world's largest particle physics laboratory, which is situated in the northwest suburbs of Geneva on the Franco–Swiss border...
, Geneva
Geneva
Geneva In the national languages of Switzerland the city is known as Genf , Ginevra and Genevra is the second-most-populous city in Switzerland and is the most populous city of Romandie, the French-speaking part of Switzerland...
which will produce cataclysmic conditions that will mimic the beginning of time,and the U of T supercomputer will examine the particle collisions. Part of the collaboration with LHC will be to answer questions about why matter has mass and what comprises the Universe's mass? Additional areas of research will be models of greenhouse gas-induced global warming and the effect on Arctic sea ice. The international ATLAS project will be explored by the new supercomputer to discover forces which govern the universe.
External links
- SciNet HPC Consortium – Official website
- Top 500 List – November 2008
- UofT’s Official Announcement of IBM contract – August 2008