Cielo: NNSA Capability Supercomputer
Cielo’s capabilities were designed and developed jointly by LANL and SNL under the Advanced Computing at Extreme Scale (ACES) partnership. The system is physically located at LANL in the Nicholas Metropolis Center for Modeling and Simulation.
Asteroid mitigation simulation
The facility is designed to scale with the increasingly powerful compute resources to deliver the required end-to-end services for users. Within the center are networking, archival storage, visualization servers, global file systems, and system software, all enhanced to support Cielo’s size and architecture. The ACES partnership is responsible for the support of Cielo, including interactions with the vendor.
This simulation of an asteroid mitigation by Bob Weaver (LANL) has had over 86,000 views.
About
Cielo is a capability-class advanced simulation and computing system. A single calculation can run across all nodes and cores of the machine.
- 1.37 petaflops* capability system
- Installed 2010-2011 at Los Alamos National Laboratory
- Funded by the U.S. Department of Energy (DOE) National Nuclear Security Administration (NNSA) for the Advanced Simulation and Computing (ASC) Program
- Operated by Advanced Computing at Extreme Scale (ACES), a Los Alamos National Laboratory and Sandia National Laboratories partnership
- Built by Cray, Inc.
The National Nuclear Security Administration (NNSA) Advanced Simulation and Computing (ASC) Program Cielo computing system is the next-generation tri-Lab capability platform beyond the Purple system, which was retired in 2010. Cielo represents a 6x to 10x increase in the classified compute capability for NNSA over its predecessor. Cielo will be used by scientists at three national laboratories: Los Alamos (LANL), Sandia (SNL), and Lawrence Livermore (LLNL), to solve our nation's most demanding stockpile stewardship problems; that is, the large-scale application problems at the edge of our understanding of weapon physics. This fully functional, petascale system serves a diverse scientific and engineering workload. Cielo provides a robust code development and production environment scaled to meet or exceed the workload requirements.
*A “flops” is an acronym meaning floating-point operations per second. One petaflop/s (PF) is 1,000 trillion operations per second.
Facts and Figures
Cielo Facts and Figures | |
|---|---|
Operational Lifetime |
2011
|
Theoretical Peak Performance |
1,374 TF |
HPL (Linpack) Performance |
1,110 TF using 142,272 cores |
Cabinets |
96 |
# Compute Nodes |
8,944 |
# Compute Cores |
143,104 |
Compute Processor
|
Dual AMD Opteron™ 6136 eight-core "Magny-Cours" Socket G34 @ 2.4 GHz |
Compute Memory |
286 TB DDR3 @ 1333 MHz |
Compute Memory BW |
763 TB/s |
Service Nodes |
272 AMD Opteron™ 2427 six-core "Istanbul" Socket F @ 2.2 GHz |
External Login Nodes |
Qty 4 Dell PowerEdge R815 Servers |
User Disk Storage |
7.6 PB User Available Capacity |
Parallel File System |
Cray DVS and Lustre |
Parallel File System BW |
~160 GB/s |
High Speed Interconnect |
Cray Gemini 3D Torus in a 16 x 12 x 24 (XYZ) Topology |
Bi-section BW |
6.57 x 4.38 x 4.38 (XYZ) TB/s |
System Footprint |
~3,000 sq ft including storage |
Power Requirement |
3,980 KW running HPL |
Operating System |
Cray Linuz Environment |
