Los Alamos National LaboratoryCielo
Alliance for Computing at Extreme Scale (ACES)

Cielo supercomputer

Cielo: NNSA Capability Supercomputer

Cielo’s capabilities were designed and developed jointly by LANL and SNL under the Advanced Computing at Extreme Scale (ACES) partnership. The system is physically located at LANL in the Nicholas Metropolis Center for Modeling and Simulation.

Asteroid killer simulation video
2:50

Asteroid mitigation simulation

The facility is designed to scale with the increasingly powerful compute resources to deliver the required end-to-end services for users. Within the center are networking, archival storage, visualization servers, global file systems, and system software, all enhanced to support Cielo’s size and architecture. The ACES partnership is responsible for the support of Cielo, including interactions with the vendor.

This simulation of an asteroid mitigation by Bob Weaver (LANL) has had over 86,000 views.

About

Cielo is a capability-class advanced simulation and computing system. A single calculation can run across all nodes and cores of the machine.

The National Nuclear Security Administration (NNSA) Advanced Simulation and Computing (ASC) Program Cielo computing system is the next-generation tri-Lab capability platform beyond the Purple system, which was retired in 2010. Cielo represents a 6x to 10x increase in the classified compute capability for NNSA over its predecessor. Cielo will be used by scientists at three national laboratories: Los Alamos (LANL), Sandia (SNL), and Lawrence Livermore (LLNL), to solve our nation's most demanding stockpile stewardship problems; that is, the large-scale application problems at the edge of our understanding of weapon physics. This fully functional, petascale system serves a diverse scientific and engineering workload. Cielo provides a robust code development and production environment scaled to meet or exceed the workload requirements.

*A “flops” is an acronym meaning floating-point operations per second. One petaflop/s (PF) is 1,000 trillion operations per second.

Facts and Figures

Cielo Facts and Figures

Operational Lifetime

2011

Theoretical Peak Performance

1,374 TF

HPL (Linpack) Performance

1,110 TF using 142,272 cores

Cabinets

96

# Compute Nodes

8,944

# Compute Cores

143,104

Compute Processor

Dual AMD Opteron 6136 eight-core "Magny-Cours" Socket G34 @ 2.4 GHz

Compute Memory

286 TB DDR3 @ 1333 MHz

Compute Memory BW

763 TB/s

Service Nodes

272 AMD Opteron 2427 six-core "Istanbul" Socket F @ 2.2 GHz

External Login Nodes

Qty 4 Dell PowerEdge R815 Servers

User Disk Storage

7.6 PB User Available Capacity

Parallel File System

Cray DVS and Lustre

Parallel File System BW

~160 GB/s

High Speed Interconnect

Cray Gemini 3D Torus in a 16 x 12 x 24 (XYZ) Topology

Bi-section BW

6.57 x 4.38 x 4.38 (XYZ) TB/s

System Footprint

~3,000 sq ft including storage

Power Requirement

3,980 KW running HPL

Operating System

Cray Linuz Environment