DOE COE Performance Portability Meeting 2017
DOE Centers of Excellence Performance Portability Meeting 2017
August 22 - August 24, 2017
Denver, Colorado
The Department of Energy (DOE) Centers of Excellence (COEs) Performance Portability meeting is an opportunity for the five COEs to share ideas, progress, and challenges toward the goal of performance portability across DOE's large upcoming advanced architecture supercomputer procurements. The need for applications to run effectively on multiple vendor advanced architecture solutions (as well as on standard "cluster" technology) is pervasive across application teams within DOE and is a specified goal of the DOE's exascale plans for risk mitigation. The two primary goals of this meeting are to:
- Inform application teams and tool developers of activities and methodologies being used across the COEs, and foster informal relationships that can help DOE participants benefit from activities beyond their own COE.
- Identify major challenges toward the goal of performance portability, and work with the vendors and tool providers on determining implementations and solutions that will meet their own performance criteria without inadvertently impairing performance results elsewhere.
Post-Meeting Report
Recognizing the immense challenges of porting and optimizing large applications to the advanced architecture systems planned for deployment within the National Nuclear Security Administration (NNSA) and Office of Science (SC) labs between 2016 and 2019, the DOE has established a COE at each laboratory siting one of these systems. These COEs provide direct vendor expertise to the application teams and in turn, give the vendors deeper insight into how applications are run on those systems. Each of the five current COEs has a mission to optimize a set of applications for their specific platform—however the application teams are motivated to maintain a code base that will run effectively across diverse vendor offerings. Making use of open standards, libraries, and software abstractions that allow for minimal code disruption without negatively impacting performance potential is the preferred path to programming, but it constitutes a large, as-yet-unsolved challenge.
This meeting, a follow-on to the first in 2016 (DOE COE Performance Portability Meeting 2016) , will build upon past individual COE meetings or workshops in that it will provide a forum for best practices and ideas to be shared and will focus squarely on the issue of achieving high performance on these emerging platforms without greatly sacrificing portability and maintainability of applications.
This meeting is open to the following: application developers at the existing five COEs who are working on preparing their codes, COE collaborators at universities & other organizations, vendors chosen to provide the next-generation platforms, and solution-providers (DOE or third party) who are developing software tools aimed at helping application teams approach the challenges of performance portability.
Because it will be open to all vendors participating in the COEs (IBM, Cray, Intel, and NVIDIA), all accepted presentations must be free of NDA material with appropriate review & release from your organization for public posting. Participants are asked to join in the spirit of cooperation, and a base set of "ground rules" will be suggested to help ensure a productive and non-competitive meeting. Note that attendance by parties outside of the six DOE labs, the lab collaborators, and the four vendors involved in current COEs will be limited and by invitation only.
If you did not explicitly receive an invitation from your COE lead, please submit your request to the steering committee to attend at coepp-meeting@lanl.gov.
We plan to have a combination of talks, panel discussions, breakout sessions, working lunches, and time for informal interactions with colleagues from other COEs. A final report will be prepared to help guide future work within the COEs.
The agenda and abstract of talks, breakouts and posters are available below:
Session 1 - Performance Portability, Best Practices, Words of Wisdom
- Levesque, John: Setting expectations for Performance Portability between Companion Accelerator and Manycore Systems (pdf)
- O'Connor, Mark: When Performance Portability is less than Perfect: Matching Applications to Architectures (pdf)
- Rupp, Karl: Exascale Computing Without Templates (pdf)
- John Pennycook: Implications of a Metric for Performance Portability: Necessity of Specialization and Application-Specific Abstractions (pdf)
Session 2 - Memory Hierarchy (on-node)
- Olivier, Stephen: Memory Management Extensions for OpenMP 5.0 (pdf)
- Scogland, Tom: Deep Copy and Unified Memory in OpenMP (pdf)
- Newburn, CJ: A declarative approach to managing memory properties (pdf)
- Williams, Sean: Simplified Interface to Complex Memory (SICM) (pdf)
- Beckingsale, David: Umpire: Resource Management for Heterogeneous Memory Hierarchies (pdf)
Session 3 - Applications Experience 1
- Gunter, David: Kokkos port of CoMD mini-app for Trinity-class systems and NVIDIA Pascal nodes (pdf)
- Pankajakshan, Ramesh: SW4Lite: Performance Portability using RAJA (pdf)
- Vaquero, Alejandro: Performance Portability Experiments with the Grid C++ Lattice QCD Library (pdf)
- Ryujin, Brian: Experiences Porting a Multiphysics Code to GPUs (pdf)
Session 4 - Abstractions, DSL
- Bergen, Ben: The Flexible Computational Science Infrastructure (FleCSI) (pdf)
- Newburn, CJ: HiHAT, a way forward to perf portability with retargetable infrastructure (pdf)
- Kunen, Adam: Recent experiences with RAJA nested loop abstractions and CHAI (pdf)
- Edwards, Carter: Kokkos' Task-DAG Parallel Capabilities & Evolution of Kokkos’ Back-ends (pdf)
Session 5 - Applications Experience 2
- Howard, Micah: Performance Portability in SPARC – Sandia’s Hypersonic CFD Code for Next-Generation Platforms (pdf)
- Scott Parker: Portability and Performance of the Nekbone Mini-App (pdf)
- Friesen, Brian: Performance Portability Experiences at NERSC (pdf)
- Tharrington, Arnold: Portability Initiatives for Scientific Computing and Simulation: Molecular Dynamics as a Case Study (pdf)
Session 6 - Hierarchical Memory, off-node, I/O
- Gonsiorowski, Elsa: SCR and Preparing for Burst Buffers (pdf)
- Tessier, Francois: Toward portable I/O performance by leveraging system abstractions of deep memory and interconnect hierarchies (pdf)
Session 7 - Languages, Compilers, Frameworks, Tools
- Bertolli, Carlo: Performance Analysis and Optimizations for Lambda-based Applications in OpenMP 4.5 (pdf)
- Larkin, Jeff: Early Results of OpenMP 4.5 Portability on NVIDIA GPUs (pdf)
- Katz, Max: Adaptive Mesh Refinement for Exascale (pdf)
- Jin, Xiaoyong: Performance portability via Nim metaprogramming (pdf)
- Liakh, Dmitry: Portable Heterogeneous High Performance Computing via Domain‐Specific Virtualization (pdf)
- Edwards, Carter: High Performance Atomics are Critical for Thread Scalability (pdf)
- Bercea, Gheorghe-Teodor: Towards a portable OpenMP data sharing implementation for NVIDIA accelerators in the CLANG/LLVM toolchain (pdf)
Session 8 - Applications Experience 3
- Pearce, Olga: Experiences Utilizing CPUs and GPUs for Computation Simultaneously on a Heterogeneous Node (pdf)
- Lo, Li-Ta: Performance Portable Halo and Halo Center Finding in HACC (pdf)
- Quadros, Roshan: Scaling Post-meshing Operations on Next Generation Platforms (pdf)
- Govett, Mark: Parallelization and Performance of the NIM Weather Model for CPU, GPU and MIC Processors (pdf)
- Teranishi, Keita: Portability and Scalability of Sparse Tensor Decompositions on CPU/MIC/GPU Architectures (pdf)
- Blake, Robert: Melodee: Solving ODEs with platform-specific code generation (pdf)
Vendor Performance Portability Panel
- CJ Newburn (pdf), Principal HPC Architect for NVIDIA Compute Software
- John Levesque (pdf), Director of Supercomputing Center of Excellence at Cray
- John Pennycook (pdf), HPC Application Engineer at Intel
- Kathryn O’Brien, Principal Research Staff Member at IBM T.J. Watson Research Center
- Jonathan Gallmeier (pdf), AMD
- Geraint North (pdf), Distinguished Engineer of ARM HPC Tools
Breakouts
Breakout 1- Topic 1 - Multi-Level Memory Current Experiences; Abstractions for managing memory for systems with diverse memory resources (pdf)
Session 1: Lead: Ian Karlin (LLNL)
Session 2: Lead: CJ Newburn (NVIDIA) - Topic 2 - Performance, Portability and Productivity: Definitions & Metrics (pdf)
Session Lead: John Pennycook (Intel) - Topic 3 - Productivity Issues in Preparing Large Codes for Performance Portability (pdf)
Session Lead: Anshu Dubey (ANL)
Breakout 2
- Topic 1 - OpenMP 5.0 features (pdf)
Session Leads: Tom Scogland (LLNL) & Kevin O'Brien (IBM) - Topic 2 - SC Facilities Performance Portability Best Practices Website (pdf)
Session Lead: Jack Deslippe (LBNL) - Topic 3 - ISO/C++17 and Beyond - Parallelism and Concurrency (pdf)
Session Lead: Carter Edwards (SNL)
Posters
- Arlie Capps, Peter Robinson, Joseph Chavez: Progress Porting ALE3D to the GPU (pdf)
- Appelhans, David: Overlapping Data Movement and Compute with XLF and OpenMP 4: Experiences in UMT (pdf)
- Gorentla Venkata, Manjunath; Ferrol Aderholdt: SharP: Towards Programming Hierarchical-Heterogeneous Memory based Extreme-Scale Systems (pdf)
- Peles, Slaven: Performance portability of numerical time integrators in SUNDIALS library (pdf)
- North, Geraint: Experiences with performance portability across ARM microarchitectures (pdf)
- Sadayappan, Saday: High-performance GPU code generation for high-order stencils: Alleviating register pressure (pdf)
- Aravind Sukumaran-Rajam: TTLG: Tensor Transpose Library for GPUs (pdf)
- Gonsiorowski, Elsa: MACSio Development and Proxy Application Validation (pdf)
- Luo, Lixiang: OpenACC/OpenMP4.5 Interoperability (pdf)
- Liao, Chunhua "Leo": AutoPar: Semantics-Aware Automatic Insertion of OpenMP directives (pdf)
Registration is required to attend this event. Register at https://www.regonline.com/coeperformanceportabilitymeeting. Please contact Gloria Montoya-Rivera (gmon@lanl.gov) for any registration questions.
Registration cancellation or refunds will be charged a $25 cancellation processing fee. No refunds will be issued after August 1, 2017.
The meeting will be held at the Crowne Plaza Downtown Denver. A block of discounted rooms has been reserved and is available at https://aws.passkey.com/e/49241800. Hotel reservations can also be made by calling the hotel directly at (303) 573-1450. The group code is DOE COE. Participants are highly encouraged to reserve rooms at the Crowne Plaza, since the fee for using their conference facilities is contingent on filling a specified number of guest rooms.
The room block will close on August 1, 2017. After that date, the hotel will only honor the government rate if they have the availability.
The meeting organizers are seeking speakers to give short (nominally 20 minutes) talks on progress, ideas, and/or challenges on performance portability. Aside from I/O and hackathon outbriefs we are will prioritize talks that focus on results and experiences from multiple architectures in the following topical areas:
- Algorithmic and application development aimed at performance portability on diverse advanced architectures
- Software tools, libraries, abstractions, and standards intended to help applications address performance portability
- Early application experiences (both positive and negative) attempting to run portably across diverse platforms
- Portable approaches application and library teams are taking to "manage the memory hierarchy", and optimize data placement and movement
- Lessons learned from hack-a-thons and similar events
- Early experiences with burst buffers and other I/O approaches
- Productivity concerns related to the above topics
We are also soliciting ideas for breakout sessions and volunteers to lead these sessions related to the above topics.
Potential speakers or session topic leaders are asked to submit a title and short abstract of their proposed talk/session by July 14, 2017. The committee will review submissions and provide speakers and session leaders with a response by July 28, 2017.
All submissions should be sent to the steering committee at coepp-meeting@lanl.gov
The following "ground rules" have been established for participants and speakers:
- Talks and discussions must refrain from discussing information held under non-disclosure agreements. Contact your steering committee representative (below) if you need specific guidance.
- In the spirit of the meeting, talks and discussions should address general challenges to the goal of performance portability and approaches that might be applied to overcome those challenges.
- The focus of talks and discussions should be on portable, non-vendor-specific solutions as seen from the application developer perspective (that is, abstractions that hide vendor-specific solutions are acceptable). It is expected that a particular focus of the meeting will be to address possible evolutions of current standards (for example, OpenMP and C++) to better support performance portability.
- Projections to future machines should not be presented.
- Talks and discussions must be unclassified and non-sensitive in nature.
- Speakers and participants (both labs and vendors) should accept that DOE will have multiple target platforms as part of their national strategy and join the discussion in the spirit of cooperation. All COEs are working toward the goal of making these platforms the most useful and high performance they can be without the threat of "vendor lock-in."
- Hai Ah Nam, LANL hnam@lanl.gov
- Charles Ferenbaugh, LANL cferenba@lanl.gov
- Rob Neely, LLNL neely4@llnl.gov
- Bert Still, LLNL still1@llnl.gov
- Ian Karlin, LLNL karlin1@llnl.gov
- Mike Glass, SNL mwglass@sandia.gov
- Rob Hoekstra, SNL rjhoeks@sandia.gov
- Tjerk Straatsma, ORNL straatsmatp@ornl.gov
- Wayne Joubert, ORNL joubert@ornl.gov
- Rebecca Hartman-Baker, NERSC rjhartmanbaker@lbl.gov
- Jack Deslippe, NERSC jrdeslippe@lbl.gov
- Tim Williams, ANL tjwilliams@anl.gov
- Kalyan Kumaran, ANL kumaran@alcf.anl.gov
- Nick Romero, ANL naromero@anl.gov
- Victor Lee, Intel victor.w.lee@intel.com
- John Pennycook, Intel john.pennycook@intel.com
- John Levesque, Cray levesque@cray.com
- Kathryn O'Brien, IBM kmob@us.ibm.com
- Cyril Zeller, NVIDIA czeller@nvidia.com
- CJ Newburn, NVIDIA cnewburn@nvidia.com