Ming Du and Chris Jacobsen have been awarded 250,000 node hours on the Argonne Leadership Computing Facility’s Theta computer by the U.S. Department of Energy’s (DOE’s) Advanced Scientific Computing Research (ASCR) program Leadership Computing Challenge (ALCC). The computer time will be used for “Distributed large wavefield propagation and 3D reconstruction beyond the depth of focus limit.” Du is an Advanced Photon Source (APS) postodoctoral appointee; Jacobsen is an Argonne National Laboratory Distinguished Fellow and Special Advisor to the APS Director for Imaging and Instrumentation, and a professor in the Department of Physics and Astronomy at Northwestern University. In all, 24 projects were awarded a total of 5.74 million node hours on the ALCF’s Theta supercomputer “to pursue challenging, high-risk, high-payoff simulations,” according to the ALCF web site.
The object of their research is to develop methods to image beyond the depth of focus limit of conventional x-ray microscopy. As the Advanced Photon Source sets out on its upgrade project to dramatically increase x-ray brightness, it will enable new capabilities for 3D nanoscale imaging ― but to do this on the thick samples that are accessible in x-ray imaging, one must use computationally-demanding techniques to incorporate beam propagation effects. This ALCC project uses “automatic differentiation” and "adjoint sensitivity analysis" to do so; the former was described in a recent article in Science Advances co-authored by Du, Jacobsen, Doğa Gürsoy, Youssef S. G. Nashed, and Saugat Kandel, while the latter has the potential to reconstruct large-volume samples more efficiently.
The mission of the ALCC is to provide an allocation program for projects of interest to the DOE with an emphasis on high-risk, high-payoff simulations in areas directly related to the DOE mission and for broadening the community of researchers capable of using leadership computing resources. ALCC awards support scientists from industry, academia and national laboratories advancing scientific and technological research in DOE mission areas including Biophysics, Energy Efficiency in Aerospace and Combustion technologies, Fusion Energy, Geosciences, High Energy Physics, Materials Sciences, Nuclear Engineering, and Nuclear Physics.
For the 2020-2021 ALCC campaign, 96 proposals were submitted, requesting an aggregate total of 350,560,000 node-hours, far above the 16,400,000 node-hours available. The 60 projects selected through a competitive review process received allocations totaling 16,400,000 node-hours.
The Argonne Leadership Computing Facility’s Theta is an 11.69-petaflops supercomputer based on Intel processors and interconnect technology, an advanced memory architecture, and a Lustre-based parallel file system, all integrated by Cray’s HPC software stack. The award to Ming Du and Chris Jacobsen was one of 24 projects (a total of 5.74 million node hours) awarded time at the Argonne Leadership Computing Facility to pursue challenging, high-risk, high-payoff simulations.
See: Ming Du1, Youssef S. G. Nashed2, Saugat Kandel1, Doğa Gürsoy1,2, and Chris Jacobsen1,2*, “Three dimensions, two microscopes, one code: Automatic differentiation for x-ray nanotomography beyond the depth of focus limit,” Sci. Adv. 6, eaay3700 (27 March 2020). DOI: 10.1126/sciadv.aay3700
Author affiliations: 1Northwestern University, 2Argonne National Laboratory
Correspondence: *cjacobsen@anl.gov
The authors acknowledge the National Institute of Mental Health, National Institutes of Health, for support under grant R01 MH115265. This research used resources of the Advanced Photon Source and the Argonne Leadership Computing Facility, which are U.S. Department of Energy (DOE) Office of Science User Facilities operated for the DOE Office of Science by Argonne National Laboratory under contract no. DE-AC02-06CH11357.
The U.S. Department of Energy's APS is one of the world’s most productive x-ray light source facilities. Each year, the APS provides high-brightness x-ray beams to a diverse community of more than 5,000 researchers in materials science, chemistry, condensed matter physics, the life and environmental sciences, and applied research. Researchers using the APS produce over 2,000 publications each year detailing impactful discoveries, and solve more vital biological protein structures than users of any other x-ray light source research facility. APS x-rays are ideally suited for explorations of materials and biological structures; elemental distribution; chemical, magnetic, electronic states; and a wide range of technologically important engineering systems from batteries to fuel injector sprays, all of which are the foundations of our nation’s economic, technological, and physical well-being.
Argonne National Laboratory seeks solutions to pressing national problems in science and technology. The nation's first national laboratory, Argonne conducts leading-edge basic and applied scientific research in virtually every scientific discipline. Argonne researchers work closely with researchers from hundreds of companies, universities, and federal, state and municipal agencies to help them solve their specific problems, advance America's scientific leadership and prepare the nation for a better future. With employees from more than 60 nations, Argonne is managed by UChicago Argonne, LLC, for the U.S. DOE Office of Science.
The U.S. Department of Energy's Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, visit the Office of Science website.