One of the inescapable realities of various imaging techniques is called the "phase problem," which simply refers to the loss of phase information inherent in the nature of imaging methods such as x-ray diffraction. Though it might be inconvenient, it can be dealt with by using various mathematical methods to retrieve the phase data from the image with inverse computation. Such methods, however, are not only time-consuming but require a great deal of computer power as they must run through multiple iterations to converge on a solution, which prevents real-time imaging. A group of researchers working at the U.S. Department of Energy’s Advanced Photon Source (APS) has demonstrated a new approach to this perennial obstacle by using a deep-learning neural network trained and optimized for enhanced accuracy to perform fast three-dimensional (3-D) nanoscale imaging from coherent x-ray data. The work was published in Applied Physics Review.
Neural networks, which simulate the operation of the brain's intricate networks of nerve cells for applications such as artificial intelligence, have been criticized because it can be difficult to adequately ‘train’ them for the task at hand. If the training data or methodology are insufficient, the neural network will fail to perform as desired; as the computer programmer's cliché goes, garbage in, garbage out. They may also be influenced by subtle biases in the data or may be unable to extrapolate and generalize beyond the limits of their training.
In the present work, the investigators from Argonne National Laboratory, the University of Illinois at Chicago, Stats Perform, and Northwestern University confronted this problem by using explicitly “physics-aware” training that incorporates atomistic simulation data to create diverse training sets based on the physics of the material under study (Fig.1). The neural network predictions are then further refined in the final stage. The research team validated this 3-D convolutional encoder-decoder network (3D-CDI-NN) on 3-D coherent x-ray diffraction data of gold nanoparticles performed at the X-ray Science Division Microscopy Group’s 34-ID-C x-ray beamline at the APS.
The 3D-CDI-NN model begins with an offline training stage in which polyhedral shapes are created by randomly clipping an fcc crystal lattice structure, applying and relaxing various stresses in the Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) molecular dynamics program, and voxelizing the data into a 32 x 32 x 32 voxel grid. The grid is then entered into 3D-CDI-NN and its predictions matched with the known solutions. The 3D-CDI-NN framework is quite accurate in its predictions, even when working with under-sampled diffraction patterns.
To enhance the accuracy of 3D-CDI-NN even further, the investigators added a refinement that uses a reverse-mode automatic differentiation (AD) technique on the strain predictions and diffraction data. This proved especially effective in recovering details of structural strain within the examined crystal samples, even when not using oversampled data, as is generally done when using AD techniques.
The researchers demonstrated the effectiveness of the trained 3D-CDI-NN model with real-world data by imaging gold nanoparticles. Although the initial predictions showed some underestimation of surface details and local strain, which may be due to downsampling of the coherent x-ray data to the 32 x 32 x 32 size, these details were recovered after the AD refinement procedure. The 3D-CDI-NN framework was demonstrated to be about 4 times faster compared to typical iterative phase retrieval methods.
As fourth-generation synchrotron x-ray sources like the upgraded APS come online in the near future, with ever brighter and more versatile beams and capable of generating extremely large datasets, faster and more efficient phase retrieval techniques will be essential. The research team notes that machine-learning neural network approaches such as 3D-CDI-NN can be trained and improved to surpass traditional phase retrieval techniques by hundreds of times in speed while placing far lesser demands on scarce computing resources. Such automated methods can also be effectively scaled to different requirements and can even operate on datasets as they are still being collected.
Perhaps the phase problem can never be eliminated completely, but neural networks may make it easier to live with. ― Mark Wolverton
See: Henry Chan1,2*, Youssef S.G. Nashed3, Saugat Kandel4, Stephan O. Hruszkewycz1, Subramanian K.R.S. Sankaranarayanan1,2, Ross J. Harder1, and Mathew J. Cherukara1**, “Rapid 3D Nanoscale Coherent Imaging via Physics-aware Deep Learning,” Appl. Phys. Rev. 8, 021407 (2021). DOI: 10.1063/5.0031486
Author affiliations:1Argonne National Laboratory, 2University of Illinois at Chicago, 3Stats Perform, 4Northwestern University
Correspondence: *hchan@anl.gov, **mcherukara@anl.gov
The authors gratefully acknowledge the computing resources provided and operated by the Joint Laboratory for System Evaluation (JLSE) at Argonne National Laboratory. This work was performed at the Center for Nanoscale Materials (CNM) and the APS, both Office of Science user facilities supported by the U.S. Department of (DOE) Office of Science-Basic Energy Sciences, under Contract No. DE-AC02- 06CH11357. This work was also supported by Argonne LDRD 2018-019-N0: A.I C.D.I: Atomistically Informed Coherent Diffraction Imaging and by the U.S. DOE Office of Science-Basic Energy Sciences Data, Artificial Intelligence and Machine Learning at DOE Scientific User Facilities program under Award Number 34532. Development of the automatic differentiation refinement was supported by the U.S. DOE Office of Science-Basic Energy Sciences, Materials Science and Engineering Division. An award of computer time was provided by the Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program.
The U.S. Department of Energy's APS is one of the world’s most productive x-ray light source facilities. Each year, the APS provides high-brightness x-ray beams to a diverse community of more than 5,000 researchers in materials science, chemistry, condensed matter physics, the life and environmental sciences, and applied research. Researchers using the APS produce over 2,000 publications each year detailing impactful discoveries, and solve more vital biological protein structures than users of any other x-ray light source research facility. APS x-rays are ideally suited for explorations of materials and biological structures; elemental distribution; chemical, magnetic, electronic states; and a wide range of technologically important engineering systems from batteries to fuel injector sprays, all of which are the foundations of our nation’s economic, technological, and physical well-being.
Argonne National Laboratory seeks solutions to pressing national problems in science and technology. The nation's first national laboratory, Argonne conducts leading-edge basic and applied scientific research in virtually every scientific discipline. Argonne researchers work closely with researchers from hundreds of companies, universities, and federal, state and municipal agencies to help them solve their specific problems, advance America's scientific leadership and prepare the nation for a better future. With employees from more than 60 nations, Argonne is managed by UChicago Argonne, LLC, for the U.S. DOE Office of Science.
The U.S. Department of Energy's Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, visit the Office of Science website.