A surprising use for distorted data to glean hidden laws

Many complex physical systems, such as materials, chemical systems and nuclear matter, are difficult or impossible to study directly. In a Nature Communications paper, Los Alamos National Laboratory scientists describe how “nuisance data” from large-scale stochastic simulations can be used to train machine learning algorithms to recognize the underlying laws and interactions of complex systems.

Read the paper    

Why this matters: Due to a phenomenon called metastability, a trapped simulation can skip over important phases of the system being studied, producing data that give incorrect equilibrium averages and a highly distorted picture of the full system. But according to this team’s finding, metastable information is useful for machine learning models before undergoing computational cleanup.

• In many of the most interesting physical regimes, simulations using the popular Markov-chain Monte Carlo methods are difficult to run even on high-performance computers, as they can take a long time to converge to the right statistical output. This is often a bottleneck for modeling approaches that use data produced by such Markov chains for downstream learning tasks. 
• The machine learning approach developed in this new study aids the process of learning complex models for such systems by using data produced by Markov chains long before they fully converge.

What they found: 

• Even when a simulation or a physical system is trapped in a metastable state and producing the wrong global statistics, the small fluctuations within the metastable state can still contain enough information to reconstruct the true model governing the system.
• The team, led by Abhijith Jayakumar, developed a mathematically rigorous theory of how local learning algorithms perform when given data from metastable samples and how the local method can recover the correct interactions even from globally biased data.

The big picture: Metastability is a fundamental feature of many physical systems in nature, not just a practical obstacle in large-scale simulations. The findings of this paper apply to general classes of such systems where time dynamics can be modeled as a random process.

Funding: U.S. Department of Energy’s Office of Science Advanced Scientific Computing Research Program and the Laboratory Directed Research and Development program of Los Alamos National Laboratory 

Editor’s note: Read about the birth of the Monte Carlo method at Los Alamos. Today, Monte Carlo methods underpin everything from nuclear security and Earth systems modeling to artificial intelligence and high-performance computing.

LA-UR-26-27766