U.S. Department of Energy

Pacific Northwest National Laboratory

Imaging of Dynamic Processes in Electron Microscopy

Electron microscopes are used often in the biological and material sciences particularly for their ability to capture the dynamic evolution of microstructures with a very high spatial and temporal resolution. 

The last few years has seen a dramatic increase in the number of experimental studies in the electron microscope that use in-situ liquid stages to study dynamic processes. This has led to hardware and software advances that enable the acquisition of images at high rates over long periods of time in order to capture events that occur very quickly.  

Examples of driving scientific problems being explored in electron microscopy include nanoparticle nucleation and growth, electrochemical performance, and nanolipoprotein disc imaging.  Each of these applications relies on input from a user to adjust the parameters on the microscope during acquisition in order to successfully complete an experiment. 

AIM's role in this use case is to better inform users with quantitative information during an experiment.  This information will be used to help guide the decision process towards minimizing sample damage and maximizing the success rate for the experimental process.

Challenge

Microscopists are faced with two main challenges during data acquisition.

  • The electron beam, while illuminating the sample for imaging, also causes a reaction in the sample. Some reactions are good, while others, when performed under non-optimal dosages, can cause damages. 
  • The ability to capture images at the rate during which changes take place is challenging. Interesting events happen over very short time periods, requiring image data is captured at very high rates (up to 1,000 frames/second).

Approach

Microscopists have architected a compressed sensing approach that enables high rates of image acquisition while also exposing the samples to minimal electron dosage. The focus for AIM in this use case relies on the ability to analyze these compressed data streams for events that scientists are unable to detect (or even visualize) during an experiment. These events may include changes in sample drift, particle nucleation, precipitates, deposition, and noise. 

AIM’s research in this use case focuses on enabling users to interact with the compressed image streams through the following research efforts:

  • Change detection in subsampled images over time
  • Comparisons of different experiments, detecting similarities/differences to previous experiments
  • Triggering image reconstructions when events occur
  • Recognizing cognitive depletion in users
  • Deep learning for automatic event detection
  • Decision tree creation for the analysis of experimental parameters and outcomes
  • Faster reconstruction methods

Benefit

AIM will impact the field of electron microscopy in the following areas:

  • Decreased time toward scientific discovery. Researchers spend multiple days to simply set up a single experiment, adjust dosage levels, and repeat experiments. AIM's goal is to reduce the number of times researchers needs to repeat experiments and change parameters.
  • Enabling time-dependent decisions. The delay between the event and the opportunity to make a decision to change setting is too long to be effective.  AIM will develop methods to convey changes on the microscope to the user before they occur.
  • Decreased cost. The cost to run an electron microscope for a single publication can cost between $20-40K. The majority of this cost is often in finding the right parameters to run an experiment. AIM will enable faster methods for setting up and communicating to the user when experiments are being conducted correctly or incorrectly.
  • Enabling novices. Most users need 2-5 years of training to be considered expert microscopists.  Learning to use the microscope takes enormous amounts of time and requires numerous failed experiments. AIM will develop technologies that provide novice users with feedback to help them make fewer mistakes so they can become expert users faster.
  • Experimental maturity. The vision for AIM and electron microscopy would be to enable an advanced maturation of electron microscopy, so experiments could be conducted without a lot of input from users. AIM would be able to provide algorithms that have the capability to not only monitor experiments and provide feedback, but also adjust the experimental parameters automatically to enable a "self driving electron microscope."
| Pacific Northwest National Laboratory