4D scanning transmission electron microscopy

4D scanning transmission electron microscopy (4D STEM) is a subset of scanning transmission electron microscopy (STEM) which utilizes a pixelated electron detector to capture a convergent beam electron diffraction (CBED) pattern at each scan location. This technique captures a 2 dimensional reciprocal space image associated with each scan point as the beam rasters across a 2 dimensional region in real space, hence the name 4D STEM. Its development was enabled by evolution in STEM detectors and improvements computational power. The technique has applications in visual diffraction imaging, phase orientation and strain mapping, phase contrast analysis, among others.

The name 4D STEM is common in literature, however it is known by other names: 4D STEM EELS, ND STEM (N- since the number of dimensions could be higher than 4), position resolved diffraction (PRD), spatial resolved diffractometry, momentum-resolved STEM, "nanobeam precision electron diffraction", scanning electron nano diffraction (SEND), nanobeam electron diffraction (NBED), or pixelated STEM.

History
The use of diffraction patterns as a function of position dates back to the earliest days of STEM, for instance the early review of John M. Cowley and John C. H. Spence in 1978 or the analysis in 1983 by Laurence D. Marks and David J. Smith of the orientation of different crystalline segments in nanoparticles. Later work includes the analysis of diffraction patterns as a function of probe position in 1995, where Peter Nellist, B.C. McCallum and John Rodenburg attempted electron ptychography analysis of crystalline silicon. There is also fluctuation electron microscopy (FEM) technique, proposed in 1996 by Treacy and Gibson, which also included quantitative analysis of the differences in images or diffraction patterns taken at different locations on a given sample.

The field of 4D STEM remained underdeveloped due to the limited capabilities of detectors available at the time. The earliest work used either Grigson coils to scan the diffraction pattern, or an optical camera pickup from a phosphur screen. Later on CCD detectors became available, but while these are commonly used in transmission electron microscopy (TEM) they had limited data acquisition rates, could not distinguish where on the detector an electron strikes with high accuracy, and had low dynamic range which made them undesirable for use in 4D STEM.

In the late 2010s, the development of hybrid pixel array detectors (PAD) with single electron sensitivity, high dynamic range, and fast readout speeds allowed for practical 4D STEM experiments.

Operating Principle
While the process of data collection in 4D STEM is identical to that of standard STEM, each technique utilizes different detectors and collects different data. In 4D STEM there is a pixelated electron detector located at the back focal plane which collects the CBED pattern at each scan location. An image of the sample can be constructed from the CBED patterns by selecting an area in reciprocal space and assigning the average intensity of that area in each CBED pattern to the real space pixel the pattern corresponds to.

It is also possible for there to be a(n) ADF or HAADF image taken concurrently with the CBED pattern collection, depending on where the detector is located on the microscope. An annular dark-field image taken may be complementary to a bright-field image constructed from the captured CBED images.

The use of a hollow detector with a hole in the middle can allow for transmitted electrons to be passed to an EELS detector while scanning. This allows for the simultaneous collection of chemical spectra information and structure information.

Detectors
In traditional TEM, imaging detectors use phosphorescent scintillators paired with a charge coupled device (CCD) to detect electrons. While these devices have good electron sensitivity, they lack the necessary readout speed and dynamic range necessary for 4D STEM. Additionally, the use of a scintillator can worsen the point spread function (PSF) of the detector due to the electron's interaction with the scintillator resulting in a broadening of the signal. In contrast, traditional annular STEM detectors have the necessary readout speed, but instead of collecting a full CBED pattern the detector integrates the collected intensity over a range of angles into a single data point. The development of pixelated detectors in the 2010s with single electron sensitivity, fast readout speeds, and high dynamic range has enabled 4D STEM as a viable experimental method.

4D STEM detectors are typically built as either a monolithic active pixel sensor (MAPS) or as a hybrid pixel array detector (PAD).

Monolithic active pixel sensor (MAPS)
A MAPS detector consists of a complementary metal–oxide–semiconductor (CMOS) chip paired with a doped epitaxial surface layer which converts high energy electrons into many lower energy electrons that travel down to the detector. MAPS detectors must be radiation hardened as their direct exposure to high energy electrons makes radiation damage a key concern.

Due to its monolithic nature and straightforward design, MAPS detectors can attain high pixel densities on the order of 4000 x 4000. This high pixel density when paired with low electron doses can enable single electron counting for high efficiency imaging. Additionally, MAPS detectors tend to have electron high sensitivities and fast readout speeds, but suffer from limited dynamic range.

Pixel array detector (PAD)
PAD detectors consist of a photodiode bump bonded to an integrated circuit, where each solder bump represents a single pixel on the detector.

These detectors typically have lower pixel densities on the order of 128 x 128 but can achieve much higher dynamic range on the order of 32 bits. These detectors can achieve relatively high readout speeds on the order of 1 ms/pixel but are still lacking compared to their annular detector counterparts in STEM which can achieve readout speeds on the order of 10 μs/pixel.

Detector noise performance is often measured by its detective quantum efficiency (DQE) defined as: $DQE=\frac{SNR_o^2}{SNR_i^2}$

where $$SNR_o^2$$ is output signal to noise ratio squared and $$SNR_i^2$$ is the input signal to noise ratio squared. Ideally the DQE of a sensor is 1 indicating the sensor generates zero noise. The DQE of MAPS, APS and other direct electron detectors tend to be higher than their CCD camera counterparts.

Computational Methods
A major issue in 4D STEM is the large quantity of data collected by the technique. With upwards of 100s of TB of data produced over the course of an hour of scanning, finding pertinent information is challenging and requires advanced computation.

Analysis of such large datasets can be quite complex and computational methods to process this data are being developed. Many code repositories for analysis of 4D STEM are currently in development including: HyperSpy,, LiberTEM, Pycroscopy, and.

AI driven analysis is possible. However, some methods require databases of information to train on which currently do not exist. Additionally, lack of metrics for data quality, limited scalability due to poor cross-platform support across different manufacturers, and lack of standardization in analysis and experimental methods brings up questions of comparability across different datasets as well as reproducibility.

Selected Applications
4D STEM has been utilized in a wide array of applications, the most common uses include virtual diffraction imaging, orientation and strain mapping, and phase contrast analysis which are covered below. The technique has also been applied in: medium range order measurement, Higher order Laue zone (HOLZ) channeling contrast imaging, Position averaged CBED, fluctuation electron microscopy, biomaterials characterization, and medical fields (microstructure of pharmaceutical materials and orientation mapping of peptide crystals). This list is in no way exhaustive and as the field is still relatively young more applications are actively being developed.

Virtual Diffraction (Dark Field / Bright Field) Imaging
Virtual diffraction imaging is a method developed to generate real space images from diffraction patterns. This technique has been used in characterizing material structures since the 90s but more recently has been applied in 4D STEM applications. This technique often works best with scanning electron nano diffraction (SEND), where the probe convergence angle is relatively low to give separated diffraction disks (thus also giving a resolution measured in nm, not Å). A "virtual detector," is not a detector at all but rather a method of data processing which integrates a subset of pixels in diffraction patterns at each raster position to create a bright-field or dark-field image. A region of interest is selected on some representative diffraction pattern, and only those pixels within the aperture summed to form the image. This virtual aperture can be any size/shape desired and can be created using the 4D dataset gathered from a single scan. This ability to apply different apertures to the same dataset is possible because of having the whole diffraction pattern in the 4D STEM dataset. This eliminates a typical weaknesses in conventional STEM operation as STEM bright-field and dark-field detectors are placed at fixed angles and cannot be changed during imaging.

With a 4D dataset bright/dark-field images can be obtained by integrating diffraction intensities from diffracted and transmitted beams respectively. Creating images from these patterns can give nanometer or atomic resolution information (depending on the pixel step size and the range of diffracted angles used to form the image) and is typically used to characterize the structure of nanomaterials. Additionally, these diffraction patterns can be indexed and analyzed using other 4DSTEM techniques, such as orientation and phase mapping, or strain mapping. A key advantage of performing virtual diffraction imaging in 4D STEM is the flexibility. Any shape of aperture could be used: a circle (cognate with traditional TEM bright/dark field imaging), a rectangle, an annulus (cognate with STEM ADF/ABF imaging), or any combination of apertures in a more complex pattern. The use of regular grids of apertures is particularly powerful at imaging a crystal with high signal to noise and minimising the effects of bending and has been used by McCartan et al. ; this also allowed the imaging of an array of superlattice spots associated with a particular crystal ordering in part of the crystal as a result of chemical segregation.

Virtual diffraction imaging has been used to map interfaces, select intensity from selected areas of the diffraction plane to form enhanced dark field images, map positions of nanoscale precipitates, create phase maps of beam sensitive battery cathode materials, and measure degree of crystallinity in metal-organic frameworks (MOFs).

Recent work has further extended the possibilities of virtual diffraction imaging, by applying a more digital approach adapted from one developed for orientation and phase mapping, or strain mapping. In these methods, the diffraction spot positions in a 4D dataset are determined for each diffraction pattern and turned into a list, and operations are performed on the list, not on the whole images. For dark field imaging, the centroid positions for the list of diffraction spots can be simply compared against a list of centroid positions for where spots are expected and intensity only added where diffraction spot centroids agree with the selected positions. This gives far more selectivity than simply integrating all intensity in an aperture (particularly because it ignores diffuse intensity that does not fall in spots), and consequently, much higher contrast in the resulting images and has recently been submitted to arXiv.

Phase Orientation Mapping
Phase orientation mapping is typically done with electron back scattered diffraction in SEM which can give 2D maps of grain orientation in polycrystalline materials. The technique can also be done in TEM using Kikuchi lines, which is more applicable for thicker samples since formation of Kikuchi lines relies on diffuse scattering being present. Alternatively, in TEM one can utilize precession electron diffraction (PED) to record a large number of diffraction patterns and through comparison to known patterns, the relative orientation of grains in can be determined. 4D STEM can also be used to map orientations, in a technique called Bragg spot imaging. The use of traditional TEM techniques typically results in better resolution than the 4D STEM approach but can fail in regions with high strain as the DPs become too distorted.

In Bragg spot imaging, first correlation analysis method is performed to group diffraction patterns (DPs) using a correlation method between 0 (no correlation) and 1 (exact match); then the DP's are grouped by their correlation using a correlation threshold. A correlation image can then be obtained from each group. These are summed and averaged to obtain an overall representative diffraction template from each grouping. Different orientations can be assigned colors which helps visualize individual grain orientations. With proper tilting and utilizing precession electron diffraction (PED) it is even possible to make 3D tomographic renderings of grain orientation and distribution. Since the technique is computationally intensive, recent efforts have been focused on a machine learning approach to analysis of diffraction patterns.

Strain Mapping
TEM can measure local strains and is often used to map strain in samples using condensed beam electron diffraction CBED. The basis of this technique is to compare an unstrained region of the sample's diffraction pattern with a strained region to see the changes in the lattice parameter. With STEM, the disc positions diffracted from an area of a specimen can provide spatial strain information. The use of this technique with 4D STEM datasets includes fairly involved calculations.

Utilizing SEND, bright and dark field images can be obtained from diffraction patterns by integration of direct and diffracted beams respectively, as discussed previously. During 4D STEM operation the ADF detector can be used to visualize a particular region of interest through a collection of scattered electrons to large angles to correlate probe location with diffraction during measurements. There is a tradeoff between resolution and strain information; since larger probes can average strain measurements over a large volume, but moving to smaller probe sizes gives higher real space resolution. There are ways to combat this issue such as spacing probes further apart than the resolution limit to increase the field of view.

This strain mapping technique has been applied in many crystalline materials and has been extended to semi-crystalline and amorphous materials (such as metallic glasses) since they too exhibit deviations from mean atomic spacing in regions of high strain

Differential phase contrast
The differential phase contrast imaging technique (DPC) can be used in STEM to map the local electromagnetic field in samples by measuring the deflection of the electron beam caused by the field at each scan point. Traditionally the movement of the beam was tracked using segmented annular field detectors which surrounded the beam. DPC with segmented detectors has up to atomic resolution. The use of a pixelated detector in 4D STEM and a computer to track the movement of the "center of mass" of the CBED patterns was found to provide comparable results to those found using segmented detectors. 4D STEM allows for phase change measurements along all directions to be measured without the need to rotate the segmented detector to align with specimen orientation. The ability to measure local polarization in parallel with the local electric field has also been demonstrated with 4D STEM.

DPC imaging with 4D STEM is up to 2 orders of magnitude slower than DPC with segmented detectors and requires advanced analysis of large four-dimensional datasets.

Ptychography
The overlapping CBED measurements present in a 4D STEM dataset allow for the construction of the complex electron probe and complex sample potential using the ptychography technique. Ptychographic reconstructions with 4D STEM data were shown to provide higher contrast than ADF, BF, ABF, and segmented DPC imaging in STEM. The high signal-to-noise ratio of this technique under 4D STEM makes it attractive for imaging radiation sensitive specimens such as biological specimens The use of a pixelated detector with a hole in the middle to allow the unscattered electron beam to pass to a spectrometer has been shown to allow ptychographic analysis in conjunction with chemical analysis in 4D STEM.

MIDI STEM
This technique MIDI-STEM (matched illumination and detector interferometry-STEM), while being less common, is used with ptychography to create higher contrast phase images. The placement of a phase plate with zones of 0 and π/2 phase shift in the probe forming aperture creates a series of concentric rings in the resulting CBED pattern. The difference in counts between the 0 and π/2 regions allows for direct measurement of local sample phase. The counts in the different regions could be measured via complex standard detector geometries or the use of a pixelated detector in 4D STEM. Pixelated detectors have been shown to utilize this technique with atomic resolution.

(MIDI)-STEM produces image contrast information with less high-pass filtering than DPC or ptychography but is less efficient at high spatial frequencies than those techniques. (MIDI)-STEM used in conjunction with ptychography has been shown to be more efficient in providing contrast information than either technique individually.