Images (4D-STEM)#
- class pyemaps.stackimg.StackImage(imgfn, nformat=E_SH, ndtype=E_FLOAT, dim=(MIN_IMAGESIZE, MIN_IMAGESIZE, MIN_IMAGESTACK), noffset=8)#
Bases:
objectAn experimental diffraction image, or a stack of them, for 4D-STEM analysis with the licensed stem4d module:
indexing a diffraction pattern against a database (
indexImage());bright field, dark field and annular images (
generateBDF());images from a detector mask (
generateMaskedImage()).
An image is read from a file or taken from a numpy array, depending on nformat:
E_SH: pyEMAPS’ own small-header format (.im3). The 8-byte header holds four short integers: the width, the height, the number of stacks and the data type.E_RAW: a headered raw file. The pixel data starts noffset bytes into the file, stored row by row, and you give the data type and dimensions yourself.E_NPY: a 2-D numpy array, set through thedataproperty. The file name is ignored.
The image is read into the backend by
loadImage(), or by the analysis methods, which load it themselves.To analyse a TIFF, DigitalMicrograph or EMD file, such as a 4D-STEM scan, create the image with
from_file().Images are 100 to 512 pixels per side (
MIN_IMAGESIZE,MAX_IMAGESIZE), with 1 to 5000 stacks (MIN_IMAGESTACK,MAX_IMAGESTACK).- __init__(imgfn, nformat=E_SH, ndtype=E_FLOAT, dim=(MIN_IMAGESIZE, MIN_IMAGESIZE, MIN_IMAGESTACK), noffset=8)#
- Parameters:
imgfn (str) – path of the image file. Ignored for
E_NPY.nformat (int, optional) – the image format:
E_SH(the default),E_RAWorE_NPY.ndtype (int, optional) – the pixel data type, used for
E_RAWfiles. Defaults toE_FLOAT. Seendtype.dim (tuple of int, optional) – the image dimensions
(width, height, stacks), used forE_RAWfiles. Defaults to(100, 100, 1).noffset (int, optional) – where the pixel data starts in an
E_RAWfile, in bytes. Defaults to 8.
- Raises:
XDPImageError – if a parameter is invalid.
- property source#
The file the image was read from by
from_file(), orNone.
- property scan_shape#
The scan shape of an image read by
from_file():()for one pattern,(n,)for a series,(rows, columns)for a 2-D scan.Nonefor other images.
- property fname#
Path of the image file. Ignored when the format is
E_NPY.
- property nformat#
The image format:
- property ndtype#
The pixel data type of a raw image:
E_INT(single image) orEM_INT(stack): 2-byte integers.E_FLOATorEM_FLOAT: 4-byte floating-point numbers.E_DOUBLEorEM_DOUBLE: 8-byte floating-point numbers.
- property noffset#
Where the pixel data starts in a raw file, in bytes: usually the size of the file’s header.
- property dim#
The image dimensions
(width, height, stacks). Width and height are 100 to 512 pixels, and there can be 1 to 5000 stacks. Loading an image replaces them with the loaded image’s dimensions.
- property data#
The image data, as a numpy array.
After
loadImage(), it holds the loaded image; aftergenerateBDF()orgenerateMaskedImage(), the generated image.With the format set to
E_NPY, set it to a 2-D numpy array of any boolean, integer or floating-point type, 100 to 512 pixels per side, before callingloadImage(). It’s converted to float32 when loaded.
- classmethod from_file(path, binning=1, scan_roi=None, scan_shape=None, reader=None, dataset=None)#
Create an image from an image file in any format that
read_image()reads: TIFF (.tif,.tiff), a single pattern or a multi-page stack such as a 4D-STEM scan; Gatan DigitalMicrograph (.dm3,.dm4); MRC; Quantum Detectors Merlin (.mib); PNG, BMP, JPEG and NumPy; EMD (.emd) from Thermo Fisher Velox or in the Berkeley EMD format, such as py4DSTEM’s 4D-STEM datacubes (these needpip install "pyemaps[emd]"); and pyEMAPS’ own.im3.A single pattern is analysed as a numpy image (
E_NPY). A stack, such as a 4D-STEM scan, is copied to a temporary small-header file, removed when the image is deleted;scan_shapekeeps the scan’s shape, sogenerateBDF()andgenerateMaskedImage()know the scan width.The 4D-STEM features take patterns of 100 to 512 pixels per side and up to 5000 patterns. Larger patterns need binning, and larger scans a scan_roi; pyEMAPS doesn’t reduce them for you.
- Parameters:
path (str or os.PathLike) – the file.
binning (int, optional) – average binning x binning pixel blocks into one. Scale the camera constant and pattern center for
indexImage()by1 / binningtoo.scan_roi (tuple of int, optional) – the part of the scan to read:
(row, column, rows, columns), or(first, count)for a series.scan_shape (tuple of int, optional) – the scan shape,
(rows, columns), for a stack whose file doesn’t record it, such as an ImageJ stack or a Merlin file. Seeread_image().reader (str, optional) – the reader to use whatever the file’s extension, such as
'TIFF'or'MRC'; seeimage_readers().dataset (int or str, optional) – for a DigitalMicrograph or EMD file with several images, the one to read; see
read_image().
- Returns:
the image, ready for the analysis methods.
- Return type:
- Raises:
FileNotFoundError – if the file doesn’t exist.
ImportError – for an EMD file, if h5py isn’t installed.
XDPImageError – if the file can’t be read, or its patterns are the wrong size or too many; the message suggests the binning or scan_roi to use.
- loadImage(rmode=EL_ONE, stack=1)#
Read the image into the stem4d backend for analysis. The image’s
dimanddataare then updated from the loaded image.- Parameters:
rmode (int, optional) –
EL_ONE(the default) to read one stack, orEL_MOREto read them all.stack (int, optional) – which stack to read with
EL_ONE, counting from 1.
- Raises:
XDPImageError – if the image can’t be loaded, for example because the file is missing or invalid, a numpy image is missing or invalid, or no valid license is found. A license problem is also explained in a printed message.
- viewExpImage(peakDP=False, iStack=1, title='', iShow=False)#
Display the loaded image. After indexing, it can also draw the indexed diffraction peaks on top of it.
- Parameters:
peakDP (bool, optional) – draw the diffraction peaks found by the last peak search or indexing.
iStack (int, optional) – the stack number. It’s only checked against the number of stacks: the image shown is the one in
data.title (str, optional) – the figure title.
iShow (bool, optional) – label the peaks with their Miller indices.
- Raises:
XDPImageError – if the image data can’t be converted for display.
If no image is loaded, or there are no peaks to draw, it prints a message and shows nothing.
- static showMatchedDBDP()#
Display the database diffraction pattern that best matches the experimental pattern in the last
indexImage()call, with its diffraction peaks marked. Use it to check the match. See the sampleal_ediom.py.- Returns:
-1 if the pattern can’t be retrieved or displayed.
- static displayFitImage(nr, nc)#
Display the fit image from peak search. Used internally for debugging.
- Parameters:
nr (int) – number of rows in the image.
nc (int) – number of columns in the image.
- Returns:
0 on success, otherwise the error code.
- Return type:
int
- static displaySearchMask(nr, nc)#
Display the mask used in the peak search. Shown by
indexImage()when bDebug is set.- Parameters:
nr (int) – number of rows in the mask.
nc (int) – number of columns in the mask.
- Returns:
- Return type:
int
- static displaySearchKernel(nr, nc)#
Display the kernel used in the peak search. Shown by
indexImage()when bDebug is set.- Parameters:
nr (int) – number of rows in the kernel.
nc (int) – number of columns in the kernel.
- Returns:
0 on success, -1 if the kernel can’t be retrieved.
- Return type:
int
- static showDPDBMMask(mr, mc)#
Display the stereographic projection map of the loaded diffraction pattern database. Each point on the map is an orientation with a pattern in the database:
- Parameters:
mr (int) – the map’s height, from
loadDPDB().mc (int) – the map’s width, from
loadDPDB().
- Returns:
-1 if the map can’t be retrieved.
- static loadDPDB(dbfn, bShowDBMap=False)#
Load a diffraction pattern database file into stem4d, for peak search and indexing.
Create the database with
generateDPDB(); see the sampleal_dpgen.py. A database file can be kept and reused.- Parameters:
dbfn (str) – path of the database file.
bShowDBMap (bool, optional) – display the database’s stereographic projection map after loading it.
- Returns:
(code, height, width): 0 on success, and the height and width of the stereographic projection map. On failure, the code isn’t 0 and both sizes are 0.- Return type:
tuple
The stereographic projection map represents the database: each point is an orientation with a pattern in it.
- static showMatchingIndexMap(fr, fc)#
Display the matching index map from the last
indexImage()call: a heat map over the database’s stereographic projection map. Its peak marks the orientation where the experimental and database patterns match best.- Parameters:
fr (int) – the map’s height, from
loadDPDB().fc (int) – the map’s width, from
loadDPDB().
- Returns:
the error code if the map can’t be retrieved, otherwise
None.
- indexImage(dpdbfn, cc=DEF_CC, sigma=DEF_SIGMA, img_center=DEF_ICENTER, rmin=DEF_RMIN, search_box=DEF_BOXSIZE, scaling_option=(DEF_XSCALE, DEF_TSCALE), filter_threshold=DEF_FILTER_THRESHOLD, peak_threshold=DEF_SEARCH_THRESHOLD, ssel=1, bDebug=False)#
Find the diffraction peaks in the experimental image and index them against a diffraction pattern database. This is a licensed feature.
The method loads the database and the image itself, one stack at a time. The database is made by
generateDPDB(); see the samplesal_dpgen.pyandal_ediom.py.- Parameters:
dpdbfn (str) – path of the diffraction pattern database file.
cc (float, optional) – camera constant, in pixels per 1/Å.
sigma (float, optional) – estimated width of the diffraction peaks, in pixels.
img_center (tuple of float, optional) – center of the diffraction pattern
(column, row), in pixels.rmin (float, optional) – radius around the center, in pixels, excluded from the search. It removes the central beam and the diffuse intensity around it.
search_box (float, optional) – size of the peak search box, in pixels. For selected area diffraction, 3 to 4 times sigma is recommended.
scaling_option (tuple of int, optional) – how the experimental and the database pattern intensities are scaled for matching: two integers, each 0 to 5, not both 0.
filter_threshold (float, optional) – screening threshold for the database patterns, from 0.0 to 1.0. Use 0 to search all of them.
peak_threshold (float, optional) – peak threshold, from 0.0 to 1.0, used to drop noisy peaks.
ssel (int, optional) – the stack to index, counting from 1.
bDebug (bool, optional) – also display the intermediate images: the search mask and kernel, and the peaks found before indexing.
- Returns:
0 on success. -1, or the backend’s error code, if the peak search or the indexing fails; fewer than four peaks found counts as a failure.
- Return type:
int
- Raises:
XDPImageError – if scaling_option is out of range, or the database or the image can’t be loaded.
ValueError – if img_center is outside the image.
The image measurements, cc to rmin, need to be accurate: errors in them can spoil the peak search and the indexing. The remaining parameters tune the search and the indexing.
A typical run displays:
The stereographic projection map of the database, once it’s loaded:
The experimental image, once it’s loaded, so you can check it:
The indexing result, drawn on the experimental image:
The matching index map (
showMatchingIndexMap()). The best match between the database patterns and the experimental pattern is in red:
Afterwards,
showMatchedDBDP()displays the best matching database pattern:
The results, printed to standard output:
Number of peaks found: 45 -----------------Diffraction Pattern(DP) Indexing Results-------------------- ***** DP stack loaded and indexed: 1 ***** Best matching database DP: 1715 ***** DP rotation angle: 284.191 (degrees) ***** Number of diffraction peaks indexed: 11 ***** Experimental and database DP correlation maximum: 0.988308 ***** Beam direction: 0.0575418 0.0599394 0.232559 ***** Reliability index (> 5.0): 30.5407 -----------------Indexed Diffraction Pattern Details------------------ DP # Miller Index Distance DP Loc Intensity 1 2 -2 0 18.805359 114.912613 109.021492 14.05505 2 -2 2 0 20.888754 81.346603 87.833191 100.00000 3 3 1 -1 22.201900 95.978333 120.995316 28.63353 4 -1 -3 1 22.830843 117.979164 86.309898 9.12775 5 -3 -1 1 23.202806 101.336617 75.915146 17.33628 6 1 3 -1 23.803307 78.843224 111.661034 97.01090 7 5 -1 -1 36.265705 112.098000 132.817802 2.99402 8 -5 1 1 36.616173 85.120682 65.116249 4.92946 9 -1 5 -1 36.805595 62.239502 100.821404 33.52511 10 -4 4 0 39.269745 66.114388 77.537254 18.77301 11 4 4 -2 42.499973 76.063011 134.779076 2.83128
A reliability index above 5.0 means the indexing succeeded. If it doesn’t, try adjusting filter_threshold and peak_threshold.
- generateBDF(center=(0.0, 0.0), rads=(0.0, 0.0), scol=0.0, bShow=False)#
Generate a bright field (BF), dark field (DF), annular bright field (ABF) or annular dark field (ADF) image from a 4D-STEM image stack, with a virtual detector. Each stack is the diffraction pattern at one scan position. This is a licensed feature; see the sample
adf.py.- Parameters:
center (tuple of float, optional) – center of the detector
(column, row), in pixels of the diffraction patterns.rads (tuple of float, optional) – inner and outer radius of the detector, in pixels; they must differ. An inner radius of 0 gives a bright field image if the detector covers the central beam, or a dark field image if it doesn’t. A positive inner radius gives an annular bright or dark field image.
scol (int or float) – number of scan positions per row: the width of the output image. The number of stacks must be a multiple of it. For a 2-D scan read by
from_file(), it defaults to the scan’s width.bShow (bool, optional) – display the image.
- Returns:
(code, data). The code is 0 on success, and data is the image as a numpy array, also stored indata. On failure, the code is negative, data isNoneand a message is printed.- Return type:
tuple
Note
The image must have more than one stack. All stacks are loaded.
- generateMaskedImage(maskfn, scancol=0.0, bShow=False)#
Generate an image from a 4D-STEM image stack with a detector mask: each pixel of the output is the sum of the masked pixels in one stack. This is a licensed feature; see the sample
adf.py.- Parameters:
maskfn (str) – path of the mask, an image file in pyEMAPS’ small-header format, with the same width and height as the stacks, and integer or float data. Pixels above 0.5 are included. To convert a raw mask file, use
convertImage().scancol (int or float) – number of scan positions per row: the width of the output image. The number of stacks must be a multiple of it. For a 2-D scan read by
from_file(), it defaults to the scan’s width.bShow (bool, optional) – display the image.
- Returns:
(code, data). The code is 0 on success, and data is the image as a numpy array, also stored indata. On failure, the code is negative and data isNone.- Return type:
tuple
- static convertImage(imgfn, nformat=E_RAW, ndtype=E_FLOAT, dim=(MIN_IMAGESIZE, MIN_IMAGESIZE, MIN_IMAGESTACK), noffset=8, binning=1, scan_roi=None, scan_shape=None, reader=None, dataset=None)#
Convert a raw image file, a numpy image, or any image file
read_image()reads, to pyEMAPS’ small-header format (.im3).The stem4d module needs auxiliary images, such as the masks for
generateMaskedImage(), in this format. See the sampleconvert_image.py.- Parameters:
imgfn (str or numpy.ndarray) – path of the image file, or with
nformat=E_NPY, the image as a 2-D numpy array.nformat (int, optional) – the input format:
E_RAW(the default),E_NPY, orE_FILEfor any formatread_image()reads, such as TIFF or EMD.E_FILEconverts every pattern of a stack, and needs no license.ndtype (int, optional) – the pixel data type of a raw file. Defaults to
E_FLOAT. Seendtype.dim (tuple of int, optional) – the dimensions
(width, height, stacks)of a raw file. Defaults to(100, 100, 1).noffset (int, optional) – where the pixel data starts in a raw file, in bytes. Defaults to 8.
binning (int, optional) – with
E_FILE, average binning x binning pixel blocks into one.scan_roi (tuple of int, optional) – with
E_FILE, the part of the scan to convert:(row, column, rows, columns), or(first, count)for a series.scan_shape (tuple of int, optional) – with
E_FILE, the scan shape of a stack whose file doesn’t record it; seeread_image().reader (str, optional) – with
E_FILE, the reader to use; seeread_image().dataset (int or str, optional) – with
E_FILE, for an EMD file with several images, the one to convert.
- Returns:
the path of the converted file.
- Return type:
str
- Raises:
ValueError – if nformat isn’t
E_RAW,E_NPYorE_FILE, the file name is invalid, or the numpy image isn’t a numpy array.FileNotFoundError – if the file isn’t found.
XDPImageError – if the image can’t be loaded or saved, or with
E_FILE, if the file can’t be read or its patterns are the wrong size for the 4D-STEM features.
A file name without a folder is looked for in the folder set by the
PYEMAPS_DATAenvironment variable, or else in the current working directory.The output is saved in a
sh_convertedfolder next to the input file, with the same name and the extension.im3. A numpy image is saved in ash_convertedfolder in thePYEMAPS_DATAfolder, or else in the current working directory, namednumpy_image_followed by the date and time. That folder must be writable.
Reading image files#
Read experimental diffraction patterns from image files.
read_image() reads:
TIFF files (
.tif,.tiff): single images and multi-page stacks such as 4D-STEM scans;Gatan DigitalMicrograph files (
.dm3,.dm4): single images, stacks and 4D-STEM datacubes, with their calibration and the microscope settings DigitalMicrograph recorded;MRC files (
.mrc,.mrcs,.st): images and stacks, such as 3D electron diffraction tilt series;Quantum Detectors Merlin files (
.mib): 4D-STEM frames from Medipix3 detectors;PNG, BMP and JPEG pictures (8 or 16-bit grayscale), and NumPy arrays (
.npy);EMD files (
.emd): Thermo Fisher Velox images and image series, with the microscope settings Velox records, and Berkeley EMD files, including py4DSTEM’s 4D-STEM datacubes, with their calibration. EMD files need the h5py package:pip install "pyemaps[emd]";pyEMAPS’ own small-header files (
.im3,.img).
It returns the patterns as numpy arrays, with the scan shape. pyEMAPS reads all of these
itself, except EMD, with no other package. Reading files needs no license; analysing
them with the 4D-STEM features does (see
StackImage.from_file).
TIFF files can be 8 to 64-bit integers or 16 to 64-bit floats, uncompressed or compressed with LZW, Deflate (zip) or PackBits, including BigTIFF and ImageJ stacks. Large stacks read fastest uncompressed or with Deflate; LZW is decoded in Python, at a few MB per second.
- class pyemaps.expimage.ExperimentalImage(data, scan_shape, reader, axes, metadata, source)#
Experimental diffraction patterns read from a file by
read_image(): one pattern, or a stack of them, one per scan position.- __init__(data, scan_shape, reader, axes, metadata, source)#
- property data#
The patterns, as a numpy array of shape
(patterns, rows, columns), in scan order: along a scan row first. 16-bit integer data, and integer data that fits in 16 bits, isint16; everything else isfloat32.
- property scan_shape#
The scan shape:
()for a single pattern,(n,)for a series,(rows, columns)for a 2-D scan.
- property npatterns#
The number of patterns.
- property pattern_shape#
The size of each pattern,
(rows, columns), in pixels.
- property reader#
The reader that read the file:
TIFF,DM,EMD,MRC,MIB,PICTURE,NPYorpyEMAPS.
- property axes#
The pattern axes, rows then columns: dictionaries with the
name,size,scale(per pixel, after binning),offsetandunits. DigitalMicrograph and EMD files record the calibration, and MRC files a pixel size in Å; for the other formats, the scale is the binning and the units are empty.
- property metadata#
The file’s descriptive metadata. For TIFF files: the first image’s descriptive tags by name (
ImageDescription,Software,DateTime…), thecompression, andimagejorshapedwith the settings ImageJ or tifffile recorded. For Velox files:high_voltage_kV,camera_length_m,detectorandinstrument, and the full metadata of the first frame undervelox. For Berkeley EMD files: theformat, thedatasetand itsattributes, and thescan_axes. For DigitalMicrograph files:high_voltage_kVandcamera_length_mmwhen recorded,scan_axes, the image’sname, and its tags undertags. For MRC files: themode,labelsand extended header type. For Merlin files: thepixel_type,exposure_s, the first frame’sheaderfields, and the settings of the.hdrfile next to it underhdr. For pictures: theformat,modeand whether it islossy(JPEG).
- property source#
The path of the file.
- pyemaps.expimage.image_readers()#
The readers
read_image()uses, with their file extensions.- Returns:
(name, extensions)pairs.- Return type:
list of tuple
- pyemaps.expimage.read_image(path, reader=None, binning=1, scan_roi=None, scan_shape=None, dataset=None)#
Read experimental diffraction patterns from a file.
- Parameters:
path (str or os.PathLike) – the file. TIFF, DigitalMicrograph and EMD (HDF5) files are recognised by their content, whatever their extension;
.im3and.imgfiles by their header.reader (str, optional) – the reader to use whatever the file’s extension, such as
'TIFF'or'MRC'. Seeimage_readers().binning (int, optional) – average square blocks of binning x binning pixels into one. Pixels left over at the right and bottom edges are dropped. The camera constant and pattern center you give
indexImage()must be scaled by1 / binningtoo.scan_roi (tuple of int, optional) – the part of the scan to read:
(row, column, rows, columns)for a 2-D scan,(first, count)for a series of patterns, counting from 0. Only those patterns are read from a TIFF file.scan_shape (tuple of int, optional) – the scan shape,
(rows, columns), for a stack whose file doesn’t record it, such as an ImageJ stack or a Merlin file. The patterns must fill it exactly. By default, the shape the file recorded, or else a series of patterns.dataset (int or str, optional) – for a DigitalMicrograph or EMD file with several images, the one to read: its number, counting from 0, or its name in the file. By default, the one with the most patterns, such as the 4D-STEM scan rather than an overview image.
- Returns:
the patterns.
- Return type:
- Raises:
FileNotFoundError – if the file doesn’t exist.
ImportError – for an EMD file, if h5py isn’t installed. The message says how to install it.
ValueError – if the file isn’t one these readers read, holds no grayscale image (an RGB image, for example), uses a compression they can’t decode, or an option is invalid. The message says which.
- pyemaps.expimage.write_im3(path, data)#
Write patterns to a pyEMAPS small-header file (
.im3), which the 4D-STEM features read.- Parameters:
path (str or os.PathLike) – the file to write.
data (numpy.ndarray) – one pattern
(rows, columns)or a stack(patterns, rows, columns). 16-bit integers are written as such; everything else as 32-bit floats.