Phase Object Reconstruction for 4D-STEM using Deep Learning, (4D-STEM Training Data)

<strong>Overview </strong> This repository contains 742,688 samples of simulated Convergent Beam Electron Diffraction patterns (CBEDs); the training data for the paper "Phase Object Reconstruction for 4D-STEM using Deep Learning". The folder contains multiple hdf5 datasets. Each dataset has a corresponding Excel-sheet containing detailed information and simulation parameters for every datapoint, as well as a summary-report containing the parameter distributions, hdf5-infos and random number generator settings. This makes every dataset reproducible, using the simulation codes provided in https://github.com/ThFriedrich/ap_data_generation. <strong>Technical details</strong> Every Datapoint consists of a 3x3 set of adjacent Convergent Beam Electron Diffraction pattern (CBEDs), the coherent exit wave phase and amplitude in real and reciprocal space, and the probe functions phase and amplitude in real space. All patterns are 64x64 pixel in 16 bit unsigned integer data format. Every hdf5 file has the following structure: Attributes 'Seed': 6108236<br> 'State': 251786606 ...<br> 'Type': 'twister'<br> 'arch': 'glnxa64'<br> 'gpu': 'NVIDIA GeForce RTX 3080'<br> 'matlab_ver': '2021a' Dataset 'features' Size: 64x64x9x5000<br> Datatype: H5T_STD_U16LE (uint16) Dataset 'labels_k' Size: 64x64x2x5000<br> Datatype: H5T_STD_U16LE (uint16) Dataset 'labels_r' Size: 64x64x2x5000<br> Datatype: H5T_STD_U16LE (uint16) Dataset 'probe_r' Size: 64x64x2x5000<br> Datatype: H5T_STD_U16LE (uint16) Dataset 'meta' Size: 19x5000<br> Datatype: H5T_IEEE_F32LE (single) The data was written to hdf5 in matlab. When reading from these files consider possibly different storage conventions (Row major vs. column major format). Data may need to be transposed accordingly. The integer arrays were scaled to use the full range of the uint16 datatype. The scaling values are stored under "meta". To restore the original values in floating point numbers, convert the arrays like this: Matlab: <pre><code>hdf_file = ['db_h5_b_5_Training.h5']; n = 128; % load `n` k-space exit waves x = single(h5read(hdf_file, '/labels_k', [1,1,1,1], [64,64,2,n])); % `meta` contains parameters and scaling factors for a given datapoint in following order: [E_0(keV), cond_lens_outer_aper_ang(mrad), collection angle(rA), step_size(A), scale_cbed_1 ... scale_cbed_9, scale_phase_k, scale_amp_k, scale_phase_r, scale_amp_r, scale_probe_phase_r, scale_probe_amp_r] s = h5read(hdf_file, '/meta', [14,1], [2,n]); amplitude = zeros(64,64,n); phase = zeros(64,64,n); for ix = 1:n phase(:,:,n) = (x(:,:,1,n)*s(1,ix) / 65536) - pi; amplitude(:,:,n) = (x(:,:,2,n)*s(2,ix)) / 65536; end % The 9 CBEDs correspond to a 3x3 kernel of patterns. The order in [x,y] is: %[[3, 6, 9]; % [2, 5, 8]; % [1, 4, 7]] </code></pre><strong>概述</strong> 本仓库包含742,688份模拟会聚束电子衍射（Convergent Beam Electron Diffraction, CBED）图案样本，为论文《基于深度学习的4D-STEM相位物体重建》所用的训练数据。该文件夹包含多个HDF5（Hierarchical Data Format 5）格式数据集，每个数据集均配有对应的Excel表格，记录了每条数据的详细信息与仿真参数，同时附带一份汇总报告，涵盖参数分布、HDF5文件信息以及随机数生成器设置。借助https://github.com/ThFriedrich/ap_data_generation 中提供的仿真代码，所有数据集均可复现。 <strong>技术细节</strong> 每条数据由一组3×3的相邻会聚束电子衍射（CBED）图案、实空间与倒易空间中的相干出射波相位与振幅，以及实空间中的探针函数相位与振幅构成。所有图案均为64×64像素，采用16位无符号整数数据格式。每个HDF5文件包含以下结构： 属性： 'Seed': 6108236 'State': 251786606 'Type': 'twister' 'arch': 'glnxa64' 'gpu': 'NVIDIA GeForce RTX 3080' 'matlab_ver': '2021a' 数据集'features': 尺寸为64×64×9×5000，数据类型为H5T_STD_U16LE（uint16） 数据集'labels_k': 尺寸为64×64×2×5000，数据类型为H5T_STD_U16LE（uint16） 数据集'labels_r': 尺寸为64×64×2×5000，数据类型为H5T_STD_U16LE（uint16） 数据集'probe_r': 尺寸为64×64×2×5000，数据类型为H5T_STD_U16LE（uint16） 数据集'meta': 尺寸为19×5000，数据类型为H5T_IEEE_F32LE（单精度浮点数） 本数据使用Matlab写入HDF5文件，读取时需注意不同的存储约定（行优先与列优先格式），可能需要对数据进行转置处理。 整数数组已按uint16数据类型的全范围进行缩放，缩放值存储于'meta'字段中。若要恢复为浮点数格式的原始数值，可按如下方式转换数组： Matlab代码示例： <pre><code>hdf_file = ['db_h5_b_5_Training.h5']; n = 128; % 加载`n`个k空间出射波 x = single(h5read(hdf_file, '/labels_k', [1,1,1,1], [64,64,2,n])); % `meta`包含给定数据点的参数与缩放因子，顺序如下： % [E_0(keV), cond_lens_outer_aper_ang(mrad), collection_angle(rA), step_size(A), scale_cbed_1 ... scale_cbed_9, scale_phase_k, scale_amp_k, scale_phase_r, scale_amp_r, scale_probe_phase_r, scale_probe_amp_r] s = h5read(hdf_file, '/meta', [14,1], [2,n]); amplitude = zeros(64,64,n); phase = zeros(64,64,n); for ix = 1:n phase(:,:,ix) = (x(:,:,1,ix)*s(1,ix) / 65536) - pi; amplitude(:,:,ix) = (x(:,:,2,ix)*s(2,ix)) / 65536; end % 9张CBED图案对应3×3的邻域衍射花样，其[x,y]顺序为： %[[3, 6, 9]; % [2, 5, 8]; % [1, 4, 7]] </code></pre>