Experimental Dataset and Python Code for AdaMax Optimization: Normalized Spatial Weight-Update Dynamics, Regime Mapping, and Early-Stage Indicators on CIFAR-10

Overview This repository provides experimental datasets, numerical results, visualization outputs, and Python source code for investigating the spatial organization of adaptive weight corrections during neural network training with the AdaMax optimizer on CIFAR-10. The dataset supports the research framework presented in the manuscript "Normalized Spatial Synchronization Analysis of Adaptive Weight Corrections for Regime Mapping in Adam Optimization of Neural Networks." It extends the normalized pairwise weight-update methodology to AdaMax, an infinity-norm-based variant of Adam, and provides data for examining optimizer-dependent correction heterogeneity, empirical training-regime maps, and early-stage correction descriptors. The central objective is to characterize how the internal organization of optimizer-induced weight updates depends on the first- and second-moment parameters (β1, β2), the learning rate, and the dimensionality of the input representation. 1. Experimental Configuration The experiments are based on the CIFAR-10 image classification dataset and a multilayer perceptron (MLP) with one hidden layer containing 50 sigmoid neurons and a single sigmoid output neuron. The computational protocol includes: Dataset: CIFAR-10. Classification task: binary one-class-versus-rest classification for the ten CIFAR-10 classes. Input representation: grayscale conversion followed by random linear projection. Input dimensionalities: d = 32, 256, and 2048. Optimizer: AdaMax. Learning rate: α = 0.3 for the supplied source-code configuration. First-moment parameter: 0 ≤ β1 < 1. Second-moment parameter: 0 ≤ β2 < 1. Parameter-grid spacing: Δβ1 = Δβ2 = 0.002. Hidden-layer width: 50 neurons. Training duration: 5 epochs per parameter configuration. Mini-batch size: 64. Positive training examples: 40 per target class. Binary training-set size: 80 examples per classification task. Each optimizer configuration is evaluated through a separate training procedure under the specified experimental conditions. 2. Normalized Spatial Weight-Update Observables The methodology analyzes the actual neuron-wise weight corrections produced by AdaMax rather than relying exclusively on gradients or global training metrics. For each hidden neuron i, the correction magnitude x_i(t) is defined as the Euclidean norm of its optimizer-induced weight-update vector. Two principal dimensionless observables are calculated for neighboring neuron indices: Normalized pairwise correction amplitude (A_i): A_i(t) = [x_i(t)^2 + x_(i+1)(t)^2] / [2⟨x(t)^2⟩ + ε] This quantity represents the intensity of neighboring weight corrections relative to the mean-square correction level of the hidden layer. Local correction mismatch or synchronization-loss coefficient (S_i): S_i(t) = |x_(i+1)(t) − x_i(t)| / [x_(i+1)(t) + x_i(t) + ε] The coefficient quantifies the relative difference between correction magnitudes associated with consecutive neuron indices. Additionally, the global spatial synchronization-loss measure is calculated as the root-mean-square value of S_i across adjacent neuron pairs. Normalized individual-neuron correction profiles are also recorded. These observables describe the spatial organization of adaptive updates under a fixed neuron-index ordering. They do not imply physical coupling between adjacent neurons or establish conventional dynamical synchronization. 3. Empirical Optimization-Regime Mapping The dataset enables the construction and comparison of optimizer-parameter maps in the (β1, β2) plane. The investigated characteristics include: Maximum normalized pairwise amplitude (A_max). Standard deviation of normalized pairwise amplitude (A_std). Mean global synchronization-loss coefficient (S_mean). Maximum local correction mismatch (S_local_max). Normalized individual-neuron correction profiles. Final mean-squared error and classification accuracy. Gradient and weight-update norms. The numerical outputs support two-dimensional parameter maps, three-dimensional surfaces, and spatially resolved fields describing correction heterogeneity as a function of neuron-pair index and optimizer parameters. The full spatial representations retain the coordinates (i, β1, β2), with the corresponding normalized observable encoded as the field value. These maps identify empirical differences in correction organization and should not be interpreted as rigorous evidence of mathematical bifurcations or chaotic attractors. 4. Early-Stage Correction Descriptors The accompanying Python program calculates early-stage correction statistics using the first 30% of optimizer-update steps. The recorded descriptors include: Early normalized amplitude mean, standard deviation, and maximum. Early global synchronization-loss mean and standard deviation. Early training loss. Early gradient norm. Early optimizer-update norm. Early update variance and coefficient of variation. These quantities enable exploratory comparisons between normalized spatial correction characteristics and conventional optimization diagnostics. The dataset provides a basis for investigating potential early indicators of changes in optimizer dynamics. However, the supplied AdaMax experiments alone do not establish validated prediction of future optimization failure, detection lead time, or false-alarm performance. 5. Repository Contents The deposited materials include: Python source code AdaMax_cifar10_AS_early_warning_COMPLETE.py The script implements CIFAR-10 preprocessing, binary task construction, MLP training, AdaMax weight updates, normalized spatial observables, parameter-grid scanning, numerical result export, and visualization generation. Experimental result archives The ZIP archives contain outputs from AdaMax parameter-grid experiments. The program generates numerical summaries, compressed NumPy matrices, regime maps, and spatial visualizations, including: CSV summaries of optimizer configurations and correction descriptors. NPZ arrays containing parameter grids and normalized observable fields. Two-dimensional empirical regime maps. Three-dimensional parameter surfaces. Neuron-pair-resolved A_i and S_i profiles. Color-coded spatial fields in the coordinates (i, β1, β2). Individual neuron-pair surfaces. The deposited archives correspond to the experimental configurations indicated in their filenames. The source code permits regeneration of the outputs under the specified settings. 6. Scientific Relevance and Reuse This dataset supports research on: Adaptive neural network optimization. Spatial heterogeneity of optimizer-induced weight corrections. Normalized pairwise correction observables. Empirical optimization-regime mapping. Comparison of Adam-family optimization algorithms. Early-stage analysis of neural network training dynamics. Reproducible numerical investigations of optimizer hyperparameters. The materials can be used to compare conventional loss- and gradient-based diagnostics with normalized descriptions of internal weight-update organization, investigate the dependence of correction patterns on optimizer parameters, and develop further experimental protocols for training-dynamics monitoring. The dataset is intended for scientific analysis and methodological reproducibility. It does not independently establish universal instability thresholds, deterministic chaos, or operationally validated early-warning performance.