Code for create Dataset3

Dataset Partition Generation and Reproducibility This repository provides the image-level partitioning procedure used to construct the three labeled dataset configurations evaluated in the study: Dataset 1, Dataset 2 (Reference-like), and Dataset 3. The same mixed-integer linear programming (MILP) framework is used as the common partitioning engine. The three configurations differ only in their target numbers of images and class annotations assigned to the training, validation, and test subsets. 1. Input annotation format The partitioning script expects a CSV annotation file containing at least the following columns: filename class xmin ymin xmax ymax The filename column identifies the source aerial image, class contains one of the four tree-species class identifiers (1, 2, 3, or 4), and the remaining columns define the bounding box coordinates. Multiple bounding boxes may occur in the same image. All annotations belonging to a given image are always assigned to the same dataset subset. The four class identifiers are: Class 1: Norway spruce Class 2: Silver fir Class 3: Scots pine Class 4: European beech 2. Image-level partitioning Partitioning is performed at the image level, rather than independently at the annotation level. For each image, the script first constructs a four-dimensional class-count vector: [n(C1), n(C2), n(C3), n(C4)] where each element represents the number of bounding boxes of the corresponding tree species in that image. The complete annotation dataset is therefore represented as an image-by-class count matrix. This representation allows the optimizer to assign complete images to the training, validation, or test subset while simultaneously considering the class composition of each image. This design prevents annotations belonging to the same aerial image from appearing in more than one subset. 3. MILP formulation The partitioning problem is formulated as a mixed-integer linear program. For each image (i) and dataset subset (s), a binary decision variable is defined: [x_{i,s} =\begin{cases}1, & \text{if image } i \text{ is assigned to subset } s,\0, & \text{otherwise.}\end{cases}] The optimization is subject to three main conditions. 3.1 Unique image assignment Each image must belong to exactly one subset: [\sum_s x_{i,s}=1.] Consequently, an image cannot simultaneously occur in the training, validation, and test partitions. 3.2 Target image counts The number of images assigned to each subset is constrained according to the target configuration. For subset (s), [\sum_i x_{i,s}=N_s,] where (N_s) is the target number of images in that subset. 3.3 Class-distribution objective Because an aerial image may contain several objects from one or more species, exact class balancing cannot generally be achieved by independently assigning bounding boxes. Instead, positive and negative deviation variables are introduced for each subset/class combination. The optimizer minimizes the deviation between the number of annotations assigned to a subset and the corresponding target class count. The deviations are normalized relative to the class-specific target: [w_{s,c}=\frac{1}{\max(1,T_{s,c})},] where (T_{s,c}) is the target number of annotations for class (c) in subset (s). The resulting objective is [\min\sum_{s,c}w_{s,c}\left(d^{+}{s,c}+d^{-}{s,c}\right).] This normalization prevents classes containing large numbers of annotations from dominating the optimization and gives proportionally greater importance to deviations in less frequent classes. The optimization is solved using scipy.optimize.milp, which uses the HiGHS optimization backend. 4. Dataset configurations The same optimization procedure can be used for all three dataset configurations by changing the target image counts and target class counts. The split order used below is: [train, validation, test] and the class order is: [C1, C2, C3, C4] 4.1 Dataset 1 Dataset 1 represents the first experimental partition of the original labeled collection. The published Dataset 1 contains: target_image_counts = np.array([ 2539, # train 1003, # validation 896, # test ]) The corresponding class-annotation targets are: target_class_counts = np.array([ [1486, 1281, 494, 2236], # train [ 865, 626, 52, 762], # validation [1098, 444, 47, 737], # test ]) This produces the following aggregate partition: Split Images C1 C2 C3 C4 Total boxes Train 2,539 1,486 1,281 494 2,236 5,497 Validation 1,003 865 626 52 762 2,305 Test 896 1,098 444 47 737 2,326 The corresponding image shares are approximately: 57.21% / 22.60% / 20.19% for training, validation, and test, respectively. Although Dataset 1 originated from a nominal 50/25/25 partitioning design, the archived experimental partition has the image counts reported above. The fixed targets should therefore be used when reproducing the Dataset 1 configuration reported in the study. 4.2 Dataset 2 (Reference-like) Dataset 2 was designed as a reference-like partition and preserves the image-level separation between training, validation, and test data. The target image counts are: target_image_counts = np.array([ 3707, # train 412, # validation 319, # test ]) The class targets are: target_class_counts = np.array([ [2714, 2007, 511, 3229], # train [ 302, 224, 57, 359], # validation [ 433, 120, 25, 147], # test ]) The resulting aggregate partition is: Split Images C1 C2 C3 C4 Total boxes Train 3,707 2,714 2,007 511 3,229 8,461 Validation 412 302 224 57 359 942 Test 319 433 120 25 147 725 Dataset 2 is referred to as Reference-like throughout the study. 4.3 Dataset 3 Dataset 3 was constructed to provide a more distributionally consistent 60/20/20 image-level partition. The target image counts are: target_image_counts = np.array([ 2663, # train 888, # validation 887, # test ]) The corresponding class targets are: target_class_counts = np.array([ [2069, 1411, 356, 2242], # train [ 690, 470, 119, 747], # validation [ 690, 470, 118, 746], # test ]) The resulting aggregate partition is: Split Images C1 C2 C3 C4 Total boxes Train 2,663 2,069 1,411 356 2,242 6,078 Validation 888 690 470 119 747 2,026 Test 887 690 470 118 746 2,024 Dataset 3 therefore contains approximately: 60% training 20% validation 20% test of the original labeled images. The objective of this configuration was not to create equal numbers of examples for the four tree species. Instead, it was designed to preserve more similar class-distribution patterns across the training, validation, and test subsets while maintaining image-level separation. 5. Configuring the script The optimization function itself does not need to be changed between the three configurations. To reproduce a particular dataset, set: SPLIT_NAMES = ["train", "val", "test"] and specify the corresponding target_image_counts and target_class_counts before calling: assignment = optimize_split( image_class_counts=image_class_counts, target_image_counts=target_image_counts, target_class_counts=target_class_counts, ) For example, Dataset 3 can be configured as: target_image_counts = np.array([ 2663, 888, 887, ], dtype=int) target_class_counts = np.array([ [2069, 1411, 356, 2242], [ 690, 470, 119, 747], [ 690, 470, 118, 746], ], dtype=int) The same code can then be run using the Dataset 1 or Dataset 2 targets listed above. 6. Ratio-based generation The script also supports automatic generation of targets from desired split ratios using the largest-remainder procedure. For example: SPLIT_RATIOS = np.array([0.60, 0.20, 0.20]) produces the Dataset 3 target image counts: 2663 / 888 / 887 from the 4,438 labeled images. Similarly, a nominal 50/25/25 configuration can be requested using: SPLIT_RATIOS = np.array([0.50, 0.25, 0.25]) However, ratio-based generation should not be used when the objective is to reproduce the exact aggregate counts of the archived experimental partitions. For exact reproduction of the reported configurations, the fixed target arrays provided above should be used. 7. Leakage verification After optimization, the script explicitly verifies that each filename occurs in only one partition. Thus, Train ∩ Validation = ∅ Train ∩ Test = ∅ Validation ∩ Test = ∅ at the image level. This is particularly important for object-detection datasets because several bounding boxes can originate from the same aerial image. 8. Generated files For each configuration, the script produces separate annotation CSV files for: train validation test and additionally generates: split_manifest.csv split_summary.csv The manifest records the subset assigned to each image, while the summary reports the number of images, bounding boxes, and class-specific annotations in each subset. 9. Exact reproducibility The optimization procedure reproduces the partitioning methodology and aggregate targets used in the study. Because a mixed-integer optimization problem can contain multiple equally optimal image assignments, identical aggregate counts do not necessarily imply identical filename membership across independent solver executions or software versions. For this reason, the Zenodo archive additionally provides the final split manifests used in the experiments. The manifest files should be regarded as the authoritative source when exact reproduction of the training, validation, and test image membership used in the reported experiments is required. The optimization script is provided to document and reproduce the procedure through which image-level, class-aware dataset partitions can be constructed. 10. Summary All three dataset configurations are derived from the same labeled image collection and use the same four tree-species classes. Their principal difference is the distribution of images and annotations among the training, validation, and test subsets. The common partitioning framework: groups annotations by image; constructs class-count vectors for each image; assigns each complete image to exactly one subset; enforces the requested number of images in each subset; minimizes normalized deviations from class-specific annotation targets; verifies the absence of image-level overlap; and exports the resulting annotation files, manifest, and partition summary. This common optimization framework enables controlled investigation of how dataset partition structure affects the performance of subsequent tree-species detection and pseudo-labeling experiments.