Chess Recognition Dataset (ChessReD)

Chess Recognition Dataset (ChessReD)The Chess Recognition Dataset (ChessReD) comprises a diverse collection of images of chess formations captured using smartphone cameras; a sensor choice made to ensure real-world applicability. It was collected by capturing images of chessboards with various chess piece configurations. The chess opening theory was used to guarantee the variability of those configurations. In particular, the Encyclopaedia of Chess Openings (ECO) classifies opening sequences into five volumes with 100 subcategories each that are uniquely identified by an ECO code. 20 ECO codes were selected from each volume. Subsequently, each code of this set was randomly matched to an already played chess game that followed the particular opening sequence denoted by the ECO code; thus creating a set of 100 chess games. Finally, using the move-by-move information provided by Portable Game Notations (PGNs) that are used to record chess games, the selected games were played out on a physical chessboard with images being captured after each move. Three distinct smartphone models were used to capture the images. Each model has different camera specifications, such as resolution and sensor type, that introduce further variability in the dataset. The images were also taken from diverse angles, ranging from top-view to oblique angles, and from different perspectives (e.g., white player perspective, side view, etc.). These conditions simulate real-world scenarios where chessboards can be captured from a bystander's arbitrary point of view. Additionally, the dataset includes images captured under different lighting conditions, with both natural and artificial light sources introducing these variations. The dataset is accompanied by detailed annotations providing information about the chess pieces formation in the images. Therefore, the number of annotations for each image depends on the number of chess pieces depicted in it. There are 12 category ids in total (i.e., 6 piece types per colour) and the chessboard coordinates are in the form of algebraic notation strings (e.g., "a8"). These annotations were automatically extracted from Forsyth-Edwards Notations (FENs) that were available by the games' PGNs. Each FEN string describes the state of the chessboard after each move using algebraic notation for the piece types (e.g., "N" is knight) , capitalization for the piece colours (i.e., white pieces are denoted with uppercase letters, while black pieces with lowercase letters), and digits to denote the number of empty squares. Thus, by matching the captured images to the corresponding FENs, the state of the chessboard in each image was already known and annotations could be extracted. To further facilitate research in the chess recognition domain, bounding-box and chessboard corner annotations are also provided for a subset of 20 chess games. The corners are annotated based on their location on the chessboard (e.g., "bottom-left") with respect to the white player's view. This discrimination between these different types of corners provides information about the orientation of the chessboard that can be leveraged to determine the image's perspective and viewing angle. Dataset specificationsThe dataset consists of 100 chess games, each with an arbitrary number of moves and therefore images, amounting to a total of 10,800 images being collected. It was split into training, validation, and test sets following an 60/20/20 split, which led to a total of 6,479 training images, 2,192 validation images, and 2,129 test images. Since two consecutive images of a chess game differ only by one move, the split was performed on game-level to ensure that quite similar images would not end up in different sets. The split was also stratified over the three distinct smartphone cameras (Apple iPhone 12, Huawei P40 pro, Samsung Galaxy S8) that were used to capture the images. The three smartphone cameras introduced variations to the dataset based on the distinct characteristics of their sensors. For instance, while the image resolution for the Huawei phone was 3072x3072, the resolution for the remaining two models was 3024x3024. While annotations about the position of the pieces in algebraic notation are available for every image in the dataset, bounding box and chessboard corner annotations are provided only for a subset of 20 randomly selected games from the train, validation, and test sets. For this subset a 70/15/15 split stratified over the smartphone cameras was followed, which led to a total of 14 training games (1,442 images), 3 validation games (330 images), and 3 test games (306 images) being annotated. This subset of ChessReD is denoted as ChessReD2K.

象棋识别数据集（Chess Recognition Dataset，ChessReD）包含使用智能手机摄像头采集的多样化象棋布局图像——选用手机摄像头旨在确保数据集具备真实世界适用性。该数据集通过采集不同象棋棋子布局的棋盘图像构建，布局多样性则依托象棋开局理论予以保障。具体而言，《象棋开局百科全书》（Encyclopaedia of Chess Openings，ECO）将开局序列划分为5卷，每卷含100个子类别，且均通过ECO代码唯一标识。从每卷中选取20个ECO代码，随后将该集合中的每个代码随机匹配至一局已完成的、遵循该ECO代码对应开局序列的象棋对局，最终得到100组象棋对局。

最后，依托用于记录象棋对局的可移植对局格式（Portable Game Notations，PGNs）提供的逐步走棋信息，将选定的对局在实体棋盘上复现，并在每一步走棋后采集图像。本次采集使用了三款不同型号的智能手机，每款机型的摄像头参数（如分辨率、传感器类型）各不相同，进一步增加了数据集的多样性。

图像采集自多种视角，涵盖俯视到斜摄角度，以及不同的观测立场（例如白方视角、侧面视角等），这些设置模拟了旁观者任意视角下拍摄棋盘的真实场景。此外，数据集还包含不同光照条件下采集的图像，自然光与人工光源均引入了光照变化。

该数据集附带详细的标注信息，可提供图像内的象棋棋子布局情况。因此，每张图像的标注数量取决于其中呈现的象棋棋子数目。数据集中共包含12个类别ID（即每方阵营各6种棋子类型），棋盘坐标采用代数记谱法（algebraic notation）字符串形式（例如"a8"）。这些标注可从对局PGNs对应的Forsyth-Edwards记谱法（Forsyth-Edwards Notations，FENs）中自动提取，每个FEN字符串通过棋子类型的代数记谱（例如"N"代表骑士）、大小写区分阵营（白方棋子使用大写字母，黑方棋子使用小写字母）以及数字表示空方格数量，来描述每步走棋后的棋盘状态。因此，通过将采集的图像与对应的FEN字符串匹配，即可获知每张图像内的棋盘状态，进而完成标注提取。

为进一步推动象棋识别领域的研究，数据集还为20局随机选取的象棋对局子集提供了边界框（bounding-box）与棋盘角点（chessboard corner）标注。棋盘角点标注基于白方视角下的棋盘位置（例如"左下角"）进行标注，这类不同类型角点的区分可提供棋盘朝向信息，用于推断图像的透视关系与拍摄角度。

数据集规格
该数据集共包含100组象棋对局，每组对局的走棋步数与对应图像数量均不固定，最终累计采集得到10800张图像。数据集按照60/20/20的比例划分为训练集、验证集与测试集，最终分别包含6479张训练图像、2192张验证图像与2129张测试图像。由于象棋对局的连续两张图像仅相差一步走棋，因此按对局级别进行划分，以避免高度相似的图像被分配至不同集合中。本次划分同时针对三款采集所用的智能手机机型（苹果iPhone 12、华为P40 Pro、三星Galaxy S8）进行了分层划分，三款机型的传感器特性差异为数据集引入了额外的采集差异。例如，华为机型的图像分辨率为3072×3072，其余两款机型的分辨率均为3024×3024。

数据集内所有图像均提供了棋子位置的代数记谱标注，但边界框与棋盘角点标注仅针对训练集、验证集与测试集中随机选取的20局对局子集提供。针对该子集，按照70/15/15的比例并结合智能手机机型进行分层划分，最终得到14局训练对局（含1442张图像）、3局验证对局（含330张图像）与3局测试对局（含306张图像）。该ChessReD子集被命名为ChessReD2K。