Chess Recognition Dataset (ChessReD)

<strong>Chess Recognition Dataset (ChessReD)</strong>The Chess Recognition Dataset (ChessReD) comprises a diverse collection of images of chess formations captured using smartphone cameras; a sensor choice made to ensure real-world applicability. <br>It was collected by capturing images of chessboards with various chess piece configurations. The chess opening theory was used to guarantee the variability of those configurations. In particular, the Encyclopaedia of Chess Openings (ECO) classifies opening sequences into five volumes with 100 subcategories each that are uniquely identified by an ECO code. 20 ECO codes were selected from each volume. Subsequently, each code of this set was randomly matched to an already played chess game that followed the particular opening sequence denoted by the ECO code; thus creating a set of 100 chess games. Finally, using the move-by-move information provided by Portable Game Notations (PGNs) that are used to record chess games, the selected games were played out on a physical chessboard with images being captured after each move.<br>Three distinct smartphone models were used to capture the images. Each model has different camera specifications, such as resolution and sensor type, that introduce further variability in the dataset. The images were also taken from diverse angles, ranging from top-view to oblique angles, and from different perspectives (e.g., white player perspective, side view, etc.). These conditions simulate real-world scenarios where chessboards can be captured from a bystander's arbitrary point of view. Additionally, the dataset includes images captured under different lighting conditions, with both natural and artificial light sources introducing these variations.<br>The dataset is accompanied by detailed annotations providing information about the chess pieces formation in the images. Therefore, the number of annotations for each image depends on the number of chess pieces depicted in it. There are 12 category ids in total (i.e., 6 piece types per colour) and the chessboard coordinates are in the form of algebraic notation strings (e.g., "a8"). These annotations were automatically extracted from Forsyth-Edwards Notations (FENs) that were available by the games' PGNs. Each FEN string describes the state of the chessboard after each move using algebraic notation for the piece types (e.g., "N" is knight) , capitalization for the piece colours (i.e., white pieces are denoted with uppercase letters, while black pieces with lowercase letters), and digits to denote the number of empty squares. Thus, by matching the captured images to the corresponding FENs, the state of the chessboard in each image was already known and annotations could be extracted. To further facilitate research in the chess recognition domain, bounding-box and chessboard corner annotations are also provided for a subset of 20 chess games. The corners are annotated based on their location on the chessboard (e.g., "bottom-left") with respect to the white player's view. This discrimination between these different types of corners provides information about the orientation of the chessboard that can be leveraged to determine the image's perspective and viewing angle.<br><strong>Dataset specifications</strong>The dataset consists of 100 chess games, each with an arbitrary number of moves and therefore images, amounting to a total of 10,800 images being collected. It was split into training, validation, and test sets following an 60/20/20 split, which led to a total of 6,479 training images, 2,192 validation images, and 2,129 test images. Since two consecutive images of a chess game differ only by one move, the split was performed on game-level to ensure that quite similar images would not end up in different sets. The split was also stratified over the three distinct smartphone cameras (Apple iPhone 12, Huawei P40 pro, Samsung Galaxy S8) that were used to capture the images. The three smartphone cameras introduced variations to the dataset based on the distinct characteristics of their sensors. For instance, while the image resolution for the Huawei phone was 3072x3072, the resolution for the remaining two models was 3024x3024.<br>While annotations about the position of the pieces in algebraic notation are available for every image in the dataset, bounding box and chessboard corner annotations are provided only for a subset of 20 randomly selected games from the train, validation, and test sets. For this subset a 70/15/15 split stratified over the smartphone cameras was followed, which led to a total of 14 training games (1,442 images), 3 validation games (330 images), and 3 test games (306 images) being annotated. This subset of ChessReD is denoted as ChessReD2K.<br><strong>国际象棋识别数据集（Chess Recognition Dataset，ChessReD）</strong>本国际象棋识别数据集（ChessReD）包含了使用智能手机摄像头采集的多样化国际象棋棋局图像——选用手机摄像头是为了确保数据集具备实际应用可行性。<br>该数据集通过采集带有不同棋子布局的国际象棋棋盘图像构建而成，我们采用国际象棋开局理论来确保这些布局的多样性。具体而言，《国际象棋开局百科》（Encyclopaedia of Chess Openings，ECO）将开局序列划分为5卷，每卷包含100个子类别，每个类别由唯一的ECO代码标识。我们从每一卷中选取20个ECO代码，随后将每个选取的代码随机匹配至一局已完成的、遵循该ECO代码对应开局序列的国际象棋对局，最终得到100组国际象棋对局。最后，借助用于记录国际象棋对局的可移植棋局记法（Portable Game Notations，PGN）所提供的逐步走棋信息，我们在实体棋盘上复现这些选定的对局，并在每一步走棋后拍摄图像。<br>本次图像采集使用了三款不同型号的智能手机，每款机型的摄像头参数（如分辨率、传感器类型）各不相同，这进一步增加了数据集的多样性。采集的图像涵盖了多种拍摄角度，从俯视到斜拍均有覆盖，同时包含不同视角（如白方视角、侧面视角等）。这些设置旨在模拟现实场景中旁观者以任意视角拍摄国际象棋棋盘的情况。此外，数据集还包含不同光照条件下拍摄的图像，自然光与人工光源均被用于引入光照差异。<br>本数据集附带详细的标注信息，用于描述图像中的国际象棋棋子布局。因此，每张图像的标注数量取决于其中呈现的棋子数量。标注总共有12个类别ID（即每方阵营各6种棋子类型），棋盘坐标采用代数记法字符串形式（例如"a8"）。这些标注可从对局的PGN文件中获取的福赛斯-爱德华兹记法（Forsyth-Edwards Notations，FEN）自动提取。每个FEN字符串通过以下方式描述每步走棋后的棋盘状态：使用代数记法表示棋子类型（例如"N"代表骑士），通过大小写区分棋子阵营（白方棋子使用大写字母，黑方棋子使用小写字母），并用数字表示空白方格的数量。因此，通过将拍摄的图像与对应的FEN字符串进行匹配，即可确定每张图像中的棋盘状态并提取标注。为进一步推动国际象棋识别领域的研究，我们还为20局对局的子集提供了边界框与棋盘角点标注。角点标注基于白方视角下的棋盘位置进行标识（例如"bottom-left"，即左下角）。对这些不同类型角点的区分，能够提供棋盘朝向的相关信息，可用于确定图像的拍摄视角与角度。<br><strong>数据集规格参数</strong>本数据集包含100局国际象棋对局，每局对局的走棋步数任意，因此对应的图像数量也各不相同，最终共采集得到10800张图像。数据集按照60/20/20的比例划分为训练集、验证集与测试集，其中训练集包含6479张图像，验证集包含2192张图像，测试集包含2129张图像。由于同一对局的连续两张图像仅相差一步走棋，因此我们按对局级别进行划分，以避免高度相似的图像被分配至不同的数据集中。本次划分还针对三款用于采集图像的智能手机（Apple iPhone 12、华为P40 Pro、三星Galaxy S8）进行了分层抽样，以确保各数据集中的设备分布均衡。三款智能手机的传感器特性各不相同，为数据集引入了差异：例如，华为手机的图像分辨率为3072×3072，其余两款机型的分辨率均为3024×3024。<br>尽管数据集中的每张图像都附带了以代数记法表示的棋子位置标注，但边界框与棋盘角点标注仅针对从训练集、验证集与测试集中随机选取的20局对局子集提供。针对该子集，我们同样按照70/15/15的比例进行分层划分（分层依据为采集图像所用的智能手机型号），最终得到14局训练对局（含1442张图像）、3局验证对局（含330张图像）与3局测试对局（含306张图像）。该ChessReD的子集被命名为ChessReD2K。