ydk

# YdkGen: Transformer-Based Yu-Gi-Oh! Deck Generator (Pure PyTorch) A PyTorch-native causal transformer model purpose-built for generating valid Yu-Gi-Oh! deck lists (`.ydk` files). This model is adapted to the unique constraints of trading card game (TCG) deck building—departing from standard causal language models to support unordered, repeatable card collections with Yu-Gi-Oh!-specific format and banlist rules. [Github](https://github.com/zisisnotzis/ydkgen). ## Model Architecture YdkGen implements a **causal transformer optimized for unordered, repeatable class collections**, with core design modifications to address the fundamental differences between TCG deck generation and traditional language modeling. Unlike standard causal LMs, deck generation requires handling: - Unordered card sequences (deck order does not impact play) - Repeatable cards (0–3 copies per card, per Yu-Gi-Oh! rules) - Effective duplicate ID restrictions - Fixed main/extra deck card count limits - Banlist (Limited/Forbidden) card restrictions - Continuously growing vocabulary (new Yu-Gi-Oh! cards are regularly released) - Ambiguous "correctness" (no single optimal deck build for a meta/archetype) - Under-specified deck building logic (context-dependent play patterns) ### Key Design Choices To resolve these challenges, the model uses a **minimal causal transformer backbone** paired with heavy train/test-time data augmentation and beam search inference. Critical architectural optimizations include: 1. **Prior Knowledge Embedding**: Injects all card metadata (from `cards.cdb`, `strings.conf`, and Lua scripts) directly into the embedding layer—eliminating hard-coded card ID dependencies in the embedding space and reducing vocabulary overfitting. 2. **Unseen Card Compatibility**: Dropout applied directly to card ID inputs enables theoretical support for unseen cards (with unmapped IDs) by falling back to non-ID metadata pathways (untested). 3. **Batch Construction**: Decks are shuffled and zero-padded to form fixed-length batches, with causal training used as a flexible train-all-length strategy for variable deck sizes. 4. **Multi-Mode Loss Function**: Three specialized loss modes tailor the model’s prediction objective to different deck generation strategies (details below). ### Loss Modes The model supports three distinct prediction loss modes, each optimizing for a different deck distribution forecasting strategy: - **MODE 0**: Predicts the **full deck card distribution** (normalized softmax) at every token step. The full deck can theoretically be inferred from the first token; greedy sampling of the highest-confidence card (iteratively) improves generation accuracy (analogous to diffusion model sampling). - **MODE 1**: Predicts the **remaining deck card distribution** (normalized softmax) at every token step (outputs class 0 if the remaining deck is empty). - **MODE 2**: Predicts **logits for any remaining card** (all valid remaining cards score equally). Probabilities are aggregated via `logsumexp`—*this mode exhibits poor performance and is provided only as a baseline*. ### Inference Inference compensates for the selected training loss mode and leverages **beam search** to enhance generation quality and adherence to Yu-Gi-Oh! deck rules. ## Prerequisites ### Dependencies Install required Python packages via `pip`: ```bash pip install -r requirements.txt ``` ### Required Data Files The model relies on Yu-Gi-Oh! card/format metadata and training deck data—place these files in the project root with the following structure (copy from [YGOPro](https://github.com/Fluorohydride/ygopro) or it's successors): - `deck/*.ydk` or `deck/*/*.ydk`: Training data (valid Yu-Gi-Oh! deck lists) - `cards.cdb`: Comprehensive card metadata (stats, archetypes, types, etc.) - `lflist.conf`: Official Yu-Gi-Oh! banlist (Limited/Forbidden/Restricted; uses the latest version by default) - `strings.conf`: Card set name metadata (**currently unused** in the current implementation) - `script/c*.lua`: Card effect/implementation logic (for advanced metadata extraction) ## Training Initiate training with the default configuration using the following command: ```bash python train.py ``` ### Custom Training Configuration All hyperparameters and training settings are configurable directly in `train.py` for custom runs. Below are the key tunable parameters with their default values and descriptions: ```python # Model Checkpoint Configuration FILE = 'model0' # Auto-saves/loads checkpoints to/from {FILE}{N}.pth # Hardware/Precision DEV = 'cuda' # Use 'cpu' for systems without a CUDA-enabled GPU AMP = torch.float16 # Use torch.float32 if DEV = 'cpu' (no mixed precision) # Transformer Architecture DIM = 256 # Core model embedding/hidden dimension LAYER = 5 # Number of transformer encoder/decoder layers HDIM = 16 # Dimension per attention head MLP = 4/3 # MLP hidden dimension scale factor (relative to DIM) DROP = 0.1 # Global dropout rate (applied to embeddings/attention/MLP) # Optimization LR = 0.0007 # AdamW learning rate WD = 1e-4 # AdamW weight decay (regularization) BS = 16 # Training batch size (adjust for GPU memory) # Training Loop EPOCH = 1000 # Number of checkpoint save iterations ITER = 20000 # Training steps per checkpoint save # Loss Mode (match inference mode) MODE = 0 # 0: Full deck distribution (softmax) at each token # 1: Remaining deck distribution (softmax) at each token # 2: Remaining card logits (logsumexp aggregation; baseline) ``` ## Pretrained Checkpoints Pretrained model checkpoints and tokenizer data are available via multiple mirrors (Hugging Face, HF Mirror, ModelScope) for immediate inference/finetuning. All checkpoints adhere to Yu-Gi-Oh! format rules and are trained on the provided YDK dataset. | Name | Mirror 1 (Hugging Face) | Mirror 2 (HF Mirror) | Mirror 3 (ModelScope) | Description | |---------------|-------------------------|----------------------|----------------------|-----------------------------------------------------------------------------| | [model.i](https://huggingface.co/zisisnotzis/ydkgen/resolve/main/model.i) | [model.i](https://hf-mirror.com/zisisnotzis/ydkgen/resolve/main/model.i) | [model.i](https://www.modelscope.cn/ziszis/ydkgen/resolve/main/model.i) | Tokenizer data (binary numpy.int32 array format) | | [model0.pth](https://huggingface.co/zisisnotzis/ydkgen/resolve/main/model0.pth) | [model0.pth](https://hf-mirror.com/zisisnotzis/ydkgen/resolve/main/model0.pth) | [model0.pth](https://www.modelscope.cn/ziszis/ydkgen/resolve/main/model0.pth) | Model checkpoint (MODE 0; PyTorch `.pth` format) | | [model1.pth](https://huggingface.co/zisisnotzis/ydkgen/resolve/main/model1.pth) | [model1.pth](https://hf-mirror.com/zisisnotzis/ydkgen/resolve/main/model1.pth) | [model1.pth](https://www.modelscope.cn/ziszis/ydkgen/resolve/main/model1.pth) | Model checkpoint (MODE 1; PyTorch `.pth` format) | | [model2.pth](https://huggingface.co/zisisnotzis/ydkgen/resolve/main/model2.pth) | [model2.pth](https://hf-mirror.com/zisisnotzis/ydkgen/resolve/main/model2.pth) | [model2.pth](https://www.modelscope.cn/ziszis/ydkgen/resolve/main/model2.pth) | Model checkpoint (MODE 2; PyTorch `.pth` format) – *poor performance (baseline)* | ## Dataset The model is trained on a curated small dataset of valid Yu-Gi-Oh! `.ydk` deck lists, available via the same mirrors as the pretrained checkpoints: | Name | Mirror 1 (Hugging Face Datasets) | Mirror 2 (HF Mirror Datasets) | Mirror 3 (ModelScope Datasets) | Description | |-------|----------------------------------|-------------------------------|--------------------------------|---------------------------| | [ydk](https://huggingface.co/datasets/zisisnotzis/ydk) | [ydk](https://hf-mirror.com/datasets/zisisnotzis/ydk) | [ydk](https://www.modelscope.cn/datasets/ziszis/ydk) | Curated Yu-Gi-Oh! YDK dataset | ## Inference & Evaluation Run the script to generate Yu-Gi-Oh! decks with the pretrained model (or custom checkpoints) and evaluate generation quality. Generated decks are saved as `.ydk` files for direct use in Yu-Gi-Oh! simulators. ### Run Inference/Evaluation ```bash python eval.py ``` ### Custom Inference Configuration Tune inference parameters in `eval.py` for custom deck generation—**ensure the `MODE` matches the checkpoint’s training loss mode** for valid results: ```python FILE = 'model0' # Loads checkpoint from {FILE}.pth; saves generated decks to deck/{FILE}.ydk DEV = 'cpu' # CPU is sufficient for inference (CUDA not required) MODE = 0 # Training loss mode of the checkpoint (MANDATORY MATCH) NUM = 55 # Total cards in generated deck (follows Yu-Gi-Oh! official format rules) BEAM = 256 # Beam search width (higher = better generation, slower inference; reduce for resource-constrained systems) ```