T-REx Bite 1.0

T-REx Bite adapts T-REx to the next-token-prediction paradigm by ensuring that, in each text snippet, the subject appears before the object. This alignment mimics real-world scenarios in which a model sees partial information (the subject and some context) and must then predict or "retrieve" the missing object. To keep snippets manageable within the limited context windows of smaller language models (such as GPT-2), each snippet is capped at 512 characters. We further require that the snippet explicitly mentions both subject and object, does not start a new sentence at the object, and is linked to the corresponding subgraph in T-REx Star. By applying these constraints, we obtain about 6.4 million short "bites" for training, 0.92 million for testing, and 0.75 million for validation. Each bite is a compact piece of Wikipedia text that retains the original clarity and diversity of T-REx but is tailored to ensure a direct subject--object alignment. This structure lets researchers readily evaluate how well an LLM completes the object token(s) given the preceding subject. The dataset naturally accommodates multi-token objects under modern sub-word tokenization, removing the single-token assumptions of LAMA-like methods.T-REx Bite 通过约束每个文本片段内主语始终位于宾语之前，将T-REx适配至下一词预测范式（next-token-prediction paradigm）。这种对齐方式贴合真实应用场景：模型仅能获取部分信息（主语与部分上下文），随后需要预测或“检索”缺失的宾语。为使片段适配小型语言模型（如GPT-2）有限的上下文窗口（context windows），每个片段的长度被限制为不超过512个字符。此外，我们还要求片段需明确提及主语与宾语二者，不会以宾语作为新句子的开头，且与T-REx Star中的对应子图（subgraph）建立关联。 通过施加上述约束，我们共获取约640万条简短"bites"样本用于训练、92万条用于测试，以及75万条用于验证。每条"bites"样本均为一段精简的维基百科（Wikipedia）文本，保留了T-REx原始的清晰度与多样性，同时经过定制化调整以确保主语与宾语的直接对齐。这种结构使得研究人员可以便捷地评估：在给定前文主语的情况下，大语言模型（Large Language Model，LLM）能否准确补全宾语Token（Token）。该数据集可自然适配现代子词标记化（sub-word tokenization）场景下的多Token宾语，摆脱了类似LAMA方法中仅支持单Token宾语的假设限制。