PyKEEN Benchmarking Experiment Model Files

Model Weights This repository provides weights of the models from the benchmarking study conducted in "Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework" which have been upgraded to compatible with PyKEEN 1.9. The weights are organized as zipfiles, which are named by the dataset-interaction function configuration. For each of these combinations, we chose the best according to validation Hits@10 to include into this repository. For each model, we have three files: <code>configuration.json</code> contains the (pipeline) configuration used to train the model. It can loaded as <pre class="sourceCode python"><code class="sourceCode python">import pathlib import json configuration = json.loads(pathlib.Path("configuration.json").read_text())</code></pre> Since the configuration is intended for the <em>pipeline</em>, we need some custom code to re-create the model without re-training it. <pre class="sourceCode python"><code class="sourceCode python">from pykeen.datasets import get_dataset from pykeen.models import ERModel, model_resolver configuration = configuration["pipeline"] # load the triples factory dataset = get_dataset( dataset=configuration["dataset"], dataset_kwargs=configuration.get("dataset_kwargs", None) ) model: ERModel = model_resolver.make( configuration["model"], configuration["model_kwargs"], triples_factory=dataset.training )</code></pre> Note, that this only creates the model instance, but does <em>not</em> load the weights, yet. <code>state_dict.pt</code> contains the weights, stored via <code>torch.save</code>. They can be loaded via <pre class="sourceCode python"><code class="sourceCode python">import torch state_dict = torch.load("state_dict.pt")</code></pre> We can load these weights into the model by using <code>Module.load_state_dict</code> <pre class="sourceCode python"><code class="sourceCode python">model.load_state_dict(state_dict, strict=False)</code></pre> Note that we set <code>strict=False</code>, since the exported weights do <em>not</em> contain regularizers' state, while the re-instantiated models may have regularizers. <code>results.json</code> contains the results obtained by the original runs. It can be read by <pre class="sourceCode python"><code class="sourceCode python">import pathlib import json configuration = json.loads(pathlib.Path("results.json").read_text())</code></pre> Note that some of the recently added metrics are not available in those results.