historical-english-books-similarity

# Dataset card for Historical English Books Similarity The dataset is a random sample of [Historical English Books](https://huggingface.co/datasets/NeuML/historical-english-books) title-abstract pairs and query-sentences pairs. The data is primarily from the [Victorian Era](https://en.wikipedia.org/wiki/Victorian_era) up until 1899. The title and abstracts are used as is from the upstream datasource. Each book is split by sentence and a random sample of sentences are selected per split. Logic was used to validate there is no overlap between the splits. Queries are LLM-generated using the prose of the 1800s. It also has a [BEIR](https://github.com/beir-cellar/beir)-compatible version of the `test` split that can be used to measure the accuracy of vector models trained with this data.