daidalos-project/Herodotos_dataset

---
license: agpl-3.0
---
Datasheet: Herodotos Project Dataset

For what purpose was the dataset created? Was there a specific task in
mind? Was there a specific gap that needed to be filled? Please provide
a description.

- created for Herodotos Project to train NER-Tagger (BiLSTM CRF; see:
Alexander Erdmann, David Joseph Wrisley, Benjamin Allen, Christopher
Brown, Sophie Cohen Bodénès, Micha Elsner, Yukun Feng, Brian Joseph,
Béatrice Joyeaux-Prunel and Marie-Catherine de Marneffe. 2019.
"[Practical, Efficient, and Customizable Active Learning for Named
Entity Recognition in the Digital
Humanities](https://github.com/alexerdmann/HER/blob/master/HER_NAACL2019_preprint.pdf)."
In Proceedings of North American Association of Computational
Linguistics (NAACL 2019). Minneapolis, Minnesota.)

- Goal of Herodotos Project: catalogue and compendium of ancient ethnic
groups

- For more info on the corpus see:
https://aclanthology.org/W16-4012.pdf

Who created the dataset (e.g., which team, research group) and on behalf
of which entity (e.g., company, institution, organization)?

- from the documentation: „The data files in the **Annotation** directory
were annotated for named entities by a team of Classics experts at Ohio
State University. Texts presently included are excerpts from Caesar's
Wars, both Gallic (GW) and Civil (CW), the Plinies' writings, both
Elder and Younger, and Ovid's Ars Amatoria. "

Who funded the creation of the dataset? If there is an associated grant,
please provide the name of the grantor and the grant name and number.

- unknown

Any other comments?

- No

What do the instances that comprise the dataset represent (e.g.,
documents, photos, people, countries)? Are there multiple types of
instances (e.g., movies, users, and ratings; people and interactions
between them; nodes and edges)? Please provide a description.

- Latin texts "Texts presently included are excerpts from Caesar's Wars, both Gallic (GW) and Civil (CW), the Plinies' writings, both Elder and Younger, and Ovid's Ars Amatoria."

How many instances are there in total (of each type, if appropriate)?

- 146,066 words

Does the dataset contain all po-ssible instances or is it a sample (not
necessarily random) of instances from a larger set? If the dataset is a
sample, then what is the larger set? Is the sample representative of the
larger set (e.g., geographic coverage)? If so, please describe how this representativeness was validated/verified. If it is not
representative of the larger set, please describe why not (e.g., to
cover a more diverse range of instances, because instances were withheld
or unavailable).

- sample of Latin literature (see previous answers), representative of
Classical Latin literature, might not be representative of the entire Latin
literature (time, geography)

What data does each instance consist of? "Raw" data (e.g., unprocessed
text or images) or features? In either case, please provide a
description.

- Each instance consists of raw text data

Is there a label or target associated with each instance? If so, please
provide a description.

- NER Labels: PRS-B, PRS-I, GEO-B, GEO-I, GRP-B, GRP-I or 0
- labels follow the BIO scheme
- see also: https://aclanthology.org/W16-4012.pdf

Is any information missing from individual instances? If so, please
provide a description, explaining why this information is missing (e.g.,
because it was unavailable). This does not include intentionally removed
information, but might include, e.g., redacted text.

- No

Are relationships between individual instances made explicit (e.g.,
users' movie ratings, social network links)? If so, please describe how
these relationships are made explicit.

- Relationships are made explicit according to the BIO scheme

Are there recommended data splits (e.g., training,
development/validation, testing)? If so, please provide a description of
these splits, explaining the rationale behind them.

- Text from Gallic War is split into test and train sets

Are there any errors, sources of noise, or redundancies in the dataset?
If so, please provide a description.

- Naturally occurring repetitions of names in the texts

Is the dataset self-contained, or does it link to or otherwise rely on
external resources (e.g., websites, tweets, other datasets)? If it links
to or relies on external resources, a) are there guarantees that they
will exist, and remain constant, over time; b) are there official
archival versions of the complete dataset (i.e., including the external
resources as they existed at the time the dataset was created); c) are
there any restrictions (e.g., licenses, fees) associated with any of the
external resources that might apply to a dataset consumer? Please
provide descriptions of all external resources and any restrictions
associated with them, as well as links or other access points, as appropriate.

- The dataset is self-contained and can be downloaded from GitHub
(https://github.com/Herodotos-Project/Herodotos-Project-Latin-NER-Tagger-Annotation/blob/master/README.md)

Does the dataset contain data that might be considered confidential
(e.g., data that is protected by legal privilege or by doctor--patient
confidentiality, data that includes the content of individuals'
non-public communications)? If so, please provide a description.

- No

Does the dataset contain data that, if viewed directly, might be
offensive, insulting, threatening, or might otherwise cause anxiety? If
so, please describe why. If the dataset does not relate to people, you
may skip the remaining questions in this section.

- The dataset contains descriptions of war.

Does the dataset identify any subpopulations (e.g., by age, gender)? If
so, please describe how these subpopulations are identified and provide
a description of their respective distributions within the dataset.

- A number of ethnic groups from antiquity are referred to.

Is it possible to identify individuals (i.e., one or more natural
persons), either directly or indirectly (i.e., in combination with other
data) from the dataset? If so, please describe how.

- Only historical individuals

Does the dataset contain data that might be considered sensitive in any
way (e.g., data that reveals race or ethnic origins, sexual
orientations, religious beliefs, political opinions or union
memberships, or locations; financial or health data; biometric or
genetic data; forms of government identification, such as social
security numbers; criminal history)? If so, please provide a
description.

- Only historical individuals

Any other comments?

- No

How was the data associated with each instance acquired? Was the data
directly observable (e.g., raw text, movie ratings), reported by
subjects (e.g., survey responses), or indirectly inferred/derived from
other data (e.g., part-of-speech tags, model-based guesses for age or
language)?

- The data consists of publicly available texts

If the data was reported by subjects or indirectly inferred/derived from
other data, was the data validated/verified? If so, please describe how.

- unknown

What mechanisms or procedures were used to collect the data (e.g.,
hardware apparatuses or sensors, manual human curation, software
programs, software APIs)? How were these mechanisms or procedures
validated?

- from the documentation: „All texts are in Latin taken from the [Latin
Library Collection](https://www.theLatinlibrary.com/) (collected by
[CLTK](https://github.com/cltk/Latin_text_Latin_library)) or the
[Perseus Latin
Collection](http://www.perseus.tufts.edu/hopper/collection?collection=Perseus:collection:Greco-Roman).
"

If the dataset is a sample from a larger set, what was the sampling
strategy (e.g., deterministic, probabilistic with specific sampling
probabilities)?

- unknown

Who was involved in the data collection process (e.g., students,
crowdworkers, contractors) and how were they compensated (e.g., how much
were crowdworkers paid)?

- <https://aclanthology.org/W16-4012.pdf> S. 87: "an undergraduate, a
graduate, and a professor of Classics, each with at least 4 years of
experience studying Latin"

Over what timeframe was the data collected? Does this timeframe match
the creation timeframe of the data associated with the instances (e.g.,
recent crawl of old news articles)? If not, please describe the
timeframe in which the data associated with the instances was created.
Were any ethical review processes conducted (e.g., by an institutional
review board)? If so, please provide a description of these review
processes, including the outcomes, as well as a link or other access
point to any supporting documentation. If the dataset does not relate to
people, you may skip the remaining questions in this section.

- unknown

Did you collect the data from the individuals in question directly, or
obtain it via third parties or other sources (e.g., websites)? Were the
individuals in question notified about the data collection? If so,
please describe (or show with screenshots or other information) how
notice was provided, and provide a link or other access point to, or
otherwise reproduce, the exact language of the notification itself.

- not applicable

Did the individuals in question consent to the collection and use of
their data? If so, please describe (or show with screenshots or other
information) how consent was requested and provided, and provide a link
or other access point to, or otherwise reproduce, the exact language to
which the individuals consented.

- not applicable

If consent was obtained, were the consenting individuals provided with a
mechanism to revoke their consent in the future or for certain uses? If
so, please provide a description, as well as a link or other access
point to the mechanism (if appropriate).

- not applicable

Has an analysis of the potential impact of the dataset and its use on
data subjects (e.g., a data protection impact analysis) been conducted?
If so, please provide a description of this analysis, including the
outcomes, as well as a link or other access point to any supporting
documentation.

- not applicable

Any other comments?

- No

Preprocessing/cleaning/labeling

Was any preprocessing/cleaning/labeling of the data done (e.g.,
discretization or bucketing, tokenization, part-of-speech tagging, SIFT
feature extraction, removal of instances, processing of missing values)?
If so, please provide a description. If not, you may skip the remainder
of the questions in this section.

- The data was manually annotated for NEs.

Was the "raw" data saved in addition to the preprocessed/cleaned/labeled
data (e.g., to support unanticipated future uses)? If so, please provide a link or other
access point to the "raw" data.

- The data can be downloaded from:
https://github.com/clmarr/Herodotos-beta/tree/f22fdd92b3318cfb8fc93b004b0947aea14ce9c2/Annotation_1-1-19

Any other comments?

-No

Uses

Has the dataset been used for any tasks already? If so, please provide a
description.

- It has been used to train an NER-Tagger for Latin. See:
<https://aclanthology.org/W16-4012.pdf> and
https://github.com/alexerdmann/HER/blob/master/HER_NAACL2019_preprint.pdf

Is there a repository that links to any or all papers or systems that
use the dataset? If so, please provide a link or other access point.
What (other) tasks could the dataset be used for?

- See:
https://github.com/alexerdmann/HER/blob/master/HER_NAACL2019_preprint.pdf

Is there anything about the composition of the dataset or the way it was
collected and preprocessed/cleaned/labeled that might impact future
uses? For example, is there anything that a dataset consumer might need
to know to avoid uses that could result in unfair treatment of
individuals or groups (e.g., stereotyping, quality of service issues) or
other risks or harms (e.g., legal risks, financial harms)? If so, please
provide a description. Is there anything a dataset consumer could do to
mitigate these risks or harms?

- Strong class imbalance (most tokens are non-entities)

Are there tasks for which the dataset should not be used? If so, please
provide a description.

- No

Any other comments?

- No

Distribution

Will the dataset be distributed to third parties outside of the entity
(e.g., company, institution, organization) on behalf of which the
dataset was created? If so, please provide a description. How will the dataset will be distributed (e.g., tarball on website, API,
GitHub)?

- The data can be downloaded from:
https://github.com/clmarr/Herodotos-beta/tree/f22fdd92b3318cfb8fc93b004b0947aea14ce9c2/Annotation_1-1-19

Does the dataset have a digital object identifier (DOI)?

- No

When will the dataset be distributed?

- The data can be downloaded from:
- <https://github.com/clmarr/Herodotos-beta/tree/f22fdd92b3318cfb8fc93b004b0947aea14ce9c2/Annotation_1-1-19>
- <https://github.com/Herodotos-Project/Herodotos-Project-Latin-NER-Tagger-Annotation>

Will the dataset be distributed under a copyright or other intellectual
property (IP) license, and/or under applicable terms of use (ToU)? If
so, please describe this license and/or ToU, and provide a link or other
access point to, or otherwise reproduce, any relevant licensing terms or
ToU, as well as any fees associated with these restrictions.

- [AGPL-3.0 license](https://github.com/Herodotos-Project/Herodotos-Project-Latin-NER-Tagger-Annotation/blob/master/LICENSE)

Have any third parties imposed IP-based or other restrictions on the
data associated with the instances? If so, please describe these
restrictions, and provide a link or other access point to, or otherwise
reproduce, any relevant licensing terms, as well as any fees associated
with these restrictions.

- unknown

Do any export controls or other regulatory restrictions apply to the
dataset or to individual instances? If so, please describe these
restrictions, and provide a link or other access point to, or otherwise
reproduce, any supporting documentation.

- unknown

Any other comments?

- No

Maintenance

Who will be supporting/hosting/maintaining the dataset?

- from the documentation: "Contact [[e-mail]](mailto:[e-mail]) or any of the co-authors with questions regarding this repository."

How can the owner/curator/manager of the dataset be contacted (e.g.,
email address)?

- [[e-mail]](mailto:[e-mail])

Is there an erratum? If so, please provide a link or other access point.
Will the dataset be updated (e.g., to correct labeling errors, add new
instances, delete instances)? If so, please describe how often, by whom,
and how updates will be communicated to dataset consumers (e.g., mailing
list, GitHub)?

- new instances for the Ancient Greek language will be added in the
future

If the dataset relates to people, are there applicable limits on the
retention of the data associated with the instances (e.g., were the
individuals in question told that their data would be retained for a
fixed period of time and then deleted)? If so, please describe these
limits and explain how they will be enforced.

- not applicable

Will older versions of the dataset continue to be
supported/hosted/maintained? If so, please describe how. If not, please
describe how its obsolescence will be communicated to dataset consumers.

- unknown

If others want to extend/augment/build on/contribute to the dataset, is
there a mechanism for them to do so? If so, please provide a
description. Will these contributions be validated/verified? If so, please describe how. If not, why not? Is there a process for
communicating/distributing these contributions to dataset consumers? If
so, please provide a description.

- unknown

Any other comments?

- No