swedish-dynaword
收藏资源简介：
# 🧨 Swedish Dynaword <!-- START README TABLE --> | | | | ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Version** | 0.0.18 ([Changelog](/CHANGELOG.md)) | | **Language** | Swedish (sv, swe) | | **License** | Openly Licensed, See the respective dataset | | **Models** | Currently there is no models trained on this dataset | | **Contact** | If you have question about this project please create an issue [here](https://huggingface.co/datasets/danish-foundation-models/swedish-dynaword/discussions) | <!-- END README TABLE --> ## Table of Contents - [🧨 Swedish Dynaword](#-swedish-dynaword) - [Table of Contents](#table-of-contents) - [Dataset Description](#dataset-description) - [Dataset Summary](#dataset-summary) - [Loading the dataset](#loading-the-dataset) - [Languages](#languages) - [Domains](#domains) - [Licensing](#licensing) - [Dataset Structure](#dataset-structure) - [Data Instances](#data-instances) - [Data Fields](#data-fields) - [Data Splits](#data-splits) - [Dataset Creation](#dataset-creation) - [Curation Rationale](#curation-rationale) - [Annotations](#annotations) - [Source Data](#source-data) - [Data Collection and Processing](#data-collection-and-processing) - [Dataset Statistics](#dataset-statistics) - [Contributing to the dataset](#contributing-to-the-dataset) - [Citation Information](#citation-information) - [License information](#license-information) - [Personal and Sensitive Information](#personal-and-sensitive-information) - [Bias, Risks, and Limitations](#bias-risks-and-limitations) - [Notice and takedown policy](#notice-and-takedown-policy) ## Dataset Description <!-- START-DESC-STATS --> - **Number of samples**: 545.45M - **Number of tokens (Llama 3)**: 24.49B - **Average document length in tokens (min, max)**: 44.89 (2, 8.14M) <!-- END-DESC-STATS --> ### Dataset Summary The Swedish dynaword is a collection of Swedish free-form text datasets from various domains. All of the datasets in the Swedish Dynaword are openly licensed and deemed permissible for training large language models. Swedish dynaword is continually developed, which means that the dataset will actively be updated as new datasets become available. If you would like to contribute a dataset see the [contribute section](#contributing-to-the-dataset). ### Loading the dataset ```py from datasets import load_dataset name = "danish-foundation-models/swedish-dynaword" ds = load_dataset(name, split = "train") sample = ds[1] # see "Data Instances" below ``` or load it by streaming the data ```py ds = load_dataset(name, split = "train", streaming=True) dataset_iter = iter(ds) sample = next(iter(dataset_iter)) ``` You can also load a single subset at a time: ```py ds = load_dataset(name, "lag1800", split = "train") ``` As Swedish dynaword is continually expanding and curated you can make sure that you get the same dataset every time by specifying the revision: You can also load a single subset at a time: ```py ds = load_dataset(name, revision="{desired revision}") ``` ### Languages This dataset includes the following languages: - Swedish (swe-Latn) In addition it likely contains small amounts of English due to code-switching and Scandinavian languages due to language misclassificaitons due to their similarity. Language is denoted using [BCP-47](https://en.wikipedia.org/wiki/IETF_language_tag), using the langauge code ISO [639-3](https://en.wikipedia.org/wiki/List_of_ISO_639_language_codes) and the script code [ISO 15924](https://en.wikipedia.org/wiki/ISO_15924). <!-- START-LANGUAGE TABLE --> | Language | Sources | N. Tokens | |:-----------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------| | sv | [dalpilen-1860], [lag1800], [svensk-tidskrift], [statens-offentliga-utredningar], [biblioteksbladet], [riksdagen-forfattningssamling], [riksdagen-reglementen], [riksdagen-register], [riksdagen-skrivelser], [riksdagen-utredningar], [riksdagen-berattelser], [riksdagen-motioner], [riksdagen-betankanden], [riksdagen-propositioner], [riksdagen-protokoll], [cellar], [flashback-dator], [flashback-droger], [flashback-ekonomi], [flashback-fordon], [flashback-hem], [flashback-kultur], [flashback-livsstil], [flashback-mat], [flashback-om-flashback], [flashback-ovrigt], [flashback-politik], [flashback-resor], [flashback-samhalle], [flashback-sex], [flashback-sport], [flashback-vetenskap], [familjeliv-adoption], [familjeliv-allmanna-ekonomi], [familjeliv-allmanna-familjeliv], [familjeliv-allmanna-fritid], [familjeliv-allmanna-husdjur], [familjeliv-allmanna-hushem], [familjeliv-allmanna-kropp], [familjeliv-allmanna-noje], [familjeliv-allmanna-samhalle], [familjeliv-allmanna-sandladan], [familjeliv-anglarum], [familjeliv-expert], [familjeliv-foralder], [familjeliv-gravid], [familjeliv-kansliga], [familjeliv-medlem-allmanna], [familjeliv-medlem-foraldrar], [familjeliv-medlem-planerarbarn], [familjeliv-medlem-vantarbarn], [familjeliv-pappagrupp], [familjeliv-planerarbarn], [familjeliv-sexsamlevnad], [familjeliv-svartattfabarn], [lb-open], [poeter], [wikipedia-sv], [europarl-sv], [laakartidningen], [fsv-aldrelagar], [fsv-aldrereligiosprosa], [fsv-nysvenskbibel], [fsv-nysvenskdalin], [fsv-nysvenskkronikor], [fsv-nysvensklagar], [fsv-nysvenskovrigt], [fsv-profanprosa], [fsv-verser], [fsv-yngrelagar], [fsv-yngrereligiosprosa], [fsv-yngretankebocker], [standsriksdagen-adelsstandet], [standsriksdagen-bihang], [standsriksdagen-bondestandet], [standsriksdagen-borgarstandet], [standsriksdagen-prastestandet], [standsriksdagen-riksdagsakter], [standsriksdagen-riksdagsbeslut], [strindbergromaner], [strindbergbrev], [dramadialog], [bibel1917], [psalmboken], [akademiliv], [dagens-arena], [gu-journalen], [forskning-framsteg], [sv-covid-19] | 24.49B | | **Total** | | 24.49B | [dalpilen-1860]: data/dalpilen-1860/dalpilen-1860.md [lag1800]: data/lag1800/lag1800.md [svensk-tidskrift]: data/svensk-tidskrift/svensk-tidskrift.md [statens-offentliga-utredningar]: data/statens-offentliga-utredningar/statens-offentliga-utredningar.md [biblioteksbladet]: data/biblioteksbladet/biblioteksbladet.md [riksdagen-forfattningssamling]: data/riksdagen-forfattningssamling/riksdagen-forfattningssamling.md [riksdagen-reglementen]: data/riksdagen-reglementen/riksdagen-reglementen.md [riksdagen-register]: data/riksdagen-register/riksdagen-register.md [riksdagen-skrivelser]: data/riksdagen-skrivelser/riksdagen-skrivelser.md [riksdagen-utredningar]: data/riksdagen-utredningar/riksdagen-utredningar.md [riksdagen-berattelser]: data/riksdagen-berattelser/riksdagen-berattelser.md [riksdagen-motioner]: data/riksdagen-motioner/riksdagen-motioner.md [riksdagen-betankanden]: data/riksdagen-betankanden/riksdagen-betankanden.md [riksdagen-propositioner]: data/riksdagen-propositioner/riksdagen-propositioner.md [riksdagen-protokoll]: data/riksdagen-protokoll/riksdagen-protokoll.md [cellar]: data/cellar/cellar.md [flashback-dator]: data/flashback-dator/flashback-dator.md [flashback-droger]: data/flashback-droger/flashback-droger.md [flashback-ekonomi]: data/flashback-ekonomi/flashback-ekonomi.md [flashback-fordon]: data/flashback-fordon/flashback-fordon.md [flashback-hem]: data/flashback-hem/flashback-hem.md [flashback-kultur]: data/flashback-kultur/flashback-kultur.md [flashback-livsstil]: data/flashback-livsstil/flashback-livsstil.md [flashback-mat]: data/flashback-mat/flashback-mat.md [flashback-om-flashback]: data/flashback-om-flashback/flashback-om-flashback.md [flashback-ovrigt]: data/flashback-ovrigt/flashback-ovrigt.md [flashback-politik]: data/flashback-politik/flashback-politik.md [flashback-resor]: data/flashback-resor/flashback-resor.md [flashback-samhalle]: data/flashback-samhalle/flashback-samhalle.md [flashback-sex]: data/flashback-sex/flashback-sex.md [flashback-sport]: data/flashback-sport/flashback-sport.md [flashback-vetenskap]: data/flashback-vetenskap/flashback-vetenskap.md [familjeliv-adoption]: data/familjeliv-adoption/familjeliv-adoption.md [familjeliv-allmanna-ekonomi]: data/familjeliv-allmanna-ekonomi/familjeliv-allmanna-ekonomi.md [familjeliv-allmanna-familjeliv]: data/familjeliv-allmanna-familjeliv/familjeliv-allmanna-familjeliv.md [familjeliv-allmanna-fritid]: data/familjeliv-allmanna-fritid/familjeliv-allmanna-fritid.md [familjeliv-allmanna-husdjur]: data/familjeliv-allmanna-husdjur/familjeliv-allmanna-husdjur.md [familjeliv-allmanna-hushem]: data/familjeliv-allmanna-hushem/familjeliv-allmanna-hushem.md [familjeliv-allmanna-kropp]: data/familjeliv-allmanna-kropp/familjeliv-allmanna-kropp.md [familjeliv-allmanna-noje]: data/familjeliv-allmanna-noje/familjeliv-allmanna-noje.md [familjeliv-allmanna-samhalle]: data/familjeliv-allmanna-samhalle/familjeliv-allmanna-samhalle.md [familjeliv-allmanna-sandladan]: data/familjeliv-allmanna-sandladan/familjeliv-allmanna-sandladan.md [familjeliv-anglarum]: data/familjeliv-anglarum/familjeliv-anglarum.md [familjeliv-expert]: data/familjeliv-expert/familjeliv-expert.md [familjeliv-foralder]: data/familjeliv-foralder/familjeliv-foralder.md [familjeliv-gravid]: data/familjeliv-gravid/familjeliv-gravid.md [familjeliv-kansliga]: data/familjeliv-kansliga/familjeliv-kansliga.md [familjeliv-medlem-allmanna]: data/familjeliv-medlem-allmanna/familjeliv-medlem-allmanna.md [familjeliv-medlem-foraldrar]: data/familjeliv-medlem-foraldrar/familjeliv-medlem-foraldrar.md [familjeliv-medlem-planerarbarn]: data/familjeliv-medlem-planerarbarn/familjeliv-medlem-planerarbarn.md [familjeliv-medlem-vantarbarn]: data/familjeliv-medlem-vantarbarn/familjeliv-medlem-vantarbarn.md [familjeliv-pappagrupp]: data/familjeliv-pappagrupp/familjeliv-pappagrupp.md [familjeliv-planerarbarn]: data/familjeliv-planerarbarn/familjeliv-planerarbarn.md [familjeliv-sexsamlevnad]: data/familjeliv-sexsamlevnad/familjeliv-sexsamlevnad.md [familjeliv-svartattfabarn]: data/familjeliv-svartattfabarn/familjeliv-svartattfabarn.md [lb-open]: data/lb-open/lb-open.md [poeter]: data/poeter/poeter.md [wikipedia-sv]: data/wikipedia-sv/wikipedia-sv.md [europarl-sv]: data/europarl-sv/europarl-sv.md [laakartidningen]: data/laakartidningen/laakartidningen.md [fsv-aldrelagar]: data/fsv-aldrelagar/fsv-aldrelagar.md [fsv-aldrereligiosprosa]: data/fsv-aldrereligiosprosa/fsv-aldrereligiosprosa.md [fsv-nysvenskbibel]: data/fsv-nysvenskbibel/fsv-nysvenskbibel.md [fsv-nysvenskdalin]: data/fsv-nysvenskdalin/fsv-nysvenskdalin.md [fsv-nysvenskkronikor]: data/fsv-nysvenskkronikor/fsv-nysvenskkronikor.md [fsv-nysvensklagar]: data/fsv-nysvensklagar/fsv-nysvensklagar.md [fsv-nysvenskovrigt]: data/fsv-nysvenskovrigt/fsv-nysvenskovrigt.md [fsv-profanprosa]: data/fsv-profanprosa/fsv-profanprosa.md [fsv-verser]: data/fsv-verser/fsv-verser.md [fsv-yngrelagar]: data/fsv-yngrelagar/fsv-yngrelagar.md [fsv-yngrereligiosprosa]: data/fsv-yngrereligiosprosa/fsv-yngrereligiosprosa.md [fsv-yngretankebocker]: data/fsv-yngretankebocker/fsv-yngretankebocker.md [standsriksdagen-adelsstandet]: data/standsriksdagen-adelsstandet/standsriksdagen-adelsstandet.md [standsriksdagen-bihang]: data/standsriksdagen-bihang/standsriksdagen-bihang.md [standsriksdagen-bondestandet]: data/standsriksdagen-bondestandet/standsriksdagen-bondestandet.md [standsriksdagen-borgarstandet]: data/standsriksdagen-borgarstandet/standsriksdagen-borgarstandet.md [standsriksdagen-prastestandet]: data/standsriksdagen-prastestandet/standsriksdagen-prastestandet.md [standsriksdagen-riksdagsakter]: data/standsriksdagen-riksdagsakter/standsriksdagen-riksdagsakter.md [standsriksdagen-riksdagsbeslut]: data/standsriksdagen-riksdagsbeslut/standsriksdagen-riksdagsbeslut.md [strindbergromaner]: data/strindbergromaner/strindbergromaner.md [strindbergbrev]: data/strindbergbrev/strindbergbrev.md [dramadialog]: data/dramadialog/dramadialog.md [bibel1917]: data/bibel1917/bibel1917.md [psalmboken]: data/psalmboken/psalmboken.md [akademiliv]: data/akademiliv/akademiliv.md [dagens-arena]: data/dagens-arena/dagens-arena.md [gu-journalen]: data/gu-journalen/gu-journalen.md [forskning-framsteg]: data/forskning-framsteg/forskning-framsteg.md [sv-covid-19]: data/sv-covid-19/sv-covid-19.md <!-- END-LANGUAGE TABLE --> ### Domains This dynaword consist of data from various domains (e.g., legal, books, social media). The following table and figure give an overview of the relative distributions of these domains. To see a full overview of the source check out the [source data section](#source-data) <div style="display: flex; gap: 20px; align-items: flex-start;"> <div style="flex: 1;"> <!-- START-DOMAIN TABLE --> | Domain | Sources | N. Tokens | |:-------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------| | Social Media | [flashback-dator], [flashback-droger], [flashback-ekonomi], [flashback-fordon], [flashback-hem], [flashback-kultur], [flashback-livsstil], [flashback-mat], [flashback-om-flashback], [flashback-ovrigt], [flashback-politik], [flashback-resor], [flashback-samhalle], [flashback-sex], [flashback-sport], [flashback-vetenskap], [familjeliv-adoption], [familjeliv-allmanna-ekonomi], [familjeliv-allmanna-familjeliv], [familjeliv-allmanna-fritid], [familjeliv-allmanna-husdjur], [familjeliv-allmanna-hushem], [familjeliv-allmanna-kropp], [familjeliv-allmanna-noje], [familjeliv-allmanna-samhalle], [familjeliv-allmanna-sandladan], [familjeliv-anglarum], [familjeliv-expert], [familjeliv-foralder], [familjeliv-gravid], [familjeliv-kansliga], [familjeliv-medlem-allmanna], [familjeliv-medlem-foraldrar], [familjeliv-medlem-planerarbarn], [familjeliv-medlem-vantarbarn], [familjeliv-pappagrupp], [familjeliv-planerarbarn], [familjeliv-sexsamlevnad], [familjeliv-svartattfabarn] | 14.72B | | Legal | [lag1800], [riksdagen-forfattningssamling], [riksdagen-reglementen], [cellar], [fsv-aldrelagar], [fsv-nysvensklagar], [fsv-yngrelagar], [standsriksdagen-adelsstandet], [standsriksdagen-bihang], [standsriksdagen-bondestandet], [standsriksdagen-borgarstandet], [standsriksdagen-prastestandet], [standsriksdagen-riksdagsakter], [standsriksdagen-riksdagsbeslut] | 5.21B | | Report | [statens-offentliga-utredningar], [riksdagen-register], [riksdagen-skrivelser], [riksdagen-utredningar], [riksdagen-berattelser], [riksdagen-motioner], [riksdagen-betankanden], [riksdagen-propositioner], [riksdagen-protokoll] | 3.17B | | Books | [lb-open], [poeter], [fsv-aldrereligiosprosa], [fsv-nysvenskbibel], [fsv-nysvenskdalin], [fsv-nysvenskkronikor], [fsv-nysvenskovrigt], [fsv-profanprosa], [fsv-verser], [fsv-yngrereligiosprosa], [fsv-yngretankebocker], [strindbergromaner], [strindbergbrev], [dramadialog], [bibel1917], [psalmboken] | 831.26M | | Encyclopedic | [wikipedia-sv] | 362.29M | | News | [dalpilen-1860], [svensk-tidskrift], [biblioteksbladet], [akademiliv], [dagens-arena], [gu-journalen], [forskning-framsteg], [sv-covid-19] | 87.67M | | Speeches | [europarl-sv] | 63.02M | | Medical | [laakartidningen] | 42.40M | | **Total** | | 24.49B | [dalpilen-1860]: data/dalpilen-1860/dalpilen-1860.md [lag1800]: data/lag1800/lag1800.md [svensk-tidskrift]: data/svensk-tidskrift/svensk-tidskrift.md [statens-offentliga-utredningar]: data/statens-offentliga-utredningar/statens-offentliga-utredningar.md [biblioteksbladet]: data/biblioteksbladet/biblioteksbladet.md [riksdagen-forfattningssamling]: data/riksdagen-forfattningssamling/riksdagen-forfattningssamling.md [riksdagen-reglementen]: data/riksdagen-reglementen/riksdagen-reglementen.md [riksdagen-register]: data/riksdagen-register/riksdagen-register.md [riksdagen-skrivelser]: data/riksdagen-skrivelser/riksdagen-skrivelser.md [riksdagen-utredningar]: data/riksdagen-utredningar/riksdagen-utredningar.md [riksdagen-berattelser]: data/riksdagen-berattelser/riksdagen-berattelser.md [riksdagen-motioner]: data/riksdagen-motioner/riksdagen-motioner.md [riksdagen-betankanden]: data/riksdagen-betankanden/riksdagen-betankanden.md [riksdagen-propositioner]: data/riksdagen-propositioner/riksdagen-propositioner.md [riksdagen-protokoll]: data/riksdagen-protokoll/riksdagen-protokoll.md [cellar]: data/cellar/cellar.md [flashback-dator]: data/flashback-dator/flashback-dator.md [flashback-droger]: data/flashback-droger/flashback-droger.md [flashback-ekonomi]: data/flashback-ekonomi/flashback-ekonomi.md [flashback-fordon]: data/flashback-fordon/flashback-fordon.md [flashback-hem]: data/flashback-hem/flashback-hem.md [flashback-kultur]: data/flashback-kultur/flashback-kultur.md [flashback-livsstil]: data/flashback-livsstil/flashback-livsstil.md [flashback-mat]: data/flashback-mat/flashback-mat.md [flashback-om-flashback]: data/flashback-om-flashback/flashback-om-flashback.md [flashback-ovrigt]: data/flashback-ovrigt/flashback-ovrigt.md [flashback-politik]: data/flashback-politik/flashback-politik.md [flashback-resor]: data/flashback-resor/flashback-resor.md [flashback-samhalle]: data/flashback-samhalle/flashback-samhalle.md [flashback-sex]: data/flashback-sex/flashback-sex.md [flashback-sport]: data/flashback-sport/flashback-sport.md [flashback-vetenskap]: data/flashback-vetenskap/flashback-vetenskap.md [familjeliv-adoption]: data/familjeliv-adoption/familjeliv-adoption.md [familjeliv-allmanna-ekonomi]: data/familjeliv-allmanna-ekonomi/familjeliv-allmanna-ekonomi.md [familjeliv-allmanna-familjeliv]: data/familjeliv-allmanna-familjeliv/familjeliv-allmanna-familjeliv.md [familjeliv-allmanna-fritid]: data/familjeliv-allmanna-fritid/familjeliv-allmanna-fritid.md [familjeliv-allmanna-husdjur]: data/familjeliv-allmanna-husdjur/familjeliv-allmanna-husdjur.md [familjeliv-allmanna-hushem]: data/familjeliv-allmanna-hushem/familjeliv-allmanna-hushem.md [familjeliv-allmanna-kropp]: data/familjeliv-allmanna-kropp/familjeliv-allmanna-kropp.md [familjeliv-allmanna-noje]: data/familjeliv-allmanna-noje/familjeliv-allmanna-noje.md [familjeliv-allmanna-samhalle]: data/familjeliv-allmanna-samhalle/familjeliv-allmanna-samhalle.md [familjeliv-allmanna-sandladan]: data/familjeliv-allmanna-sandladan/familjeliv-allmanna-sandladan.md [familjeliv-anglarum]: data/familjeliv-anglarum/familjeliv-anglarum.md [familjeliv-expert]: data/familjeliv-expert/familjeliv-expert.md [familjeliv-foralder]: data/familjeliv-foralder/familjeliv-foralder.md [familjeliv-gravid]: data/familjeliv-gravid/familjeliv-gravid.md [familjeliv-kansliga]: data/familjeliv-kansliga/familjeliv-kansliga.md [familjeliv-medlem-allmanna]: data/familjeliv-medlem-allmanna/familjeliv-medlem-allmanna.md [familjeliv-medlem-foraldrar]: data/familjeliv-medlem-foraldrar/familjeliv-medlem-foraldrar.md [familjeliv-medlem-planerarbarn]: data/familjeliv-medlem-planerarbarn/familjeliv-medlem-planerarbarn.md [familjeliv-medlem-vantarbarn]: data/familjeliv-medlem-vantarbarn/familjeliv-medlem-vantarbarn.md [familjeliv-pappagrupp]: data/familjeliv-pappagrupp/familjeliv-pappagrupp.md [familjeliv-planerarbarn]: data/familjeliv-planerarbarn/familjeliv-planerarbarn.md [familjeliv-sexsamlevnad]: data/familjeliv-sexsamlevnad/familjeliv-sexsamlevnad.md [familjeliv-svartattfabarn]: data/familjeliv-svartattfabarn/familjeliv-svartattfabarn.md [lb-open]: data/lb-open/lb-open.md [poeter]: data/poeter/poeter.md [wikipedia-sv]: data/wikipedia-sv/wikipedia-sv.md [europarl-sv]: data/europarl-sv/europarl-sv.md [laakartidningen]: data/laakartidningen/laakartidningen.md [fsv-aldrelagar]: data/fsv-aldrelagar/fsv-aldrelagar.md [fsv-aldrereligiosprosa]: data/fsv-aldrereligiosprosa/fsv-aldrereligiosprosa.md [fsv-nysvenskbibel]: data/fsv-nysvenskbibel/fsv-nysvenskbibel.md [fsv-nysvenskdalin]: data/fsv-nysvenskdalin/fsv-nysvenskdalin.md [fsv-nysvenskkronikor]: data/fsv-nysvenskkronikor/fsv-nysvenskkronikor.md [fsv-nysvensklagar]: data/fsv-nysvensklagar/fsv-nysvensklagar.md [fsv-nysvenskovrigt]: data/fsv-nysvenskovrigt/fsv-nysvenskovrigt.md [fsv-profanprosa]: data/fsv-profanprosa/fsv-profanprosa.md [fsv-verser]: data/fsv-verser/fsv-verser.md [fsv-yngrelagar]: data/fsv-yngrelagar/fsv-yngrelagar.md [fsv-yngrereligiosprosa]: data/fsv-yngrereligiosprosa/fsv-yngrereligiosprosa.md [fsv-yngretankebocker]: data/fsv-yngretankebocker/fsv-yngretankebocker.md [standsriksdagen-adelsstandet]: data/standsriksdagen-adelsstandet/standsriksdagen-adelsstandet.md [standsriksdagen-bihang]: data/standsriksdagen-bihang/standsriksdagen-bihang.md [standsriksdagen-bondestandet]: data/standsriksdagen-bondestandet/standsriksdagen-bondestandet.md [standsriksdagen-borgarstandet]: data/standsriksdagen-borgarstandet/standsriksdagen-borgarstandet.md [standsriksdagen-prastestandet]: data/standsriksdagen-prastestandet/standsriksdagen-prastestandet.md [standsriksdagen-riksdagsakter]: data/standsriksdagen-riksdagsakter/standsriksdagen-riksdagsakter.md [standsriksdagen-riksdagsbeslut]: data/standsriksdagen-riksdagsbeslut/standsriksdagen-riksdagsbeslut.md [strindbergromaner]: data/strindbergromaner/strindbergromaner.md [strindbergbrev]: data/strindbergbrev/strindbergbrev.md [dramadialog]: data/dramadialog/dramadialog.md [bibel1917]: data/bibel1917/bibel1917.md [psalmboken]: data/psalmboken/psalmboken.md [akademiliv]: data/akademiliv/akademiliv.md [dagens-arena]: data/dagens-arena/dagens-arena.md [gu-journalen]: data/gu-journalen/gu-journalen.md [forskning-framsteg]: data/forskning-framsteg/forskning-framsteg.md [sv-covid-19]: data/sv-covid-19/sv-covid-19.md <!-- END-DOMAIN TABLE --> </div> <div style="flex: 1;"> <p align="center"> <img src="./images/domain_distribution.png" width="400" style="margin-right: 10px;" /> </p> </div> </div> ### Licensing The following gives an overview of the licensing in the Dynaword. To get the exact license of the individual datasets check out the [overview table](#source-data). These license is applied to the constituent data, i.e., the text. The collection of datasets (metadata, quality control, etc.) is licensed under [CC-0](https://creativecommons.org/publicdomain/zero/1.0/legalcode.en). <!-- START-LICENSE TABLE --> | License | Sources | N. Tokens | |:-------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------| | CC-BY 4.0 | [dalpilen-1860], [lag1800], [svensk-tidskrift], [statens-offentliga-utredningar], [biblioteksbladet], [riksdagen-forfattningssamling], [riksdagen-reglementen], [riksdagen-register], [riksdagen-skrivelser], [riksdagen-utredningar], [riksdagen-berattelser], [riksdagen-motioner], [riksdagen-betankanden], [riksdagen-propositioner], [riksdagen-protokoll], [flashback-dator], [flashback-droger], [flashback-ekonomi], [flashback-fordon], [flashback-hem], [flashback-kultur], [flashback-livsstil], [flashback-mat], [flashback-om-flashback], [flashback-ovrigt], [flashback-politik], [flashback-resor], [flashback-samhalle], [flashback-sex], [flashback-sport], [flashback-vetenskap], [familjeliv-adoption], [familjeliv-allmanna-ekonomi], [familjeliv-allmanna-familjeliv], [familjeliv-allmanna-fritid], [familjeliv-allmanna-husdjur], [familjeliv-allmanna-hushem], [familjeliv-allmanna-kropp], [familjeliv-allmanna-noje], [familjeliv-allmanna-samhalle], [familjeliv-allmanna-sandladan], [familjeliv-anglarum], [familjeliv-expert], [familjeliv-foralder], [familjeliv-gravid], [familjeliv-kansliga], [familjeliv-medlem-allmanna], [familjeliv-medlem-foraldrar], [familjeliv-medlem-planerarbarn], [familjeliv-medlem-vantarbarn], [familjeliv-pappagrupp], [familjeliv-planerarbarn], [familjeliv-sexsamlevnad], [familjeliv-svartattfabarn], [lb-open], [poeter], [europarl-sv], [laakartidningen], [fsv-aldrelagar], [fsv-aldrereligiosprosa], [fsv-nysvenskbibel], [fsv-nysvenskdalin], [fsv-nysvenskkronikor], [fsv-nysvensklagar], [fsv-nysvenskovrigt], [fsv-profanprosa], [fsv-verser], [fsv-yngrelagar], [fsv-yngrereligiosprosa], [fsv-yngretankebocker], [standsriksdagen-adelsstandet], [standsriksdagen-bihang], [standsriksdagen-bondestandet], [standsriksdagen-borgarstandet], [standsriksdagen-prastestandet], [standsriksdagen-riksdagsakter], [standsriksdagen-riksdagsbeslut], [strindbergromaner], [strindbergbrev], [dramadialog], [bibel1917], [psalmboken], [akademiliv], [dagens-arena], [gu-journalen], [forskning-framsteg], [sv-covid-19] | 19.39B | | CC-BY-SA 4.0 | [cellar], [wikipedia-sv] | 5.09B | | **Total** | | 24.49B | [dalpilen-1860]: data/dalpilen-1860/dalpilen-1860.md [lag1800]: data/lag1800/lag1800.md [svensk-tidskrift]: data/svensk-tidskrift/svensk-tidskrift.md [statens-offentliga-utredningar]: data/statens-offentliga-utredningar/statens-offentliga-utredningar.md [biblioteksbladet]: data/biblioteksbladet/biblioteksbladet.md [riksdagen-forfattningssamling]: data/riksdagen-forfattningssamling/riksdagen-forfattningssamling.md [riksdagen-reglementen]: data/riksdagen-reglementen/riksdagen-reglementen.md [riksdagen-register]: data/riksdagen-register/riksdagen-register.md [riksdagen-skrivelser]: data/riksdagen-skrivelser/riksdagen-skrivelser.md [riksdagen-utredningar]: data/riksdagen-utredningar/riksdagen-utredningar.md [riksdagen-berattelser]: data/riksdagen-berattelser/riksdagen-berattelser.md [riksdagen-motioner]: data/riksdagen-motioner/riksdagen-motioner.md [riksdagen-betankanden]: data/riksdagen-betankanden/riksdagen-betankanden.md [riksdagen-propositioner]: data/riksdagen-propositioner/riksdagen-propositioner.md [riksdagen-protokoll]: data/riksdagen-protokoll/riksdagen-protokoll.md [cellar]: data/cellar/cellar.md [flashback-dator]: data/flashback-dator/flashback-dator.md [flashback-droger]: data/flashback-droger/flashback-droger.md [flashback-ekonomi]: data/flashback-ekonomi/flashback-ekonomi.md [flashback-fordon]: data/flashback-fordon/flashback-fordon.md [flashback-hem]: data/flashback-hem/flashback-hem.md [flashback-kultur]: data/flashback-kultur/flashback-kultur.md [flashback-livsstil]: data/flashback-livsstil/flashback-livsstil.md [flashback-mat]: data/flashback-mat/flashback-mat.md [flashback-om-flashback]: data/flashback-om-flashback/flashback-om-flashback.md [flashback-ovrigt]: data/flashback-ovrigt/flashback-ovrigt.md [flashback-politik]: data/flashback-politik/flashback-politik.md [flashback-resor]: data/flashback-resor/flashback-resor.md [flashback-samhalle]: data/flashback-samhalle/flashback-samhalle.md [flashback-sex]: data/flashback-sex/flashback-sex.md [flashback-sport]: data/flashback-sport/flashback-sport.md [flashback-vetenskap]: data/flashback-vetenskap/flashback-vetenskap.md [familjeliv-adoption]: data/familjeliv-adoption/familjeliv-adoption.md [familjeliv-allmanna-ekonomi]: data/familjeliv-allmanna-ekonomi/familjeliv-allmanna-ekonomi.md [familjeliv-allmanna-familjeliv]: data/familjeliv-allmanna-familjeliv/familjeliv-allmanna-familjeliv.md [familjeliv-allmanna-fritid]: data/familjeliv-allmanna-fritid/familjeliv-allmanna-fritid.md [familjeliv-allmanna-husdjur]: data/familjeliv-allmanna-husdjur/familjeliv-allmanna-husdjur.md [familjeliv-allmanna-hushem]: data/familjeliv-allmanna-hushem/familjeliv-allmanna-hushem.md [familjeliv-allmanna-kropp]: data/familjeliv-allmanna-kropp/familjeliv-allmanna-kropp.md [familjeliv-allmanna-noje]: data/familjeliv-allmanna-noje/familjeliv-allmanna-noje.md [familjeliv-allmanna-samhalle]: data/familjeliv-allmanna-samhalle/familjeliv-allmanna-samhalle.md [familjeliv-allmanna-sandladan]: data/familjeliv-allmanna-sandladan/familjeliv-allmanna-sandladan.md [familjeliv-anglarum]: data/familjeliv-anglarum/familjeliv-anglarum.md [familjeliv-expert]: data/familjeliv-expert/familjeliv-expert.md [familjeliv-foralder]: data/familjeliv-foralder/familjeliv-foralder.md [familjeliv-gravid]: data/familjeliv-gravid/familjeliv-gravid.md [familjeliv-kansliga]: data/familjeliv-kansliga/familjeliv-kansliga.md [familjeliv-medlem-allmanna]: data/familjeliv-medlem-allmanna/familjeliv-medlem-allmanna.md [familjeliv-medlem-foraldrar]: data/familjeliv-medlem-foraldrar/familjeliv-medlem-foraldrar.md [familjeliv-medlem-planerarbarn]: data/familjeliv-medlem-planerarbarn/familjeliv-medlem-planerarbarn.md [familjeliv-medlem-vantarbarn]: data/familjeliv-medlem-vantarbarn/familjeliv-medlem-vantarbarn.md [familjeliv-pappagrupp]: data/familjeliv-pappagrupp/familjeliv-pappagrupp.md [familjeliv-planerarbarn]: data/familjeliv-planerarbarn/familjeliv-planerarbarn.md [familjeliv-sexsamlevnad]: data/familjeliv-sexsamlevnad/familjeliv-sexsamlevnad.md [familjeliv-svartattfabarn]: data/familjeliv-svartattfabarn/familjeliv-svartattfabarn.md [lb-open]: data/lb-open/lb-open.md [poeter]: data/poeter/poeter.md [wikipedia-sv]: data/wikipedia-sv/wikipedia-sv.md [europarl-sv]: data/europarl-sv/europarl-sv.md [laakartidningen]: data/laakartidningen/laakartidningen.md [fsv-aldrelagar]: data/fsv-aldrelagar/fsv-aldrelagar.md [fsv-aldrereligiosprosa]: data/fsv-aldrereligiosprosa/fsv-aldrereligiosprosa.md [fsv-nysvenskbibel]: data/fsv-nysvenskbibel/fsv-nysvenskbibel.md [fsv-nysvenskdalin]: data/fsv-nysvenskdalin/fsv-nysvenskdalin.md [fsv-nysvenskkronikor]: data/fsv-nysvenskkronikor/fsv-nysvenskkronikor.md [fsv-nysvensklagar]: data/fsv-nysvensklagar/fsv-nysvensklagar.md [fsv-nysvenskovrigt]: data/fsv-nysvenskovrigt/fsv-nysvenskovrigt.md [fsv-profanprosa]: data/fsv-profanprosa/fsv-profanprosa.md [fsv-verser]: data/fsv-verser/fsv-verser.md [fsv-yngrelagar]: data/fsv-yngrelagar/fsv-yngrelagar.md [fsv-yngrereligiosprosa]: data/fsv-yngrereligiosprosa/fsv-yngrereligiosprosa.md [fsv-yngretankebocker]: data/fsv-yngretankebocker/fsv-yngretankebocker.md [standsriksdagen-adelsstandet]: data/standsriksdagen-adelsstandet/standsriksdagen-adelsstandet.md [standsriksdagen-bihang]: data/standsriksdagen-bihang/standsriksdagen-bihang.md [standsriksdagen-bondestandet]: data/standsriksdagen-bondestandet/standsriksdagen-bondestandet.md [standsriksdagen-borgarstandet]: data/standsriksdagen-borgarstandet/standsriksdagen-borgarstandet.md [standsriksdagen-prastestandet]: data/standsriksdagen-prastestandet/standsriksdagen-prastestandet.md [standsriksdagen-riksdagsakter]: data/standsriksdagen-riksdagsakter/standsriksdagen-riksdagsakter.md [standsriksdagen-riksdagsbeslut]: data/standsriksdagen-riksdagsbeslut/standsriksdagen-riksdagsbeslut.md [strindbergromaner]: data/strindbergromaner/strindbergromaner.md [strindbergbrev]: data/strindbergbrev/strindbergbrev.md [dramadialog]: data/dramadialog/dramadialog.md [bibel1917]: data/bibel1917/bibel1917.md [psalmboken]: data/psalmboken/psalmboken.md [akademiliv]: data/akademiliv/akademiliv.md [dagens-arena]: data/dagens-arena/dagens-arena.md [gu-journalen]: data/gu-journalen/gu-journalen.md [forskning-framsteg]: data/forskning-framsteg/forskning-framsteg.md [sv-covid-19]: data/sv-covid-19/sv-covid-19.md <!-- END-LICENSE TABLE --> ## Dataset Structure The dataset contains text from different sources which are thoroughly defined in [Source Data](#source-data). ### Data Instances Each entry in the dataset consists of a single text with associated metadata <!-- START-SAMPLE --> ```py { "id": "standsriksdagen-borgarstandet_00000001", "text": "4> PROTOCOL L, HÅLLNA IIOS VÄLLOFLIGA BORGARE-STÅNDET, VID LAGTIMA RIKSDAGEN I STOCKHOLM åren SS56 o[...]", "source": "standsriksdagen-borgarstandet", "added": "2026-07-27", "created": "1856-01-01, 1856-12-31", "token_count": 489768 } ``` ### Data Fields An entry in the dataset consists of the following fields: - `id` (`str`): A unique identifier for each document. - `text` (`str`): The content of the document. - `source` (`str`): The source of the document (see [Source Data](#source-data)). - `added` (`str`): The date when the document was added to this collection. - `created` (`str`): The date range when the document was originally created. - `token_count` (`int`): The number of tokens in the sample computed using the Llama 3 tokenizer. <!-- END-SAMPLE --> ### Data Splits The entire corpus is provided in the `train` split. ## Dataset Creation ### Curation Rationale These datasets were collected and curated with the intention of making openly licensed Swedish data available. While this was collected with the intention of developing language models it is likely to have multiple other uses such as examining language development and differences across domains. ### Annotations This data generally contains no annotation besides the metadata attached to each sample such as what domain it belongs to. ### Source Data Below follows a brief overview of the sources in the corpus along with their individual license. To get more information about the individual dataset click the hyperlink in the table. <details> <summary><b>Overview Table (click to unfold)</b></summary> You can learn more about each dataset by pressing the link in the first column. <!-- START-MAIN TABLE --> | Source | Description | Domain | N. Tokens | License | |:---------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------|:------------|:---------------| | [cellar] | The official digital repository for European Union legal documents and open data | Legal | 4.73B | [CC-BY-SA 4.0] | | [familjeliv-kansliga] | Sentences from the Delicate Room subforum of [Familjeliv](https://www.familjeliv.se), a Swedish parenting forum | Social Media | 1.55B | [CC-BY 4.0] | | [flashback-politik] | Sentences from the Politics subforum of [Flashback](https://www.flashback.org), a large Swedish internet forum | Social Media | 1.49B | [CC-BY 4.0] | | [flashback-samhalle] | Sentences from the Society subforum of [Flashback](https://www.flashback.org), a large Swedish internet forum | Social Media | 1.45B | [CC-BY 4.0] | | [familjeliv-foralder] | Sentences from the Parents subforum of [Familjeliv](https://www.familjeliv.se), a Swedish parenting forum | Social Media | 962.66M | [CC-BY 4.0] | | [statens-offentliga-utredningar] | Historical Swedish government investigation reports ([*Statens offentliga utredningar*](https://spraakbanken.gu.se/en/resources/sou)) from 1922 to 1996 | Report | 909.69M | [CC-BY 4.0] | | [flashback-vetenskap] | Sentences from the Science & Technology subforum of [Flashback](https://www.flashback.org), a large Swedish internet forum | Social Media | 820.89M | [CC-BY 4.0] | | [flashback-kultur] | Sentences from the Culture & Entertainment subforum of [Flashback](https://www.flashback.org), a large Swedish internet forum | Social Media | 800.01M | [CC-BY 4.0] | | [familjeliv-gravid] | Sentences from the Pregnant subforum of [Familjeliv](https://www.familjeliv.se), a Swedish parenting forum | Social Media | 711.06M | [CC-BY 4.0] | | [riksdagen-propositioner] | Swedish government bills and parliamentary letters from Språkbanken Text's [Bicameral riksdag: Propositions and letters](https://spraakbanken.gu.se/en/resources/tkr-propositioner-skrivelser) | Report | 699.11M | [CC-BY 4.0] | | [riksdagen-protokoll] | Swedish parliamentary debate protocols from Språkbanken Text's [Bicameral riksdag: Protocols](https://spraakbanken.gu.se/en/resources/tkr-protokoll) | Report | 624.22M | [CC-BY 4.0] | | [flashback-hem] | Sentences from the Home & Garden subforum of [Flashback](https://www.flashback.org), a large Swedish internet forum | Social Media | 617.34M | [CC-BY 4.0] | | [lb-open] | Literary works from [Litteraturbanken](https://litteraturbanken.se), the Swedish Literature Bank | Books | 592.11M | [CC-BY 4.0] | | [flashback-dator] | Sentences from the Computers & IT subforum of [Flashback](https://www.flashback.org), a large Swedish internet forum | Social Media | 512.80M | [CC-BY 4.0] | | [familjeliv-allmanna-samhalle] | Sentences from the Society subforum of [Familjeliv](https://www.familjeliv.se), a Swedish parenting forum | Social Media | 494.22M | [CC-BY 4.0] | | [flashback-sport] | Sentences from the Sports subforum of [Flashback](https://www.flashback.org), a large Swedish internet forum | Social Media | 470.50M | [CC-BY 4.0] | | [familjeliv-medlem-foraldrar] | Sentences from the Member Parents subforum of [Familjeliv](https://www.familjeliv.se), a Swedish parenting forum | Social Media | 442.58M | [CC-BY 4.0] | | [familjeliv-medlem-allmanna] | Sentences from the Member General subforum of [Familjeliv](https://www.familjeliv.se), a Swedish parenting forum | Social Media | 431.58M | [CC-BY 4.0] | | [familjeliv-medlem-vantarbarn] | Sentences from the Member Expecting Children subforum of [Familjeliv](https://www.familjeliv.se), a Swedish parenting forum | Social Media | 431.54M | [CC-BY 4.0] | | [riksdagen-betankanden] | Swedish parliamentary committee reports, memorandums and opinions from Språkbanken Text's [Bicameral riksdag: Reports, memorandums and opinions](https://spraakbanken.gu.se/en/resources/tkr-bet-mem-utl) | Report | 423.50M | [CC-BY 4.0] | | [flashback-droger] | Sentences from the Drugs subforum of [Flashback](https://www.flashback.org), a large Swedish internet forum | Social Media | 403.09M | [CC-BY 4.0] | | [wikipedia-sv] | Articles from the Swedish edition of [Wikipedia](https://sv.wikipedia.org), the free encyclopedia | Encyclopedic | 362.29M | [CC-BY-SA 4.0] | | [flashback-ovrigt] | Sentences from the Other subforum of [Flashback](https://www.flashback.org), a large Swedish internet forum | Social Media | 278.09M | [CC-BY 4.0] | | [familjeliv-planerarbarn] | Sentences from the Planning for Children subforum of [Familjeliv](https://www.familjeliv.se), a Swedish parenting forum | Social Media | 276.12M | [CC-BY 4.0] | | [familjeliv-sexsamlevnad] | Sentences from the Sex & Cohabitation subforum of [Familjeliv](https://www.familjeliv.se), a Swedish parenting forum | Social Media | 260.96M | [CC-BY 4.0] | | [flashback-ekonomi] | Sentences from the Economy subforum of [Flashback](https://www.flashback.org), a large Swedish internet forum | Social Media | 248.01M | [CC-BY 4.0] | | [familjeliv-svartattfabarn] | Sentences from the Difficult to Have Children subforum of [Familjeliv](https://www.familjeliv.se), a Swedish parenting forum | Social Media | 232.18M | [CC-BY 4.0] | | [poeter] | Poem lines from [Poeter.se](https://www.poeter.se), a Swedish poetry community website | Books | 220.32M | [CC-BY 4.0] | | [familjeliv-allmanna-kropp] | Sentences from the General Threads – Body subforum of [Familjeliv](https://www.familjeliv.se), a Swedish parenting forum | Social Media | 218.79M | [CC-BY 4.0] | | [flashback-livsstil] | Sentences from the Lifestyle subforum of [Flashback](https://www.flashback.org), a large Swedish internet forum | Social Media | 189.25M | [CC-BY 4.0] | | [familjeliv-medlem-planerarbarn] | Sentences from the Member Planning for Children subforum of [Familjeliv](https://www.familjeliv.se), a Swedish parenting forum | Social Media | 170.87M | [CC-BY 4.0] | | [flashback-sex] | Sentences from the "Sex" subforum of [Flashback](https://www.flashback.org) | Social Media | 155.59M | [CC-BY 4.0] | | [flashback-fordon] | Sentences from the "Fordon & trafik" (Vehicles & Traffic) subforum of [Flashback](https://www.flashback.org) | Social Media | 150.42M | [CC-BY 4.0] | | [riksdagen-motioner] | Swedish parliamentary motions from Språkbanken Text's [Bicameral riksdag: Motions](https://spraakbanken.gu.se/en/resources/tkr-motioner) | Report | 149.83M | [CC-BY 4.0] | | [standsriksdagen-bihang] | Texts from Ståndsriksdagen: Bihang m.m., part of Språkbanken's digitised historical Swedish parliamentary records | Legal | 139.71M | [CC-BY 4.0] | | [familjeliv-allmanna-noje] | Sentences from the "Allmänna rubriker – Nöje" (General Threads – Entertainment) subforum of [Familjeliv](https://www.familjeliv.se) | Social Media | 137.74M | [CC-BY 4.0] | | [flashback-mat] | Sentences from the "Mat, dryck & tobak" (Food, Beverages & Tobacco) subforum of [Flashback](https://www.flashback.org) | Social Media | 137.17M | [CC-BY 4.0] | | [riksdagen-berattelser] | Swedish parliamentary narratives and accounts from Språkbanken Text's [Bicameral riksdag: Narratives and accounts](https://spraakbanken.gu.se/en/resources/tkr-berattelser-redogorelser-frsrdg) | Report | 129.77M | [CC-BY 4.0] | | [standsriksdagen-adelsstandet] | Texts from Ståndsriksdagen: Adelsståndet, part of Språkbanken's digitised historical Swedish parliamentary records | Legal | 126.98M | [CC-BY 4.0] | | [riksdagen-utredningar] | Swedish parliamentary government investigation texts from Språkbanken Text's [Bicameral riksdag: Government official investigations](https://spraakbanken.gu.se/en/resources/tkr-utredningar-kombet-sou) | Report | 125.48M | [CC-BY 4.0] | | [familjeliv-allmanna-hushem] | Sentences from the "Allmänna rubriker – Hus & hem" (General Threads – House & Home) subforum of [Familjeliv](https://www.familjeliv.se) | Social Media | 117.42M | [CC-BY 4.0] | | [familjeliv-allmanna-familjeliv] | Sentences from the "Allmänna rubriker – Familjeliv.se" (General Threads – Familjeliv.se) subforum of [Familjeliv](https://www.familjeliv.se) | Social Media | 98.56M | [CC-BY 4.0] | | [familjeliv-allmanna-ekonomi] | Sentences from the "Allmänna rubriker – Ekonomi & juridik" (General Threads – Economics & Law) subforum of [Familjeliv](https://www.familjeliv.se) | Social Media | 79.97M | [CC-BY 4.0] | | [familjeliv-allmanna-husdjur] | Sentences from the "Allmänna rubriker – Husdjur" (General Threads – Pets) subforum of [Familjeliv](https://www.familjeliv.se) | Social Media | 77.04M | [CC-BY 4.0] | | [standsriksdagen-borgarstandet] | Texts from Ståndsriksdagen: Borgarståndet, part of Språkbanken's digitised historical Swedish parliamentary records | Legal | 71.64M | [CC-BY 4.0] | | [familjeliv-allmanna-sandladan] | Sentences from the "Allmänna rubriker – Sandlådan" (General Threads – Sandbox) subforum of [Familjeliv](https://www.familjeliv.se) | Social Media | 68.62M | [CC-BY 4.0] | | [standsriksdagen-bondestandet] | Texts from Ståndsriksdagen: Bondeståndet, part of Språkbanken's digitised historical Swedish parliamentary records | Legal | 66.32M | [CC-BY 4.0] | | [flashback-resor] | Sentences from the "Resor" (Travel) subforum of [Flashback](https://www.flashback.org) | Social Media | 64.76M | [CC-BY 4.0] | | [europarl-sv] | Texts from the Swedish portion of the [Europarl](https://spraakbanken.gu.se/resurser/europarl-sv) corpus, transcribed proceedings of the European Parliament | Speeches | 63.02M | [CC-BY 4.0] | | [standsriksdagen-prastestandet] | Texts from Ståndsriksdagen: Prästeståndet, part of Språkbanken's digitised historical Swedish parliamentary records | Legal | 62.12M | [CC-BY 4.0] | | [riksdagen-skrivelser] | Swedish parliamentary letters and communications from Språkbanken Text's [Bicameral riksdag: Letters of the Riksdag](https://spraakbanken.gu.se/en/resources/tkr-rskr) | Report | 61.79M | [CC-BY 4.0] | | [flashback-om-flashback] | Sentences from the "Om Flashback" (About Flashback) subforum of [Flashback](https://www.flashback.org) | Social Media | 57.87M | [CC-BY 4.0] | | [familjeliv-allmanna-fritid] | Sentences from the "Allmänna rubriker – Fritid & hobby" (General Threads – Leisure & Hobby) subforum of [Familjeliv](https://www.familjeliv.se) | Social Media | 52.49M | [CC-BY 4.0] | | [riksdagen-register] | Swedish parliamentary register texts from Språkbanken Text's [Bicameral riksdag: Register](https://spraakbanken.gu.se/en/resources/tkr-register) | Report | 44.16M | [CC-BY 4.0] | | [laakartidningen] | Sentences from [Läkartidningen](https://lakartidningen.se), the peer-reviewed journal of the Swedish Medical Association, 1996–2006 | Medical | 42.40M | [CC-BY 4.0] | | [familjeliv-anglarum] | Sentences from the "Änglarum" (Angel Room) subforum of [Familjeliv](https://www.familjeliv.se) | Social Media | 29.69M | [CC-BY 4.0] | | [dagens-arena] | Sentences from Dagens Arena, distributed by [Språkbanken Text](https://spraakbanken.gu.se/resurser/da) | News | 17.80M | [CC-BY 4.0] | | [dalpilen-1860] | Historical Swedish newspaper issues from Språkbanken Text's [Dalpilen 1860's](https://spraakbanken.gu.se/en/resources/kubhist2-dalpilen-1860) | News | 17.51M | [CC-BY 4.0] | | [familjeliv-adoption] | Sentences from the "Adoption" subforum of [Familjeliv](https://www.familjeliv.se) | Social Media | 16.23M | [CC-BY 4.0] | | [sv-covid-19] | Sentences from sv-COVID-19, distributed by [Språkbanken Text](https://spraakbanken.gu.se/resurser/sv-covid-19) | News | 15.05M | [CC-BY 4.0] | | [svensk-tidskrift] | Historical Swedish conservative journal ([*Svensk Tidskrift*](https://spraakbanken.gu.se/en/resources/runeberg-svtidskr)) covering 27 annual volumes from 1891 to 1940 | News | 13.89M | [CC-BY 4.0] | | [familjeliv-expert] | Sentences from the "Fråga experten" (Ask the Expert) subforum of [Familjeliv](https://www.familjeliv.se) | Social Media | 12.52M | [CC-BY 4.0] | | [gu-journalen] | Texts from GU Journalen, distributed by [Språkbanken Text](https://spraakbanken.gu.se/resurser/gujournalen) | News | 8.85M | [CC-BY 4.0] | | [biblioteksbladet] | Early volumes of the Swedish library journal *Biblioteksbladet* (1916–1940) | News | 8.71M | [CC-BY 4.0] | | [standsriksdagen-riksdagsakter] | Texts from Tvåkammarriksdagen: Riksdagsakter, part of Språkbanken's digitised historical Swedish parliamentary records | Legal | 7.74M | [CC-BY 4.0] | | [strindbergromaner] | Texts from "August Strindbergs romaner" (August Strindberg's novels), distributed by [Språkbanken Text](https://spraakbanken.gu.se/resurser/strindbergromaner) | Books | 7.31M | [CC-BY 4.0] | | [riksdagen-reglementen] | Swedish parliamentary regulations from Språkbanken Text's [Bicameral riksdag: Regulations](https://spraakbanken.gu.se/en/resources/tkr-reglementen-sfs) | Legal | 5.69M | [CC-BY 4.0] | | [akademiliv] | Sentences from Akademiliv, distributed by [Språkbanken Text](https://spraakbanken.gu.se/resurser/akademiliv) | News | 4.42M | [CC-BY 4.0] | | [familjeliv-pappagrupp] | Sentences from the "Pappagrupp" (Daddy Group) subforum of [Familjeliv](https://www.familjeliv.se) | Social Media | 3.58M | [CC-BY 4.0] | | [fsv-yngrereligiosprosa] | Texts from the "Yngre religiös prosa" (Younger Religious Prose) collection of [Fornsvenska textbanken](http://project2.sol.lu.se/fornsvenska/), a historical Swedish text archive | Books | 2.63M | [CC-BY 4.0] | | [strindbergbrev] | Texts from "August Strindbergs brev" (August Strindberg's letters), distributed by [Språkbanken Text](https://spraakbanken.gu.se/resurser/strindbergbrev) | Books | 2.39M | [CC-BY 4.0] | | [bibel1917] | Texts from Bibeln 1917, distributed by [Språkbanken Text](https://spraakbanken.gu.se/resurser/bibel1917) | Books | 1.48M | [CC-BY 4.0] | | [forskning-framsteg] | Sentences from Forskning & Framsteg, distributed by [Språkbanken Text](https://spraakbanken.gu.se/resurser/fof) | News | 1.44M | [CC-BY 4.0] | | [dramadialog] | Texts from Svensk Dramadialog, distributed by [Språkbanken Text](https://spraakbanken.gu.se/resurser/dramadialog) | Books | 1.06M | [CC-BY 4.0] | | [fsv-aldrelagar] | Texts from the "Äldre lagar" (Older Laws) collection of [Fornsvenska textbanken](http://project2.sol.lu.se/fornsvenska/), a historical Swedish text archive | Legal | 1.01M | [CC-BY 4.0] | | [lag1800] | Historical Swedish legal texts from Språkbanken Text's [Laws from the 1800's](https://spraakbanken.gu.se/en/resources/lag1800) | Legal | 921.60K | [CC-BY 4.0] | | [fsv-aldrereligiosprosa] | Texts from the "Äldre religiös prosa" (Older Religious Prose) collection of [Fornsvenska textbanken](http://project2.sol.lu.se/fornsvenska/), a historical Swedish text archive | Books | 916.75K | [CC-BY 4.0] | | [standsriksdagen-riksdagsbeslut] | Texts from Ståndsriksdagen: Riksdagsbeslut, part of Språkbanken's digitised historical Swedish parliamentary records | Legal | 707.46K | [CC-BY 4.0] | | [fsv-nysvenskovrigt] | Texts from the "Nysvenska, övrigt" (Early Modern Swedish, Miscellaneous) collection of [Fornsvenska textbanken](http://project2.sol.lu.se/fornsvenska/), a historical Swedish text archive | Books | 611.10K | [CC-BY 4.0] | | [fsv-nysvenskkronikor] | Texts from the "Nysvenska krönikor" (Early Modern Swedish Chronicles) collection of [Fornsvenska textbanken](http://project2.sol.lu.se/fornsvenska/), a historical Swedish text archive | Books | 598.00K | [CC-BY 4.0] | | [fsv-nysvenskdalin] | Texts from the "Dalins Then Swänska Argus 1732-1734" (Dalin's Then Swänska Argus) collection of [Fornsvenska textbanken](http://project2.sol.lu.se/fornsvenska/), a historical Swedish text archive | Books | 449.85K | [CC-BY 4.0] | | [fsv-profanprosa] | Texts from the "Profan prosa" (Secular Prose) collection of [Fornsvenska textbanken](http://project2.sol.lu.se/fornsvenska/), a historical Swedish text archive | Books | 431.01K | [CC-BY 4.0] | | [fsv-verser] | Texts from the "Verser" (Verse) collection of [Fornsvenska textbanken](http://project2.sol.lu.se/fornsvenska/), a historical Swedish text archive | Books | 339.94K | [CC-BY 4.0] | | [fsv-yngretankebocker] | Texts from the "Yngre tänkeböcker" (Younger Thought Books) collection of [Fornsvenska textbanken](http://project2.sol.lu.se/fornsvenska/), a historical Swedish text archive | Books | 296.24K | [CC-BY 4.0] | | [psalmboken] | Sentences from Psalmboken (1937), distributed by [Språkbanken Text](https://spraakbanken.gu.se/resurser/psalmboken) | Books | 241.53K | [CC-BY 4.0] | | [fsv-yngrelagar] | Texts from the "Yngre lagar" (Younger Laws) collection of [Fornsvenska textbanken](http://project2.sol.lu.se/fornsvenska/), a historical Swedish text archive | Legal | 202.68K | [CC-BY 4.0] | | [riksdagen-forfattningssamling] | Swedish parliamentary constitutional texts from Språkbanken Text's [Bicameral riksdag: The constitution of the Riksdag](https://spraakbanken.gu.se/en/resources/tkr-riksdagens-forfattningssamling-rfs) | Legal | 185.96K | [CC-BY 4.0] | | [fsv-nysvenskbibel] | Texts from the "Nysvenska bibelböcker" (Early Modern Swedish Bible Books) collection of [Fornsvenska textbanken](http://project2.sol.lu.se/fornsvenska/), a historical Swedish text archive | Books | 79.17K | [CC-BY 4.0] | | [fsv-nysvensklagar] | Texts from the "Nysvenska lagar" (Early Modern Swedish Laws) collection of [Fornsvenska textbanken](http://project2.sol.lu.se/fornsvenska/), a historical Swedish text archive | Legal | 39.60K | [CC-BY 4.0] | | **Total** | | | 24.49B | | [dalpilen-1860]: data/dalpilen-1860/dalpilen-1860.md [lag1800]: data/lag1800/lag1800.md [svensk-tidskrift]: data/svensk-tidskrift/svensk-tidskrift.md [statens-offentliga-utredningar]: data/statens-offentliga-utredningar/statens-offentliga-utredningar.md [biblioteksbladet]: data/biblioteksbladet/biblioteksbladet.md [riksdagen-forfattningssamling]: data/riksdagen-forfattningssamling/riksdagen-forfattningssamling.md [riksdagen-reglementen]: data/riksdagen-reglementen/riksdagen-reglementen.md [riksdagen-register]: data/riksdagen-register/riksdagen-register.md [riksdagen-skrivelser]: data/riksdagen-skrivelser/riksdagen-skrivelser.md [riksdagen-utredningar]: data/riksdagen-utredningar/riksdagen-utredningar.md [riksdagen-berattelser]: data/riksdagen-berattelser/riksdagen-berattelser.md [riksdagen-motioner]: data/riksdagen-motioner/riksdagen-motioner.md [riksdagen-betankanden]: data/riksdagen-betankanden/riksdagen-betankanden.md [riksdagen-propositioner]: data/riksdagen-propositioner/riksdagen-propositioner.md [riksdagen-protokoll]: data/riksdagen-protokoll/riksdagen-protokoll.md [cellar]: data/cellar/cellar.md [flashback-dator]: data/flashback-dator/flashback-dator.md [flashback-droger]: data/flashback-droger/flashback-droger.md [flashback-ekonomi]: data/flashback-ekonomi/flashback-ekonomi.md [flashback-fordon]: data/flashback-fordon/flashback-fordon.md [flashback-hem]: data/flashback-hem/flashback-hem.md [flashback-kultur]: data/flashback-kultur/flashback-kultur.md [flashback-livsstil]: data/flashback-livsstil/flashback-livsstil.md [flashback-mat]: data/flashback-mat/flashback-mat.md [flashback-om-flashback]: data/flashback-om-flashback/flashback-om-flashback.md [flashback-ovrigt]: data/flashback-ovrigt/flashback-ovrigt.md [flashback-politik]: data/flashback-politik/flashback-politik.md [flashback-resor]: data/flashback-resor/flashback-resor.md [flashback-samhalle]: data/flashback-samhalle/flashback-samhalle.md [flashback-sex]: data/flashback-sex/flashback-sex.md [flashback-sport]: data/flashback-sport/flashback-sport.md [flashback-vetenskap]: data/flashback-vetenskap/flashback-vetenskap.md [familjeliv-adoption]: data/familjeliv-adoption/familjeliv-adoption.md [familjeliv-allmanna-ekonomi]: data/familjeliv-allmanna-ekonomi/familjeliv-allmanna-ekonomi.md [familjeliv-allmanna-familjeliv]: data/familjeliv-allmanna-familjeliv/familjeliv-allmanna-familjeliv.md [familjeliv-allmanna-fritid]: data/familjeliv-allmanna-fritid/familjeliv-allmanna-fritid.md [familjeliv-allmanna-husdjur]: data/familjeliv-allmanna-husdjur/familjeliv-allmanna-husdjur.md [familjeliv-allmanna-hushem]: data/familjeliv-allmanna-hushem/familjeliv-allmanna-hushem.md [familjeliv-allmanna-kropp]: data/familjeliv-allmanna-kropp/familjeliv-allmanna-kropp.md [familjeliv-allmanna-noje]: data/familjeliv-allmanna-noje/familjeliv-allmanna-noje.md [familjeliv-allmanna-samhalle]: data/familjeliv-allmanna-samhalle/familjeliv-allmanna-samhalle.md [familjeliv-allmanna-sandladan]: data/familjeliv-allmanna-sandladan/familjeliv-allmanna-sandladan.md [familjeliv-anglarum]: data/familjeliv-anglarum/familjeliv-anglarum.md [familjeliv-expert]: data/familjeliv-expert/familjeliv-expert.md [familjeliv-foralder]: data/familjeliv-foralder/familjeliv-foralder.md [familjeliv-gravid]: data/familjeliv-gravid/familjeliv-gravid.md [familjeliv-kansliga]: data/familjeliv-kansliga/familjeliv-kansliga.md [familjeliv-medlem-allmanna]: data/familjeliv-medlem-allmanna/familjeliv-medlem-allmanna.md [familjeliv-medlem-foraldrar]: data/familjeliv-medlem-foraldrar/familjeliv-medlem-foraldrar.md [familjeliv-medlem-planerarbarn]: data/familjeliv-medlem-planerarbarn/familjeliv-medlem-planerarbarn.md [familjeliv-medlem-vantarbarn]: data/familjeliv-medlem-vantarbarn/familjeliv-medlem-vantarbarn.md [familjeliv-pappagrupp]: data/familjeliv-pappagrupp/familjeliv-pappagrupp.md [familjeliv-planerarbarn]: data/familjeliv-planerarbarn/familjeliv-planerarbarn.md [familjeliv-sexsamlevnad]: data/familjeliv-sexsamlevnad/familjeliv-sexsamlevnad.md [familjeliv-svartattfabarn]: data/familjeliv-svartattfabarn/familjeliv-svartattfabarn.md [lb-open]: data/lb-open/lb-open.md [poeter]: data/poeter/poeter.md [wikipedia-sv]: data/wikipedia-sv/wikipedia-sv.md [europarl-sv]: data/europarl-sv/europarl-sv.md [laakartidningen]: data/laakartidningen/laakartidningen.md [fsv-aldrelagar]: data/fsv-aldrelagar/fsv-aldrelagar.md [fsv-aldrereligiosprosa]: data/fsv-aldrereligiosprosa/fsv-aldrereligiosprosa.md [fsv-nysvenskbibel]: data/fsv-nysvenskbibel/fsv-nysvenskbibel.md [fsv-nysvenskdalin]: data/fsv-nysvenskdalin/fsv-nysvenskdalin.md [fsv-nysvenskkronikor]: data/fsv-nysvenskkronikor/fsv-nysvenskkronikor.md [fsv-nysvensklagar]: data/fsv-nysvensklagar/fsv-nysvensklagar.md [fsv-nysvenskovrigt]: data/fsv-nysvenskovrigt/fsv-nysvenskovrigt.md [fsv-profanprosa]: data/fsv-profanprosa/fsv-profanprosa.md [fsv-verser]: data/fsv-verser/fsv-verser.md [fsv-yngrelagar]: data/fsv-yngrelagar/fsv-yngrelagar.md [fsv-yngrereligiosprosa]: data/fsv-yngrereligiosprosa/fsv-yngrereligiosprosa.md [fsv-yngretankebocker]: data/fsv-yngretankebocker/fsv-yngretankebocker.md [standsriksdagen-adelsstandet]: data/standsriksdagen-adelsstandet/standsriksdagen-adelsstandet.md [standsriksdagen-bihang]: data/standsriksdagen-bihang/standsriksdagen-bihang.md [standsriksdagen-bondestandet]: data/standsriksdagen-bondestandet/standsriksdagen-bondestandet.md [standsriksdagen-borgarstandet]: data/standsriksdagen-borgarstandet/standsriksdagen-borgarstandet.md [standsriksdagen-prastestandet]: data/standsriksdagen-prastestandet/standsriksdagen-prastestandet.md [standsriksdagen-riksdagsakter]: data/standsriksdagen-riksdagsakter/standsriksdagen-riksdagsakter.md [standsriksdagen-riksdagsbeslut]: data/standsriksdagen-riksdagsbeslut/standsriksdagen-riksdagsbeslut.md [strindbergromaner]: data/strindbergromaner/strindbergromaner.md [strindbergbrev]: data/strindbergbrev/strindbergbrev.md [dramadialog]: data/dramadialog/dramadialog.md [bibel1917]: data/bibel1917/bibel1917.md [psalmboken]: data/psalmboken/psalmboken.md [akademiliv]: data/akademiliv/akademiliv.md [dagens-arena]: data/dagens-arena/dagens-arena.md [gu-journalen]: data/gu-journalen/gu-journalen.md [forskning-framsteg]: data/forskning-framsteg/forskning-framsteg.md [sv-covid-19]: data/sv-covid-19/sv-covid-19.md [CC-0]: https://creativecommons.org/publicdomain/zero/1.0/legalcode.en [CC-BY-SA 4.0]: https://creativecommons.org/licenses/by-sa/4.0/deed.en [CC-BY 4.0]: https://creativecommons.org/licenses/by/4.0/deed.en [Apache 2.0]: https://www.apache.org/licenses/LICENSE-2.0 <!-- END-MAIN TABLE --> </details> ### Data Collection and Processing This dynaword is continually developed, which means that the dataset will actively be updated as new datasets become available. This means that the size of Dynaword increases over time as seen in the following plot: <p align="center"> <img src="./images/tokens_over_time.svg" width="600" style="margin-right: 10px;" /> </p> The data collection and processing varies depending on the dataset and is documentationed the individual datasheets, which is linked in the above table. If possible the collection is documented both in the datasheet and in the reproducible script (`data/{dataset}/create.py`). In addition to data specific processing we also run a series automated quality checks to ensure formatting (e.g. ensuring correctly formatted columns and unique IDs), quality checks (e.g. duplicate and empty string detection) and datasheet documentation checks. These checks are there to ensure a high quality of documentation and a minimal level of quality. To allow for the development of novel cleaning methodologies we do not provide more extensive cleaning. ### Dataset Statistics The following plot(s) are intended to give an overview of docuements length in the various sources. <p align="center"> <img src="./images/dataset_size_plot.svg" width="600" style="margin-right: 10px;" /> </p> ### Contributing to the dataset We welcome contributions to the dataset, including new sources, improved data filtering, and other enhancements. To get started on contributing, please see [the contribution guidelines](CONTRIBUTING.md) ## Citation Information If you use this work, please cite the [scientific article](https://arxiv.org/abs/2508.02271) introducing the Dynaword approach. Many of the sources in this dataset are distributed by [Språkbanken Text](https://spraakbanken.gu.se/en), which should also be cited: > Enevoldsen, K.C., Jensen, K.N., Kostkan, J., Szab'o, B.I., Kardos, M., Vad, K., Heinsen, J., N'unez, A.B., Barmina, G., Nielsen, J., Larsen, R., Vahlstrup, P.B., Dalum, P.M., Elliott, D., Galke, L., Schneider-Kamp, P., & Nielbo, K.L. (2025). Dynaword: From One-shot to Continuously Developed Datasets. > > Forsberg, M., Dannélls, D., Borin, L., & Berdicevskis, A. (2025). Background: Språkbanken Text. In D. Dannélls, K. Blensenius & L. Borin (Eds.), *Sixty years of Swedish computational lexicography* (pp. 161–173). ```bibtex @article{enevoldsen2025dynaword, title={Dynaword: From One-shot to Continuously Developed Datasets}, author={Enevoldsen, Kenneth and Jensen, Kristian N{\o}rgaard and Kostkan, Jan and Szab{\'o}, Bal{\'a}zs and Kardos, M{\'a}rton and Vad, Kirten and N{\'u}{\~n}ez, Andrea Blasi and Barmina, Gianluca and Nielsen, Jacob and Larsen, Rasmus and others}, journal={arXiv preprint arXiv:2508.02271}, year={2025} } @incollection{forsberg2025spraakbanken, title={Background: {S}pr{\aa}kbanken {T}ext}, author={Forsberg, Markus and Dann{\'e}lls, Dana and Borin, Lars and Berdicevskis, Aleksandrs}, booktitle={Sixty years of {S}wedish computational lexicography}, editor={Dann{\'e}lls, Dana and Blensenius, Kristian and Borin, Lars}, pages={161--173}, year={2025} } ``` Additionally, we recommend citing the relevant source datasets as well. See the individual datasheets for more information. ## License information The license for each constituent dataset is supplied in the [Source data](#source-data) table. This license is applied to the constituent data, i.e., the text. The collection of datasets (metadata, quality control, etc.) is licensed under [CC-0](https://creativecommons.org/publicdomain/zero/1.0/legalcode.en). ### Personal and Sensitive Information As far as we are aware the dataset does not contain information identifying sexual orientation, political beliefs, religion, or health connected along with a personal identifier of any non-public or non-historic figures. ### Bias, Risks, and Limitations Certain works in this collection are historical works and thus reflect the linguistic, cultural, and ideological norms of their time. As such, it includes perspectives, assumptions, and biases characteristic of the period, which may be considered offensive or exclusionary by contemporary standards. ### Notice and takedown policy We redistribute files shared with us under a license permitting such redistribution. If you have concerns about the licensing of these files, please [contact us](https://huggingface.co/datasets/danish-foundation-models/swedish-dynaword/discussions/new). If you consider that the data contains material that infringe your copyright, please: - Clearly identify yourself with detailed contact information such as an address, a telephone number, or an email address at which you can be contacted. - Clearly reference the original work claimed to be infringed - Clearly identify the material claimed to be infringing and information reasonably sufficient to allow us to locate the material. You can contact us through this channel. We will comply with legitimate requests by removing the affected sources from the next release of the corpus --- <h3 style="display: flex; align-items: center;"> <a href="https://www.foundationmodels.dk"> <img src="./docs/icon.png" width="30" style="margin-right: 10px;" /> </a> A <a href=https://www.foundationmodels.dk>Danish Foundation Models</a> dataset </h3>
