yashsingh1/xdaforums

--- license: apache-2.0 --- # XDA Forums Dataset <!-- Provide a quick summary of the dataset. --> This dataset contains all of the posts throughout the history of XDA Forums. ## Dataset Details ### Dataset Description The dataset contains the following: - Threads - Posts - Users - Categories/forums The posts are provided in both HTML and markdown format. Some of the markdown formatted text won't be perfect and may appear in a XenForo specific markup format. ### Dataset Sources This dataset is sourced from HTML [entries](https://xdaforums.com/sitemap.xml) on xdaforums.com and using the XDA Forums Android/Web APIs. ## Dataset Format The dataset contains several partitioned Parquet files with about 68 million posts. Alternatively, you can also use the original RocksDB formatted database. Take a look at the `orm.ts` file for a reference on how the RocksDB database is structured.