Invasion of Ukraine Discourse on TikTok Dataset

This is a dataset of videos and comments related to the invasion of Ukraine, published on TikTok by a number of users over the year of 2022. It was compiled by Benjamin Steel, Sara Parker and Derek Ruths at the Network Dynamics Lab, McGill University. We created this dataset to facilitate the study of TikTok, and the nature of social interaction on the platform relevant to a major political event. The dataset has been released here on Zenodo: https://doi.org/10.5281/zenodo.7534952 as well as on Github: https://github.com/networkdynamics/data-and-code/tree/master/ukraine_tiktok To create the dataset, we identified hashtags and keywords explicitly related to the conflict to collect a core set of videos (or ”TikToks”). We then compiled comments associated with these videos. All of the data captured is publically available information, and contains personally identifiable information. In total we collected approximately 16 thousand videos and 12 million comments, from approximately 6 million users. There are approximately 1.9 comments on average per user captured, and 1.5 videos per user who posted a video. The author personally collected this data using the web scraping PyTok library, developed by the author: https://github.com/networkdynamics/pytok. Due to scraping duration, this is just a sample of the publically available discourse concerning the invasion of Ukraine on TikTok. Due to the fuzzy search functionality of the TikTok, the dataset contains videos with a range of relatedness to the invasion. We release here the unique video IDs of the dataset in a CSV format. The data was collected without the specific consent of the content creators, so we have released only the data required to re-create it, to allow users to delete content from TikTok and be removed from the dataset if they wish. Contained in this repository are scripts that will automatically pull the full dataset, which will take the form of JSON files organised into a folder for each video. The JSON files are the entirety of the data returned by the TikTok API. We include a script to parse the JSON files into CSV files with the most commonly used data. We plan to further expand this dataset as collection processes progress and the war continues. We will version the dataset to ensure reproducibility. To build this dataset from the IDs here: Go to https://github.com/networkdynamics/pytok and clone the repo locally Run <code>pip install -e .</code> in the pytok directory Run <code>pip install pandas tqdm</code> to install these libraries if not already installed Run <code>get_videos.py</code> to get the video data Run <code>video_comments.py</code> to get the comment data Run <code>user_tiktoks.py</code> to get the video history of the users Run <code>hashtag_tiktoks.py</code> or <code>search_tiktoks.py</code> to get more videos from other hashtags and search terms Run <code>load_json_to_csv.py</code> to compile the JSON files into two CSV files, <code>comments.csv</code> and <code>videos.csv</code> If you get an error about the wrong chrome version, use the command line argument <code>get_videos.py --chrome-version YOUR_CHROME_VERSION</code> Please note pulling data from TikTok takes a while! We recommend leaving the scripts running on a server for a while for them to finish downloading everything. Feel free to play around with the delay constants to either speed up the process or avoid TikTok rate limiting. Please do not hesitate to make an issue in this repo to get our help with this! The <code>videos.csv</code> will contain the following columns: <code>video_id</code>: Unique video ID <code>createtime</code>: UTC datetime of video creation time in YYYY-MM-DD HH:MM:SS format <code>author_name</code>: Unique author name <code>author_id</code>: Unique author ID <code>desc</code>: The full video description from the author <code>hashtags</code>: A list of hashtags used in the video description <code>share_video_id</code>: If the video is sharing another video, this is the video ID of that original video, else empty <code>share_video_user_id</code>: If the video is sharing another video, this the user ID of the author of that video, else empty <code>share_video_user_name</code>: If the video is sharing another video, this is the user name of the author of that video, else empty <code>share_type</code>: If the video is sharing another video, this is the type of the share, stitch, duet etc. <code>mentions</code>: A list of users mentioned in the video description, if any The <code>comments.csv</code> will contain the following columns: <code>comment_id</code>: Unique comment ID <code>createtime</code>: UTC datetime of comment creation time in YYYY-MM-DD HH:MM:SS format <code>author_name</code>: Unique author name <code>author_id</code>: Unique author ID <code>text</code>: Text of the comment <code>mentions</code>: A list of users that are tagged in the comment <code>video_id</code>: The ID of the video the comment is on <code>comment_language</code>: The language of the comment, as predicted by the TikTok API <code>reply_comment_id</code>: If the comment is replying to another comment, this is the ID of that comment The date can be compiled into a user interaction network to facilitate study of interaction dynamics. There is code to help with that here: https://github.com/networkdynamics/polar-seeds. Additional scripts for further preprocessing of this data can be found there too.