Insight4news Irish news related tweet collection 15.07.2015-24.05.2017

<pre>The 1.3GB tar.gz file contains a 3.6GB (uncompressed) .txt file with 198'725'860 rows, each row of which is a tweet ID. These tweets have been collected in 15.07.2015-24.05.2017 period with the Hashtagger platform, which considered these tweets relevant to the monitored stream of news from Irish sources (The Irish Times, Irish Examiner, etc.). All 198'725'860 tweets contain at least one hashtag. Hydrate the tweet ids with Twarc and write to a file. You will need to provide Twarc with a set of Twitter API keys. For more information, see [Twarc's documentation](https://github.com/edsu/twarc#twitter-api-keys). twarc.py --hydrate tweet_ids.txt &gt; tweets.json It is probably not a good idea to hydrate all the tweets in one go, and may be better to split the file into chuncks and hydrate the tweets chunk-by-chunk. </pre>