Using Twitter Dataset for Social Listening in Singapore

This study delves into analyzing social media data sourced from Twitter within the context of Singapore, forming a crucial component of a broader social listening initiative. We provide a decade’s worth of social data from Singapore, offering invaluable insights for the research community. This work presents two analytical approaches utilizing this dataset: sentiment analysis and bursty topic detection. Sentiment analysis for direct search is based on zero shot pretrained model while busrty topic analysis is based on biterm topic model. The detailed experiments demonstrate the efficacy of the approach for analyzing social trends using Twitter data. We collected all twitter data posted in Singapore from 2008 to 2023. The geocode setting as (1.346353, 103.807526, 25km) was used in Twitter API to cover the whole of Singapore. The total number of tweets in this dataset is 96,686,894. There are 3 data files: 1. place.json includes 10k detailed places information in Singapore.2.subzones.json includes 332 subzone information in Singapore 3.tweets.json includes 96M+tweets posted in Singapore. MongoDB was used as the database to store and manage the data.