Metrics of the predictive models.

Purpose
The COVID-19 pandemic has changed customer social media engagement behavior, which challenges the establishment of effective marketing strategies to strengthen digital communication with customers and leads to new opportunities for social media competitive intelligence analytics. This study presents a new social media competitive intelligence framework that incorporates not only the detection of brand topics before and during the COVID-19 pandemic but also the prediction of customer engagement.
Design/Methodology/Approach
A sector-based empirical study is conducted to illustrate the implementation of the proposed framework. We collected tweets generated by 23 leading American catering brands before and during the pandemic. First, we used Amazon Comprehend and Latent Dirichlet allocation (LDA) to extract sentiments and topics behind unstructured text data. Second, we trained and compared the performance of six machine learning algorithms to find the optimal classifiers.
Findings
The study reveals significant shifts in social media engagement topics following the COVID-19 pandemic. Pre-pandemic topics primarily included “Food and lifestyle”, “Promotion”, “Food ordering”, “Food time”, and “Food delivery”. During the pandemic, the topics expanded to include “Social responsibility” and “Contactless ordering”. For predicting customer engagement, the performance metrics show that Random Forest and C5.0 (C50) are generally the best-performing models, with Random Forest being particularly strong for "Likes" and “Retweets”, while C50 performs best for “Replies”.
Originality
This framework differentiates itself from existing competitive intelligence frameworks by integrating the influence of external factors, such as the COVID-19 pandemic, and expanding the analysis from topic detection to customer engagement prediction. This dual focus provides a more comprehensive approach to social media competitive intelligence.