An Empirical Evaluation of Video Conferencing Systems Used in Industry, Academia, and Entertainment

Video Conferencing Systems~(VCS) are used daily--at work, in online education, and for get-togethers with friends and family. Many new VCSs have emerged in the past decade and a new market-leader has risen during the coronavirus period of 2020. Understanding how these systems work could help us improve them rapidly. However, no experimental comparison of such systems currently exists. We propose a method to compare VCSs in real-world operation and implement it as a tool. Our method considers 4 main real-world experiments. Each captures different aspects, such as communication channels~(audio, video, audio-video), types of network environments~(e.g., Ethernet, WiFi, 4G), and reports system and network utilization. We conduct real-world experiments with 3 VCSs, Zoom, Microsoft Team, and Discord. This dataset is the result of our experiments conducted with the Online Video Conference Benchmarking tool.

视频会议系统（Video Conferencing Systems，VCS）目前已广泛应用于日常办公、在线教育以及亲友聚会等诸多场景。近十年来，市场上涌现出大量新型视频会议系统，且在2020年新冠疫情期间诞生了新的市场领军者。明晰此类系统的运行机制，有助于我们快速推进其优化升级。然而目前尚无针对这类系统的系统性实验对比研究。为此，我们提出一种可在真实运行环境中对比视频会议系统的方法，并将其实现为一款专用工具。该方法包含四项核心真实场景实验，每项实验均覆盖不同的评估维度，包括通信信道类型（音频、视频、音视频混合）、网络环境类型（如以太网、WiFi、4G）等，并可采集并报告系统与网络的资源利用率。我们针对三款主流视频会议系统——Zoom、Microsoft Team以及Discord开展了真实环境下的对比实验。本数据集即为我们通过在线视频会议基准测试工具（Online Video Conference Benchmarking tool）所获取的实验结果汇总。