智慧物流生态云平台处理速度测试数据集

该数据集用来测试平台实时接入速度（抽取）120k/s binlog解析，批量抽取单核5000+/s；数据集通过智慧物流生态云平台获取，使用 Grafana v5.2.4并发监控2个线程，查看10:56:30 到 11:00:00的 Canal 解析 Event 总速率 （Binlog 日志解析的速率）达到130（50＋80）KB/s以上，大于120KB/s；
使用工具DataX 3.0执行从阿里云批量抽取数据库t_user执行数据查询，可从日志查看记录写入速度（一核CPU处理事务的 速率）为37836rec/s，大于5000rec/s，
通过人工审核的方法进行数据处理

This dataset is designed to test the real-time access speed (extraction performance) of the platform: the binlog parsing speed reaches 120k/s, and the single-core batch extraction speed exceeds 5000 records per second. The dataset is collected from the Smart Logistics Ecological Cloud Platform. We utilized Grafana v5.2.4 to monitor the concurrency of two threads, and observed that the total Canal parsing Event rate (i.e., Binlog log parsing rate) between 10:56:30 and 11:00:00 exceeded 130 (50+80) KB/s, which is higher than the 120 KB/s threshold. Using the DataX 3.0 tool, we conducted batch data extraction and query on the t_user database hosted on Alibaba Cloud. According to the logs, the record writing speed (i.e., transaction processing rate of a single CPU core) reached 37836 records per second, surpassing the 5000 records per second requirement. Data processing for this dataset was completed via manual review.

搜集汇总

背景与挑战

背景概述

该数据集用于测试智慧物流生态云平台的实时接入与批量抽取处理速度。测试结果显示，binlog解析速率达到130 KB/s以上，批量抽取时单核CPU处理事务速率为37836rec/s，均超过预设性能指标。