机器人实时数据清洗与建模数据集
收藏资源简介：
产品内容本数据集本质是一套高保真的时序数据流记录，包含：时序对齐的原始数据切片（用于追溯），对应的实时清洗日志与中间特征序列，毫秒/秒级延时的模型实时输出标签。一、 数据集构成本数据集是一个按时间戳严格对齐、结构化存储的时序数据集合，完整记录了机器人从感知到执行的全链路实时处理过程。其主要由以下五类数据组成，共同构成一个可追溯、可分析的数据管道：1. 原始感知数据流切片内容：经过时间同步后的多模态原始数据短切片（通常为1-5秒的连续窗口），用于追溯和基准测试。包含字段示例：timestamp: 毫秒级统一时间戳 (主键)。data_source: 数据来源标识（如：audio_mic_array, rgb_camera, force_sensor_hand, vibration_sensor_motor）。raw_data_payload: 经过编码的原始数据或数据存储路径指针。scene_tag: 场景标签（如：home_conversation, industrial_inspection）。2. 实时清洗日志与过程数据内容：记录数据清洗环节的输入、操作和输出，是数据集“实时清洗”能力的直接体现。包含字段示例：timestamp: 关联的时间戳。cleaning_operation: 清洗操作类型（如：noise_suppression, outlier_rejection, missing_value_interpolation）。pre_state: 操作前的数据质量指标（如信噪比、数值范围）。post_state: 操作后的数据质量指标。cleaning_params: 本次清洗采用的动态参数。3. 时序特征向量序列内容：经过实时清洗和特征提取后生成的、标准化的特征向量，是后续建模的直接输入。包含字段示例：timestamp: 毫秒级时间戳。feature_vector: 多维浮点数数组，代表融合后的特征（如：音频的MFCC特征+图像的HOG特征+振动频谱特征）。feature_dimension: 特征向量的维度描述。4. 实时模型推断结果内容：轻量化模型对实时特征向量进行在线推断的输出，是本数据集“实时建模”的核心成果。包含字段示例：timestamp: 推断对应的时间戳。model_name: 模型标识（如：emotion_classifier_v3, anomaly_detector_motor）。prediction_label: 预测标签（如：happy, normal, warning）。confidence_score: 预测置信度。additional_outputs: 其他模型输出（如：情感维度数值、异常分数、未来状态预测）。5. 驱动指令与反馈（验证闭环数据）内容：将模型推断结果转化为机器人执行指令的数据，以及部分执行反馈，用于验证实时决策的有效性。包含字段示例：timestamp: 指令生成时间戳。action_command: 结构化指令（如：{“expression”: “smile”, “intensity”: 0.8}， {“movement”: “pause”, “reason”: “vibration_alert”}）。triggering_prediction: 触发此指令的模型推断结果ID（用于关联追溯）。应用场景场景一：实时流式机器学习算法研发与基准测试如何应用：算法研发人员可将本数据集作为持续流入的数据流，模拟真实生产环境，开发和测试其在线学习、增量学习或流式异常检测算法。输入：使用数据集中的时序特征向量序列作为核心输入流，并参考实时清洗日志来优化算法对噪声和缺失值的鲁棒性。处理：研发的算法需要实时接收数据流，进行在线推断或模型更新，并输出预测结果。验证：将算法的实时输出与数据集中提供的实时模型推断结果进行比对，评估其准确性、延迟和稳定性。预期效果：研发出能处理非平稳数据分布、低延迟的实时机器学习模型。为流式算法提供一个客观的、基于真实场景的性能基准，客观比较不同算法的优劣。 场景二：多模态融合与跨模态理解模型训练如何应用：计算机视觉与语音领域的研究者可以利用本数据集时间戳严格对齐的特性，训练或验证复杂的多模态融合模型。输入：同时调用同一时间段内的多模态原始感知数据切片（如音频切片与对应的视频切片）及时序特征向量。处理：设计跨模态的注意力机制、特征融合网络或翻译模型，学习不同模态信息间的关联与协同。验证：利用数据集中实时模型推断结果（如融合后的情感标签）作为监督信号，评估模型融合的有效性。预期效果：训练出能更精准理解复杂场景（如“用户笑着说反话”）的下一代多模态AI模型。为工业场景下“声音+振动+热成像”的多传感器融合故障诊断提供训练基础。 场景三：机器人交互策略与闭环控制系统优化如何应用：机器人公司和研究机构可以利用数据集中的“感知-决策-执行”完整闭环数据，优化机器人的交互智能和控制系统。输入：将时序特征向量和实时模型推断结果作为状态输入。处理：应用强化学习、模仿学习或策略搜索方法，训练一个能将当前状态映射到最优动作的“策略网络”。验证：将新策略生成的“动作指令”与数据集中记录的驱动指令进行对比分析，或在仿真环境中以数据集的感知数据为输入，测试新策略的执行效果。预期效果：开发出更自然、更拟人、更高效的机器人交互行为策略。优化工业机器人在复杂动态环境中的实时决策与控制能力，减少停机时间。 场景四：工业预测性维护与异常诊断系统开发如何应用：工业互联网和运维团队可利用数据集中的工业场景流数据，构建和验证预测性维护模型。输入：重点使用振动、电流、温度等传感器的时序特征向量序列及其对应的实时模型推断结果（如“正常”、“警告”标签）。处理：训练时序预测模型（如LSTM, Transformer）来预测设备未来状态，或训练异常检测模型在无明确标签的情况下发现潜在故障模式。验证：利用数据中记录的从“异常预警”到后续状态变化甚至驱动指令（如停机指令）的序列，验证预警的准确性和提前量。预期效果：构建高精度的设备健康状态评估系统，实现从“事后维修”到“事前预防”的转变。显著降低非计划性停机，提升生产安全与效率。 场景五：新产品功能快速原型验证如何应用：产品经理与研发团队在构思一项基于多模态感知的新功能（如“通过表情和语调识别用户满意度”）时，无需立即部署全套硬件和收集数据。输入：直接使用数据集中相关的清洗后的特征数据和标注结果。处理：快速构建一个概念验证模型或规则引擎，验证功能逻辑的可行性。验证：在离线环境下评估功能原型的效果，大幅降低前期试错成本。预期效果：将新产品、新功能的算法验证周期从“数月”缩短至“数周”。在硬件开发完成前，即可完成核心AI算法的迭代与选型。数据范围一、 时间范围本数据集的数据采集工作为连续、不间断的长期过程，确保了数据的时效性、连续性与演进性。采集周期：2023年1月1日 至 2025年6月30日，总历时约30个月。时间特性：数据以连续的时序流形式存在，完整覆盖了工作日、周末、节假日等不同时间模式，能够反映机器人及用户在长期使用中的行为模式演变、季节性差异及算法迭代的痕迹。二、 地理与场景覆盖范围数据来源于部署在真实环境中的机器人集群，覆盖了多样化的地理区域与应用场景。服务陪伴场景：地区覆盖：主要覆盖中国境内多个主要城市及地区的家庭、社区养老中心、幼教机构及医疗机构，包括但不限于华北、华东、华南及中西部的代表性城市。环境类型：涵盖典型城市住宅、社区公共活动空间、标准教室及康复病房等多种室内环境。工业场景：地区与行业覆盖：数据采集自位于长三角、珠三角及中部产业集聚区的合作工厂与研发测试中心。行业类型：覆盖消费电子组装、汽车零部件制造、仓储物流等多个典型行业的示范产线或测试单元。三、 数据主体（用户与设备）范围服务陪伴场景用户：年龄分布：涵盖儿童（3-12岁）、成年人、老年人（60岁以上） 等多个年龄段。（用户描述仅用于定义交互发生的背景场景，而实际采集处理的所有数据均为机器人本体交互与环境感知数据，不含任何能识别特定自然人身份的个人信息。）交互关系：包括个人独立交互、亲子互动及群体互动等多种模式，确保了交互模式的多样性。工业场景操作与运维人员：角色类型：包括产线操作工、设备维护工程师、技术管理人员等。交互模式：涵盖例行巡检、维护操作、异常处理及协同作业等多种专业交互。机器人设备：型号与批次：数据来源于我司多个批次、不同型号的商用机器人与原型测试机，包含了硬件迭代和软件版本更新的多样性。四、 数据规模与采集频率总体规模：数据集原始流数据总量超过PB级，经过本说明所述流程清洗、压缩和结构化处理后，形成的本数据集规模为 5 TB。采集频率：传感器原始数据采集频率从音频的16kHz采样到视觉的30Hz帧率，直至部分工业振动传感器的1kHz高频采样。实时清洗与特征提取过程与采集同步，模型推断与决策生成延迟在毫秒至秒级，确保了全链路的“实时性”。数据量级本数据集总容量约 5 TB，由超过 150 亿条经过清洗与标注的时序数据记录构成。数据采集自 120 台自研机器人，时间跨度覆盖 2023年1月至2025年6月，完整反映了设备在不同季节和工况下的运行状态。
Product Content: This dataset is essentially a set of high-fidelity time-series data stream records, including time-aligned raw data slices (for traceability), corresponding real-time cleaning logs, intermediate feature sequences, and real-time model output labels with millisecond or second-level delay. 1. Dataset Structure This is a time-series data set strictly aligned by timestamps and stored in a structured format, which fully records the entire real-time processing pipeline of a robot from perception to execution. It consists of the following five types of data, forming a traceable and analyzable data pipeline together: 1. Raw Perception Data Stream Slices: Short multi-modal raw data slices (usually continuous windows of 1-5 seconds) that have been time-synchronized, used for traceability and benchmarking. Example fields: - timestamp: Unified millisecond-level timestamp (primary key) - data_source: Data source identifier (e.g., audio_mic_array, rgb_camera, force_sensor_hand, vibration_sensor_motor) - raw_data_payload: Encoded raw data or data storage path pointer - scene_tag: Scene tag (e.g., home_conversation, industrial_inspection) 2. Real-time Cleaning Logs and Process Data: Records the input, operations, and outputs of the data cleaning process, which directly reflects the "real-time cleaning" capability of the dataset. Example fields: - timestamp: Associated timestamp - cleaning_operation: Type of cleaning operation (e.g., noise_suppression, outlier_rejection, missing_value_interpolation) - pre_state: Data quality metrics before the operation (e.g., signal-to-noise ratio, value range) - post_state: Data quality metrics after the operation - cleaning_params: Dynamic parameters used for this cleaning operation 3. Time-series Feature Vector Sequence: Standardized feature vectors generated after real-time cleaning and feature extraction, which serve as the direct input for subsequent modeling. Example fields: - timestamp: Millisecond-level timestamp - feature_vector: Multi-dimensional floating-point array representing fused features (e.g., MFCC features of audio + HOG features of images + vibration spectrum features) - feature_dimension: Dimension description of the feature vector 4. Real-time Model Inference Results: Outputs from lightweight models performing online inference on real-time feature vectors, which are the core outcome of the "real-time modeling" capability of this dataset. Example fields: - timestamp: Timestamp corresponding to the inference - model_name: Model identifier (e.g., emotion_classifier_v3, anomaly_detector_motor) - prediction_label: Prediction label (e.g., happy, normal, warning) - confidence_score: Prediction confidence - additional_outputs: Other model outputs (e.g., emotional dimension values, anomaly scores, future state predictions) 5. Drive Commands and Feedback (Closed-loop Validation Data): Data that converts model inference results into robot execution commands, as well as partial execution feedback, used to verify the effectiveness of real-time decision-making. Example fields: - timestamp: Timestamp when the command was generated - action_command: Structured command (e.g., {"expression": "smile", "intensity": 0.8}, {"movement": "pause", "reason": "vibration_alert"}) - triggering_prediction: ID of the model inference result that triggered this command (for correlation and traceability) ## Application Scenarios Scenario 1: R&D and Benchmark Testing of Real-time Streaming Machine Learning Algorithms How to apply: Algorithm researchers can use this dataset as a continuously incoming data stream to simulate real production environments, develop and test their online learning, incremental learning, or streaming anomaly detection algorithms. Input: Use the time-series feature vector sequence in the dataset as the core input stream, and refer to the real-time cleaning logs to optimize the algorithm's robustness to noise and missing values. Processing: The developed algorithm needs to receive the data stream in real time, perform online inference or model updates, and output prediction results. Validation: Compare the real-time output of the algorithm with the real-time model inference results provided in the dataset to evaluate its accuracy, latency, and stability. Expected outcomes: Develop real-time machine learning models that can handle non-stationary data distributions with low latency. Provide an objective, real-scenario-based performance benchmark for streaming algorithms to objectively compare the pros and cons of different algorithms. Scenario 2: Training of Multimodal Fusion and Cross-modal Understanding Models How to apply: Researchers in computer vision and speech fields can leverage the strict timestamp alignment feature of this dataset to train or validate complex multimodal fusion models. Input: Simultaneously call multi-modal raw perception data slices (e.g., audio slices and corresponding video slices) and time-series feature vectors within the same time period. Processing: Design cross-modal attention mechanisms, feature fusion networks, or translation models to learn the correlation and collaboration between different modal information. Validation: Use the real-time model inference results in the dataset (e.g., fused emotion labels) as supervision signals to evaluate the effectiveness of model fusion. Expected outcomes: Train a new generation of multimodal AI models that can more accurately understand complex scenarios (e.g., "a user says sarcastic things while smiling"). Provide a training foundation for multi-sensor fusion fault diagnosis of "sound + vibration + thermal imaging" in industrial scenarios. Scenario 3: Optimization of Robot Interaction Strategies and Closed-loop Control Systems How to apply: Robot companies and research institutions can use the complete closed-loop data of "perception-decision-execution" in the dataset to optimize the robot's interactive intelligence and control systems. Input: Take the time-series feature vectors and real-time model inference results as state inputs. Processing: Apply reinforcement learning, imitation learning, or policy search methods to train a "policy network" that maps the current state to the optimal action. Validation: Compare and analyze the "action commands" generated by the new policy with the drive commands recorded in the dataset, or test the execution effect of the new policy in a simulation environment using the perception data from the dataset as input. Expected outcomes: Develop more natural, anthropomorphic, and efficient robot interaction behavior strategies. Optimize the real-time decision-making and control capabilities of industrial robots in complex dynamic environments to reduce downtime. Scenario 4: Development of Industrial Predictive Maintenance and Anomaly Diagnosis Systems How to apply: Industrial Internet and operation and maintenance teams can use the industrial scenario stream data in the dataset to build and validate predictive maintenance models. Input: Focus on using time-series feature vector sequences from sensors such as vibration, current, and temperature, along with their corresponding real-time model inference results (e.g., "normal", "warning" labels). Processing: Train time-series prediction models (e.g., LSTM, Transformer) to predict future equipment states, or train anomaly detection models to identify potential fault patterns without explicit labels. Validation: Use the sequence recorded in the data from "abnormal warning" to subsequent state changes and even drive commands (e.g., shutdown commands) to verify the accuracy and lead time of the warning. Expected outcomes: Build a high-precision equipment health status assessment system to realize the transition from "reactive maintenance" to "predictive prevention". Significantly reduce unplanned downtime and improve production safety and efficiency. Scenario 5: Rapid Prototyping Validation of New Product Functions How to apply: Product managers and R&D teams do not need to immediately deploy full hardware and collect data when conceiving new multi-modal perception-based functions (e.g., "recognizing user satisfaction through facial expressions and intonation"). Input: Directly use the cleaned feature data and annotation results related to the dataset. Processing: Rapidly build a proof-of-concept model or rule engine to verify the feasibility of the functional logic. Validation: Evaluate the effect of the functional prototype in an offline environment, greatly reducing early trial-and-error costs. Expected outcomes: Shorten the algorithm validation cycle for new products and functions from "several months" to "several weeks". Complete iteration and selection of core AI algorithms before hardware development is completed. ## Data Scope 1. Time Scope The data collection of this dataset is a continuous, uninterrupted long-term process, ensuring the timeliness, continuity, and evolution of the data. Collection period: January 1, 2023 to June 30, 2025, with a total duration of approximately 30 months. Time characteristics: The data exists as a continuous time-series stream, fully covering different time patterns such as weekdays, weekends, and holidays, and can reflect the evolution of behavioral patterns, seasonal differences, and traces of algorithm iterations of robots and users during long-term use. 2. Geographical and Scene Coverage The data comes from robot clusters deployed in real environments, covering diverse geographical regions and application scenarios. Service Companion Scenarios: Regional coverage: Mainly covers families, community senior care centers, preschool education institutions, and medical institutions in multiple major cities and regions in China, including but not limited to representative cities in North China, East China, South China, and the central and western regions. Environmental types: Covers a variety of indoor environments such as typical urban residences, community public activity spaces, standard classrooms, and rehabilitation wards. Industrial Scenarios: Regional and industry coverage: Data is collected from cooperative factories and R&D test centers located in the Yangtze River Delta, Pearl River Delta, and central industrial agglomeration areas. Industry types: Covers demonstration production lines or test units in multiple typical industries such as consumer electronics assembly, auto parts manufacturing, and warehousing and logistics. 3. Data Subject (Users and Equipment) Scope Service Companion Scenario Users: Age distribution: Covers multiple age groups including children (3-12 years old), adults, and the elderly (over 60 years old). (The user description is only used to define the background scenario where the interaction occurs, and all actually collected and processed data are robot body interaction and environment perception data, without any personal information that can identify a specific natural person.) Interaction modes: Includes multiple modes such as individual independent interaction, parent-child interaction, and group interaction, ensuring the diversity of interaction patterns. Industrial Scenario Operation and Maintenance Personnel: Role types: Include production line operators, equipment maintenance engineers, technical management personnel, etc. Interaction modes: Covers multiple professional interactions such as routine inspections, maintenance operations, anomaly handling, and collaborative work. Robot Equipment: Model and batch: The data comes from multiple batches and different models of commercial robots and prototype test machines from our company, including the diversity of hardware iterations and software version updates. 4. Data Scale and Collection Frequency Overall scale: The total amount of raw stream data in the dataset exceeds the petabyte level. After cleaning, compression, and structured processing according to the process described in this specification, the scale of this dataset is 5 TB. Collection frequency: The original data collection frequency of sensors ranges from 16kHz sampling for audio, 30Hz frame rate for vision, to 1kHz high-frequency sampling for some industrial vibration sensors. The real-time cleaning and feature extraction process is synchronized with collection, and the delay of model inference and decision generation is at the millisecond to second level, ensuring the "real-time performance" of the entire pipeline. Data scale: The total capacity of this dataset is approximately 5 TB, consisting of more than 15 billion cleaned and labeled time-series data records. The data is collected from 120 self-developed robots, with a time span covering January 2023 to June 2025, fully reflecting the operating status of the equipment in different seasons and working conditions.
