大数据运维异常任务识别工具

本组件基于HQL运行数据分析，动态识别跟踪和处置Hadoop平台异常HiveHqI任务。组件提供了包括：三类异常任务(超长运行类JOB、数据倾斜类J0B、大资源占用类JOB)和两类异常运行任务(非当前账期刷数据JOB、重点时段临时J0B的查杀能力)的动态跟踪处理能力。

This component conducts runtime data analysis based on HQL, and dynamically identifies, tracks, and handles abnormal HiveQL tasks on the Hadoop platform. It provides dynamic tracking and processing capabilities covering three categories of abnormal tasks and two categories of abnormal running jobs: the three categories are ultra-long-running jobs, data skew jobs, and high-resource-occupancy jobs; the two categories are data refresh jobs executed outside the current billing cycle and temporary jobs during peak hours, with the capability to detect and terminate such abnormal tasks.

搜集汇总

背景与挑战

背景概述

该数据集是一个用于识别和处置Hadoop平台上异常HiveHQL任务的工具，主要针对三类异常任务和两类异常运行任务。适用于智慧城市场景，由中国电信股份有限公司湖南分公司提供。