Metaplane Data Observability Platform
收藏资源简介：
## Overview Metaplane's native app for Snowflake brings end-to-end data observability directly into your Snowflake environment. Using machine learning, we help data teams prevent, detect, and resolve data quality issues before they impact critical business initiatives like AI/ML projects. Running directly within your Snowflake environment, our app ensures data security while providing end-to-end visibility across your data stack. You can get access to Metaplane under your existing Snowflake subscription/contract. <p><br/></p> ## Key features - Automated ML-powered monitoring that learns your data patterns to detect anomalies - Schema change detection to identify potential downstream impacts - Column-level lineage tracking from source to BI tools - Secure processing within your Snowflake environment through Snowpark Container Services - Integration with popular tools like dbt, Fivetran, and leading BI platforms <p><br/></p> ## Workflow **Installation & setup** - Download Metaplane's native app from Snowflake Marketplace - Launch the application, then follow the easy guided setup process to connect with Metaplane and configure the databases and schemas you wish to monitor <p><br/></p> **Initial Configuration** - The app automatically begins collecting metadata about your Snowflake environment - Metaplane's ML models start learning your data patterns to establish baselines - Set up alert routing to your preferred communication channels (Slack, MS Teams, email) - Define any custom monitoring rules or thresholds (optional) <p><br/></p> **Ongoing Operation** The app runs an hourly sync job within your Snowflake environment using Snowpark Container Services to: - Collect warehouse metadata - Parse query history for lineage analysis - Execute configured monitors to check for anomalies using machine learning - Access the Metaplane UI to view monitoring results, investigate issues, and manage alerts - Receive real-time notifications when anomalies or schema changes are detected <p><br/></p> **Results & Outputs** - Real-time alerts for data quality issues - Column-level lineage visualizations - Schema change notifications with impact analysis - Incident management and resolution tracking <p><br/></p> ## **Sources** - Warehouse metadata from Snowflake's information schema - Query history for lineage analysis - Custom monitoring configurations defined by users - Integration metadata from connected tools in your data stack <p><br/></p> The app is designed for data teams who need best-in-class observability without compromising on security or performance. Whether you're supporting critical AI initiatives, maintaining compliance requirements, or simply aiming to build trust in your data, Metaplane's native app provides the visibility and control you need—all within your secure Snowflake environment. <p><br/></p>
### 概览 Metaplane 面向 Snowflake 的原生应用可将端到端数据可观测性直接集成至您的 Snowflake 环境中。我们借助机器学习技术，协助数据团队在数据质量问题影响 AI/ML 项目等关键业务举措之前，完成问题的预防、检测与修复。本应用直接运行于您的 Snowflake 环境内，既可保障数据安全，又能为您的全数据栈提供端到端可视性。您可通过现有 Snowflake 订阅或合同获取 Metaplane 服务。 ### 核心特性 - 基于机器学习的自动化监控：可学习您的数据模式以检测异常 - 模式变更检测：可识别潜在的下游影响 - 列级血缘追踪：支持从数据源到商业智能（Business Intelligence, BI）工具的全链路追踪 - 基于 Snowpark 容器服务（Snowpark Container Services）的本地安全处理：在您的 Snowflake 环境内完成安全的数据处理 - 与主流工具集成：支持 dbt、Fivetran 等工具及头部商业智能平台的对接 ### 工作流程 #### 安装与设置 - 从 Snowflake 应用商店下载 Metaplane 原生应用 - 启动应用后，按照简易的引导式配置流程连接 Metaplane，并配置您需要监控的数据库与模式 #### 初始配置 - 应用将自动开始收集您的 Snowflake 环境元数据 - Metaplane 的机器学习模型将开始学习您的数据模式以建立基准基线 - 设置告警路由至您偏好的沟通渠道（Slack、微软 Teams、电子邮件） - 可自定义监控规则或阈值（可选） #### 持续运行 本应用将通过 Snowpark 容器服务（Snowpark Container Services）在您的 Snowflake 环境内执行每小时同步任务，以完成以下操作： - 收集数据仓库元数据 - 解析查询历史以进行血缘分析 - 执行已配置的监控任务，借助机器学习检测异常 - 访问 Metaplane 用户界面以查看监控结果、排查问题并管理告警 - 在检测到异常或模式变更时接收实时通知 #### 结果与输出 - 数据质量问题实时告警 - 列级血缘可视化图表 - 附带影响分析的模式变更通知 - 事件管理与故障追踪 #### 数据来源 - 来自 Snowflake 信息架构的数据仓库元数据 - 用于血缘分析的查询历史 - 用户自定义的监控配置 - 来自数据栈中已连接工具的集成元数据 本应用专为需要一流可观测能力且不牺牲安全性与性能的数据团队打造。无论您是支撑关键 AI 项目、满足合规要求，还是仅希望提升数据可信度，Metaplane 原生应用均可在安全的 Snowflake 环境内为您提供所需的可视性与管控能力。
