society-ethics/BlogPostOpenness

--- license: cc-by-4.0 --- ## Mission: Open and Good ML In our mission to democratize good machine learning (ML), we examine how supporting ML community work also empowers examining and preventing possible harms. Open development and science decentralizes power so that many people can collectively work on AI that reflects their needs and values. While [openness enables broader perspectives to contribute to research and AI overall, it faces the tension of less risk control](https://arxiv.org/abs/2302.04844). Moderating ML artifacts presents unique challenges due to the dynamic and rapidly evolving nature of these systems. In fact, as ML models become more advanced and capable of producing increasingly diverse content, the potential for harmful or unintended outputs grows, necessitating the development of robust moderation and evaluation strategies. Moreover, the complexity of ML models and the vast amounts of data they process exacerbate the challenge of identifying and addressing potential biases and ethical concerns. As hosts, we recognize the responsibility that comes with potentially amplifying harm to our users and the world more broadly. Often these harms disparately impact minority communities in a context-dependent manner. We have taken the approach of analyzing the tensions in play for each context, open to discussion across the company and Hugging Face community. While many models can amplify harm, especially discriminatory content, we are taking a series of steps to identify highest risk models and what action to take. Importantly, active perspectives from many backgrounds is key to understanding, measuring, and mitigating potential harms that affect different groups of people. We are crafting tools and safeguards in addition to improving our documentation practices to ensure open source science empowers individuals and continues to minimize potential harms. ## Ethical Categories The first major aspect of our work to foster good open ML consists in promoting the tools and positive examples of ML development that prioritize values and consideration for its stakeholders. This helps users take concrete steps to address outstanding issues, and present plausible alternatives to de facto damaging practices in ML development. To help our users discover and engage with ethics-related ML work, we have compiled a set of tags. These 6 high-level categories are based on our analysis of Spaces that community members had contributed. They are designed to give you a jargon-free way of thinking about ethical technology: - Rigorous work pays special attention to developing with best practices in mind. In ML, this can mean examining failure cases (including conducting bias and fairness audits), protecting privacy through security measures, and ensuring that potential users (technical and non-technical) are informed about the project's limitations. - Consentful work [supports](https://www.consentfultech.io/) the self-determination of people who use and are affected by these technologies. - Socially Conscious work shows us how technology can support social, environmental, and scientific efforts. - Sustainable work highlights and explores techniques for making machine learning ecologically sustainable. - Inclusive work broadens the scope of who builds and benefits in the machine learning world. - Inquisitive work shines a light on inequities and power structures which challenge the community to rethink its relationship to technology. Read more at https://huggingface.co/ethics Look for these terms as we’ll be using these tags, and updating them based on community contributions, across some new projects on the Hub! ## Safeguards Taking an “all-or-nothing” view of open releases ignores the wide variety of contexts that determine an ML artifact’s positive or negative impacts. Having more levers of control over how ML systems are shared and re-used supports collaborative development and analysis with less risk of promoting harmful uses or misuses; allowing for more openness and participation in innovation for shared benefits. We engage directly with contributors and have addressed pressing issues. To bring this to the next level, we are building community-based processes. This approach empowers both Hugging Face contributors, and those affected by contributions, to inform the limitations, sharing, and additional mechanisms necessary for models and data made available on our platform. The three main aspects we will pay attention to are: the origin of the artifact, how the artifact is handled by its developers, and how the artifact has been used. In that respect we: - launched a [flagging feature](https://twitter.com/GiadaPistilli/status/1571865167092396033) for our community to determine whether ML artifacts or community content (model, dataset, space, or discussion) violate our [content guidelines](https://huggingface.co/content-guidelines), - monitor our community discussion boards to ensure Hub users abide by the [code of conduct](https://huggingface.co/code-of-conduct), - robustly document our most-downloaded models with model cards that detail social impacts, biases, and intended and out-of-scope use cases, - create audience-guiding tags, such as the “Not For All Audiences” tag that can be added to the repository’s card metadata to avoid un-requested violent and sexual content, - promote use of [Open Responsible AI Licenses (RAIL)](https://huggingface.co/blog/open_rail) for [models](https://www.licenses.ai/blog/2022/8/26/bigscience-open-rail-m-license), such as with LLMs ([BLOOM](https://huggingface.co/spaces/bigscience/license), [BigCode](https://huggingface.co/spaces/bigcode/license)), - conduct research that [analyzes](https://arxiv.org/abs/2302.04844) which models and datasets have the highest potential for, or track record of, misuse and malicious use. **How to use the flagging function:** Click on the flag icon on any Model, Dataset, Space, or Discussion: <p align="center"> <br> <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ethics_soc_3/flag2.jpg" alt="screenshot pointing to the flag icon to Report this model" /> </p> Share why you flagged this item: <p align="center"> <br> <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ethics_soc_3/flag1.jpg" alt="screenshot showing the text window where you describe why you flagged this item" /> </p> In prioritizing open science, we examine potential harm on a case-by-case basis. When users flag a system, developers can directly and transparently respond to concerns. Moderators are able to disengage from discussions should behavior become hateful and/or abusive (see [code of conduct](https://huggingface.co/code-of-conduct)). Should a specific model be flagged as high risk by our community, we consider: - Downgrading the ML artifact’s visibility across the Hub in the trending tab and in feeds, - Requesting that the models be made private, - Gating access to ML artifacts (see documentation for [models](https://huggingface.co/docs/hub/models-gated) and [datasets](https://huggingface.co/docs/hub/datasets-gated)), - Disabling access. **How to add the “Not For All Audiences” tag:** Edit the model/data card → add `not-for-all-audiences` in the tags section → open the PR and wait for the authors to merge it. Once merged, the following tag will be displayed on the repository: <p align="center"> <br> <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ethics_soc_3/nfaa_tag.png" alt="screenshot showing where to add tags" /> </p> Any repository tagged `not-for-all-audiences` will display the following popup when visited: <p align="center"> <br> <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ethics_soc_3/nfaa2.png" alt="screenshot showing where to add tags" /> </p> Clicking "View Content" will allow you to view the repository as normal. If you wish to always view `not-for-all-audiences`-tagged repositories without the popup, this setting can be changed in a user's [Content Preferences](https://huggingface.co/settings/content-preferences) <p align="center"> <br> <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ethics_soc_3/nfaa1.png" alt="screenshot showing where to add tags" /> </p> Open science requires safeguards, and one of our goals is to create an environment informed by tradeoffs with different values. Hosting and providing access to models in addition to cultivating community and discussion empowers diverse groups to assess social implications and guide what is good machine learning. ## Are you working on safeguards? Share them on Hugging Face Hub! The most important part of Hugging Face is our community. If you’re a researcher working on making ML safer to use, especially for open science, we want to support and showcase your work! Here are some recent demos and tools from researchers in the Hugging Face community: - [A Watermark for LLMs](https://huggingface.co/spaces/tomg-group-umd/lm-watermarking) by John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, Tom Goldstein ([paper](https://arxiv.org/abs/2301.10226)) - [Generate Model Cards Tool](https://huggingface.co/spaces/huggingface/Model_Cards_Writing_Tool) by the Hugging Face team - [Photoguard](https://huggingface.co/spaces/RamAnanth1/photoguard) to safeguard images against manipulation by Ram Ananth Thanks for reading! 🤗 ~ Irene, Nima, Giada, Yacine, and Elizabeth, on behalf of the Ethics and Society regulars If you want to cite this blog post, please use the following: ``` @misc{hf_ethics_soc_blog_3, author = {Irene Solaiman and Giada Pistilli and Nima Boscarino and Yacine Jernite and Elizabeth Allendorf and Margaret Mitchell and Carlos Muñoz Ferrandis and Nathan Lambert and Alexandra Sasha Luccioni }, title = {Hugging Face Ethics and Society Newsletter 3: Ethical Openness at Hugging Face}, booktitle = {Hugging Face Blog}, year = {2023}, url = {https://doi.org/10.57967/hf/0487}, doi = {10.57967/hf/0487} } ```--- 许可证：CC BY 4.0 --- ## 使命：开放与优质机器学习 在我们普及优质机器学习（Machine Learning，以下简称ML）的使命中，我们探讨了支持ML社区工作如何同时赋能对潜在危害的审视与防范。开放开发与开放科学能够去中心化权力，使更多人能够协同开发符合自身需求与价值的人工智能系统。尽管[开放性为研究与人工智能整体发展带来了更广泛的视角贡献](https://arxiv.org/abs/2302.04844)，但也面临着风险管控能力减弱的矛盾。 由于ML系统具有动态且快速演进的特性，对ML制品的审核面临独特挑战。事实上，随着ML模型愈发先进，能够生成日益多样化的内容，产生有害或非预期输出的可能性也随之提升，这就需要开发稳健的审核与评估策略。此外，ML模型的复杂性及其处理的海量数据，加剧了识别与解决潜在偏见及伦理问题的难度。 作为平台主办方，我们认识到自身可能放大对用户乃至整个世界的危害所带来的责任。此类危害往往会根据具体场景，对少数群体产生不成比例的影响。我们的做法是，针对每种场景分析其中存在的矛盾，欢迎公司内部与Hugging Face社区展开讨论。尽管许多模型可能放大危害，尤其是歧视性内容，但我们正采取一系列措施，以识别高风险模型并确定应对方案。至关重要的是，来自不同背景的积极参与，是理解、衡量并缓解影响不同群体的潜在危害的关键。 我们正在开发工具与防护机制，同时优化文档实践，以确保开放科学能够赋能个体，并持续尽可能降低潜在危害。 ## 伦理类别 我们培育优质开放ML的工作的核心方向之一，是推广优先考量其利益相关方的价值与关切的ML开发工具与优质案例。这有助于用户采取具体措施解决现存问题，并为ML开发中事实上存在的有害实践提供可行的替代方案。 为帮助用户发现并参与与伦理相关的ML工作，我们编制了一套标签体系。这6个高级类别基于我们对社区成员贡献的Space的分析得出，旨在为用户提供一种无需专业术语即可思考伦理技术问题的方式： - 严谨实践类：在开发过程中特别注重遵循最佳实践。在ML领域，这意味着审视失败案例（包括开展偏见与公平性审核）、通过安全措施保护隐私，并确保潜在用户（技术与非技术人员）了解项目的局限性。 - 知情同意类：[支持](https://www.consentfultech.io/)使用受影响技术的人群的自决权。 - 社会关怀类：展示技术如何支持社会、环境与科学事业。 - 可持续发展类：强调并探索使机器学习具备生态可持续性的技术。 - 包容参与类：拓展机器学习领域的开发者与受益群体范围。 - 反思探究类：揭示不平等现象与权力结构，推动社区重新思考其与技术的关系。 可在https://huggingface.co/ethics 查看更多内容。 我们将在Hub的部分新项目中使用这些标签，并根据社区贡献进行更新，敬请留意相关术语。 ## 防护机制 对开放发布采取“非全即无”的态度，会忽略决定ML制品积极或消极影响的多样化场景。对ML系统的共享与复用方式拥有更多控制手段，能够支持协作开发与分析，同时降低推广有害使用或滥用的风险，从而为共享收益实现更开放的创新与参与。 我们直接与贡献者互动，并已解决了紧迫问题。为将此项工作推向新高度，我们正在构建基于社区的流程。这种方式既能赋能Hugging Face贡献者，也能赋能受贡献影响的群体，以告知我们在平台上发布的模型与数据所需的限制、共享方式及额外机制。我们将重点关注三个核心方面：制品的来源、开发者对制品的处理方式，以及制品的使用方式。为此，我们： - 推出了[举报功能](https://twitter.com/GiadaPistilli/status/1571865167092396033)，供社区判断ML制品或社区内容（模型、数据集、Space或讨论）是否违反我们的[内容准则](https://huggingface.co/content-guidelines)， - 监控社区讨论板块，确保Hub用户遵守[行为准则](https://huggingface.co/code-of-conduct)， - 为下载量最高的模型编写详细的模型卡片（Model Cards），其中包含社会影响、偏见、预期使用场景与非预期使用场景的说明， - 创建受众引导标签，例如可添加至仓库卡片元数据的“非全年龄段适宜”标签，以避免未获预期的暴力与色情内容， - 推广[开放负责任人工智能许可证（Open Responsible AI Licenses，RAIL）](https://huggingface.co/blog/open_rail)在[模型](https://www.licenses.ai/blog/2022/8/26/bigscience-open-rail-m-license)中的应用，例如在大语言模型（Large Language Model，LLM）[BLOOM](https://huggingface.co/spaces/bigscience/license)与[BigCode](https://huggingface.co/spaces/bigcode/license)中， - 开展研究，[分析](https://arxiv.org/abs/2302.04844)哪些模型与数据集具有最高的滥用或恶意滥用潜力，或存在滥用或恶意滥用的记录。 **如何使用举报功能：** 点击任意模型、数据集、Space或讨论页面上的举报图标： <p align="center"> <br> <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ethics_soc_3/flag2.jpg" alt="指向举报图标的截图，用于举报该模型" /> </p> 请说明举报该内容的原因： <p align="center"> <br> <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ethics_soc_3/flag1.jpg" alt="显示举报原因填写文本框的截图" /> </p> 在优先推进开放科学的过程中，我们会逐案审视潜在危害。当用户举报某系统时，开发者可以直接且透明地回应相关关切。若讨论内容出现仇恨或辱骂行为，审核员可终止该讨论（详见[行为准则](https://huggingface.co/code-of-conduct)）。 若某模型被社区标记为高风险，我们将考虑： - 降低该ML制品在Hub的热门标签页与信息流中的可见性， - 要求将模型设为私有， - 对ML制品的访问设置权限（详见[模型](https://huggingface.co/docs/hub/models-gated)与[数据集](https://huggingface.co/docs/hub/datasets-gated)的文档）， - 禁用访问权限。 **如何添加“非全年龄段适宜”标签：** 编辑模型/数据集卡片 → 在标签部分添加`not-for-all-audiences` → 提交合并请求（PR）并等待作者合并。合并后，仓库将显示以下标签： <p align="center"> <br> <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ethics_soc_3/nfaa_tag.png" alt="显示标签添加位置的截图" /> </p> 所有标记为`not-for-all-audiences`的仓库在访问时将显示以下弹窗： <p align="center"> <br> <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ethics_soc_3/nfaa2.png" alt="显示弹窗的截图" /> </p> 点击“查看内容”即可正常访问仓库。若您希望始终无需弹窗即可查看带`not-for-all-audiences`标签的仓库，可在用户的[内容偏好设置](https://huggingface.co/settings/content-preferences)中更改相关设置。 <p align="center"> <br> <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/ethics_soc_3/nfaa1.png" alt="显示内容偏好设置位置的截图" /> </p> 开放科学需要防护机制，我们的目标之一是打造一个兼顾不同价值权衡的环境。托管并提供模型访问权限，同时培育社区与讨论，能够赋能不同群体评估社会影响，并界定何为优质机器学习。 ## 您是否正在开发防护机制？请在Hugging Face Hub上分享！ Hugging Face最重要的组成部分是我们的社区。如果您是致力于提升ML使用安全性的研究人员，尤其是针对开放科学场景，我们希望支持并展示您的工作！ 以下是Hugging Face社区研究人员近期推出的一些演示与工具： - [大语言模型水印](https://huggingface.co/spaces/tomg-group-umd/lm-watermarking)，由John Kirchenbauer、Jonas Geiping、Yuxin Wen、Jonathan Katz、Ian Miers、Tom Goldstein开发（[论文](https://arxiv.org/abs/2301.10226)） - [模型卡片编写工具](https://huggingface.co/spaces/huggingface/Model_Cards_Writing_Tool)，由Hugging Face团队开发 - [Photoguard](https://huggingface.co/spaces/RamAnanth1/photoguard)，用于防范图像被篡改，由Ram Ananth开发 感谢阅读！🤗 来自伦理与社会团队的Irene、Nima、Giada、Yacine与Elizabeth 若您希望引用此博客文章，请使用以下格式： @misc{hf_ethics_soc_blog_3, author = {Irene Solaiman and Giada Pistilli and Nima Boscarino and Yacine Jernite and Elizabeth Allendorf and Margaret Mitchell and Carlos Muñoz Ferrandis and Nathan Lambert and Alexandra Sasha Luccioni }, title = {Hugging Face Ethics and Society Newsletter 3: Ethical Openness at Hugging Face}, booktitle = {Hugging Face Blog}, year = {2023}, url = {https://doi.org/10.57967/hf/0487}, doi = {10.57967/hf/0487} }

原始信息汇总

数据集概述

许可证

• 该数据集遵循 cc-by-4.0 许可证。

伦理分类

• 数据集包含六个高级别的伦理分类标签，旨在帮助用户发现和参与与伦理相关的机器学习工作：
  • 严谨的工作：注重最佳实践，包括审查失败案例、保护隐私和确保用户了解项目限制。
  • 同意的工作：支持使用和受这些技术影响的人的自决。
  • 社会意识的工作：展示技术如何支持社会、环境和科学努力。
  • 可持续的工作：强调和探索使机器学习生态可持续的技术。
  • 包容的工作：扩大机器学习世界中构建者和受益者的范围。
  • 好奇的工作：揭示不平等和权力结构，挑战社区重新思考与技术的关系。

安全措施

• 数据集提供了多种安全措施，包括：
  • 标记功能：允许社区成员标记违反内容指南的机器学习制品或社区内容。
  • 监控社区讨论：确保用户遵守行为准则。
  • 模型卡片：详细记录最受欢迎模型的社会影响、偏见和使用范围。
  • 受众导向标签：如“不适合所有观众”标签，用于避免未经请求的暴力和色情内容。
  • 开放负责任AI许可证（RAIL）：推广使用RAIL许可证，如LLMs（BLOOM、BigCode）。
  • 研究分析：分析哪些模型和数据集具有最高的滥用或恶意使用潜力。

标记功能使用方法

• 点击任何模型、数据集、空间或讨论中的标记图标：
  • 分享标记原因：在文本窗口中描述标记原因。

“不适合所有观众”标签使用方法

• 编辑模型/数据卡片 → 在标签部分添加 not-for-all-audiences → 提交PR并等待作者合并。

高风险模型处理

• 如果模型被社区标记为高风险，可能采取以下措施：
  • 降低模型在热门标签和动态中的可见性。
  • 请求将模型设为私有。
  • 限制对模型的访问。
  • 禁用访问。

搜集汇总

数据集介绍

构建方式

该数据集以Hugging Face平台上的社区贡献为基石，通过系统分析用户创建的Spaces，提炼出六项核心伦理类别，涵盖严谨性、知情同意、社会意识、可持续性、包容性与探究精神。这些类别旨在以通俗易懂的术语引导用户思考伦理技术，并基于社区反馈动态更新。数据集构建过程中，团队深入审视开放科学与风险管控之间的张力，强调在具体情境中评估机器学习工件可能带来的危害，尤其关注对少数群体的差异性影响。通过整合社区标记、内容指南和开源协议等机制，数据集逐步形成了一套兼顾开放性与安全性的治理框架。

特点

该数据集的核心特色在于其动态的伦理分类体系与多层次的安全保障机制。六项伦理标签不仅覆盖了从技术严谨性到社会公平的广泛维度，还通过社区贡献持续演进，体现了开放治理的包容性。数据集特别注重对高风险工件的识别与干预，引入“非全年龄适宜”标签、模型降级、访问限制等阶梯式措施，平衡开放共享与风险控制。此外，数据集强调上下文依赖的危害评估，支持用户通过标记功能直接参与治理，形成开发者与社区协同的反馈闭环，从而在促进创新的同时防范歧视性内容的扩散。

使用方法

用户可通过Hugging Face Hub直接访问该数据集，并利用其内置的伦理标签对模型、数据集或Spaces进行标注与分类。使用过程中，若发现潜在有害内容，可点击仓库页面上的标记图标，提交违规理由以触发审核流程。数据集还提供了“非全年龄适宜”标签的添加指南，用户可通过编辑数据卡片元数据中的tags字段（如添加not-for-all-audiences）来标识敏感内容，合并PR后该标签将自动显示并触发访问确认弹窗。对于高风险工件，平台会依据社区反馈执行降级可见性、限制访问或设为私有等操作，用户可在个人设置中调整内容偏好以控制此类内容的展示方式。

背景与挑战

背景概述

在开放科学与人工智能快速发展的时代背景下，Hugging Face团队（由Irene Solaiman、Giada Pistilli、Nima Boscarino等研究人员主导）于2023年发布了BlogPostOpenness数据集，旨在探讨开放机器学习（Open ML）中的伦理张力与治理框架。该数据集聚焦于一个核心研究问题：如何在推动模型与数据民主化的同时，有效识别并减轻可能对少数群体产生不成比例影响的潜在危害。通过分析社区贡献的Spaces案例，数据集提炼出严谨性、知情同意、社会意识、可持续性、包容性与探究性六大伦理类别，为评估开放科学中的风险与价值提供了系统性视角。该工作对人工智能治理领域产生了深远影响，尤其促进了开源社区在内容审核、模型文档化与责任许可方面的标准化实践。

当前挑战

数据集所面临的挑战体现在多个层面：在领域问题层面，开放机器学习需应对动态演进的模型系统带来的独特审核难题——随着模型能力增强，其产生有害或意外输出的可能性同步增长，且复杂模型处理的海量数据加剧了偏见与伦理风险的识别难度，尤其对不同背景的社群影响存在情境依赖性。在构建过程中，团队需在开放性与风险控制之间取得平衡，例如设计“非全有或全无”的发布策略，通过分级可见性、门控访问及社区举报功能等机制，对高风险模型实施降权、私有化或禁用等差异化管控。此外，如何确保社区驱动的审核流程（如标记功能）既透明又避免滥用，仍是持续迭代中的核心挑战。

常用场景

经典使用场景

BlogPostOpenness数据集的核心用途在于系统性地剖析开放机器学习生态中的伦理张力与治理策略。研究者可借助该数据集深入分析开源模型与数据集的潜在风险，例如有害内容生成、偏见放大及对少数群体的差异化冲击。通过整合社区标记、内容审核机制和开放许可（如RAIL）的实践案例，该数据集为评估开放科学中的风险控制杠杆提供了实证基础，推动对模型共享、使用限制及责任归属的量化研究。

实际应用

在实际部署中，该数据集支撑了Hugging Face平台的社区治理工具链，包括违规内容标记、模型可见度降级及门控访问等机制的优化。开发者可基于数据集的伦理标签识别高风险模型，并动态调整共享策略。此外，其‘非全年龄段’标签与内容偏好设置功能，已被广泛应用于避免暴力或色情内容的不当传播，为多利益相关方参与的协作式AI治理提供了可复用的基础设施。

衍生相关工作

该数据集催生了多项开创性工作，包括LLM水印技术（如Kirchenbauer等人的文本水印方案）用于追溯模型输出源头，以及Photoguard等图像防篡改工具。同时，基于其伦理分类框架，衍生出自动化模型卡生成工具与开放RAIL许可的标准化实践。这些工作共同推动了从‘事后问责’到‘事前预防’的治理范式转变，并在BLOOM、BigCode等大型开源项目中得到应用验证。