Amocy-Wang/FEA-Bench

--- # For reference on dataset card metadata, see the spec: https://github.com/huggingface/hub-docs/blob/main/datasetcard.md?plain=1 # Doc / guide: https://huggingface.co/docs/hub/datasets-cards license: other license_name: other license_link: LICENSE dataset_info: features: - name: instance_id dtype: string - name: pull_number dtype: int64 - name: repo dtype: string - name: version dtype: string - name: base_commit dtype: string - name: environment_setup_commit dtype: string - name: created_at dtype: string - name: FAIL_TO_PASS sequence: string - name: PASS_TO_PASS sequence: string splits: - name: test num_bytes: 9002054 num_examples: 1285 download_size: 1588592 dataset_size: 9002054 configs: - config_name: default data_files: - split: test path: data/test-* --- # Dataset Card for FEA-Bench <!-- Provide a quick summary of the dataset. --> A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation. ## Dataset Details ### Dataset Description <!-- Provide a longer summary of what this dataset is. --> The FEA-Bench is a benchmark with a test set that contains 1,401 task instances from 83 Github repositories. This benchmark aims to evaluate the capabilities of repository-level incremental code development. The task instances are collected from Github pull requests, which have the purpose of new feature implementation. Each task instance includes the repo and the base commit sha256, and the PR number and the status of unit test. - **Curated by:** the authors of the FEA-Bench paper: Wei Li, Xin Zhang, Zhongxin Guo, Shaoguang Mao and their collaborators. - **Language(s) (NLP):** English - **License:** Others; We list all licenses of involved github repositories in the last part. <!-- - **Funded by [optional]:** {{ funded_by | default("[More Information Needed]", true)}} - **Shared by [optional]:** {{ shared_by | default("[More Information Needed]", true)}} --> <!-- ### Dataset Sources [optional] --> <!-- Provide the basic links for the dataset. --> <!-- - **Repository:** {{ repo | default("[More Information Needed]", true)}} - **Paper [optional]:** {{ paper | default("[More Information Needed]", true)}} - **Demo [optional]:** {{ demo | default("[More Information Needed]", true)}} --> ## Uses <!-- Address questions around how the dataset is intended to be used. --> This dataset is designed to evaluate performances of LLMs on repository-level code development, which is a complicated software engineering task. - Repository-level incremental code development: The FEA-Bench can be used to evaluate a model for the the capabilities of repository-level incremental code development. Success on this task is typically measured by achieving a high/low resolved ratio. The leaderboard will soon be published as a website. ### Direct Use <!-- This section describes suitable use cases for the dataset. --> Use scripts from FEA-Bench repo to get info for task instances and organize them into prompt, which can be used to LLMs' inference. Also, you can get info or use agents to directly solve the PRs with code changes. ### Out-of-Scope Use <!-- This section addresses misuse, malicious use, and uses that the dataset will not work well for. --> This dataset is not aimed at training for LLMs. You should not take the FEA-Bench as the training dataset to avoid contamination. ## Dataset Structure <!-- This section provides a description of the dataset fields, and additional information about the dataset structure such as criteria used to create the splits, relationships between data points, etc. --> An example: ``` { "instance_id": "huggingface__accelerate-270", "pull_number": 270, "repo": "huggingface/accelerate", "version": null, "base_commit": "515fcca9ed2b36c274c595dbdff75f1c2da635de", "environment_setup_commit": "08101b9dde2b1a9658c2e363e3e9f5663ba06073", "FAIL_TO_PASS": [ "tests/test_state_checkpointing.py::CheckpointTest::test_can_resume_training", "tests/test_state_checkpointing.py::CheckpointTest::test_invalid_registration", "tests/test_state_checkpointing.py::CheckpointTest::test_with_scheduler" ], "PASS_TO_PASS": [] } ``` ## Dataset Creation ### Curation Rationale <!-- Motivation for the creation of this dataset. --> Implementing new features in repository-level codebases is a crucial application of code generation models. However, current benchmarks lack a dedicated evaluation framework for this capability. To fill this gap, we introduce FEA-Bench, a benchmark designed to assess the ability of large language models (LLMs) to perform incremental development within code repositories. ### Source Data <!-- This section describes the source data (e.g. news text and headlines, social media posts, translated sentences, ...). --> #### Data Collection and Processing <!-- This section describes the data collection and processing process such as data selection criteria, filtering and normalization methods, tools and libraries used, etc. --> We collect pull requests from 83 GitHub repositories and use rule-based and intent-based filtering to construct task instances focused on new feature development. Each task instance containing code changes is paired with relevant unit test files to ensure that the solution can be verified. #### Who are the source data producers? <!-- This section describes the people or systems who originally created the data. It should also include self-reported demographic or identity information for the source data creators if this information is available. --> Authors of 83 Github repositories list in the last part. <!-- ### Annotations [optional] --> <!-- If the dataset contains annotations which are not part of the initial data collection, use this section to describe them. --> <!-- #### Annotation process --> <!-- This section describes the annotation process such as annotation tools used in the process, the amount of data annotated, annotation guidelines provided to the annotators, interannotator statistics, annotation validation, etc. --> <!-- {{ annotation_process_section | default("[More Information Needed]", true)}} --> <!-- #### Who are the annotators? --> <!-- This section describes the people or systems who created the annotations. --> <!-- {{ who_are_annotators_section | default("[More Information Needed]", true)}} --> #### Personal and Sensitive Information <!-- State whether the dataset contains data that might be considered personal, sensitive, or private (e.g., data that reveals addresses, uniquely identifiable names or aliases, racial or ethnic origins, sexual orientations, religious beliefs, political opinions, financial or health data, etc.). If efforts were made to anonymize the data, describe the anonymization process. --> The dataset does not include any personal or sensitive information. ## Bias, Risks, and Limitations <!-- This section is meant to convey both technical and sociotechnical limitations. --> - The quantity of high-quality data suitable for repository-level incremental development is limited. High-quality and usable pull requests for new feature development are relatively scarce. Many repository-level code developments for implementing new functionalities were committed during the early stages of repositories, without going through the rigorous code review process typical of the open-source community, resulting in lower data quality that cannot be utilized. - Furthermore, the software's early-stage developments might not even have been conducted using the GitHub platform, posing a challenge for data collection and utilization. - The repository-level incremental code development may not just include new feature implementation tasks. - Only Python repositories are involved in FEA-Bench. - The inference results of the task instances from the benchmark may contain code that is harmful to computer systems. ### Recommendations <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. --> Evaluation by docker is recommended, just like SWE-bench. We will also publish a patch for SWE-bench to make it compatible for our tasks' evaluation. <!-- ## Citation [optional] --> <!-- If there is a paper or blog post introducing the dataset, the APA and Bibtex information for that should go in this section. --> **BibTeX:** To be appeared after publishing the ArXiv paper. **APA:** To be appeared after publishing the ArXiv paper. ## Dataset Card Contact For further information or questions, please contact Xin Zhang ([e-mail]). ## All involved Github repositories in the FEA-Bench | Repo Name | License | Topic | |-----------------------------------|-------------------|--------------------------------------------| | astropy/astropy | BSD-3-Clause | Scientific/Engineering::Astronomy | | django/django | BSD-3-Clause | Internet::WWW/HTTP | | matplotlib/matplotlib | Other | Scientific/Engineering::Visualization | | mwaskom/seaborn | BSD-3-Clause | Scientific/Engineering::Visualization | | pallets/flask | BSD-3-Clause | Internet::WWW/HTTP | | pvlib/pvlib-python | BSD-3-Clause | Scientific/Engineering::Physics | | pydata/xarray | Apache-2.0 | Scientific/Engineering::Information Analysis | | pydicom/pydicom | Others | Scientific/Engineering::Medical Science Apps. | | pylint-dev/astroid | LGPL-2.1 | Software Development::Libraries | | pylint-dev/pylint | GPL-2.0 | Software Development::Quality Assurance | | pyvista/pyvista | MIT | Scientific/Engineering::Information Analysis | | scikit-learn/scikit-learn | BSD-3-Clause | Scientific/Engineering::Artificial Intelligence | | sphinx-doc/sphinx | BSD-2-Clause | Text Processing::Markup | | sqlfluff/sqlfluff | MIT | Software Development::Quality Assurance | | sympy/sympy | Others | Scientific/Engineering::Mathematics | | Aider-AI/aider | Apache-2.0 | Software Development::Code Generators | | Cog-Creators/Red-DiscordBot | GPL-3.0 | Communications::Chat | | DLR-RM/stable-baselines3 | MIT | Scientific/Engineering::Artificial Intelligence | | EleutherAI/lm-evaluation-harness | MIT | Scientific/Engineering::Artificial Intelligence | | Project-MONAI/MONAI | Apache-2.0 | Scientific/Engineering::Medical Science Apps. | | PyThaiNLP/pythainlp | Apache-2.0 | Text Processing::Linguistic | | RDFLib/rdflib | BSD-3-Clause | Software Development::Libraries | | Textualize/rich | MIT | Software Development::Libraries | | Textualize/textual | MIT | Software Development::User Interfaces | | TileDB-Inc/TileDB-Py | MIT | Software Development::Libraries | | astronomer/astronomer-cosmos | Apache-2.0 | Software Development::Build Tools | | atlassian-api/atlassian-python-api| Apache-2.0 | Internet::WWW/HTTP | | aws-cloudformation/cfn-lint | MIT-0 | Software Development::Quality Assurance | | aws-powertools/powertools-lambda-python | MIT-0 | Software Development::Libraries | | aws/sagemaker-python-sdk | Apache-2.0 | Scientific/Engineering::Artificial Intelligence | | biopragmatics/bioregistry | MIT | Scientific/Engineering::Bio-Informatics | | boto/boto3 | Apache-2.0 | Software Development::Libraries | | boto/botocore | Apache-2.0 | Software Development::Libraries | | cocotb/cocotb | BSD-3-Clause | Scientific/Engineering::Electronic Design Automation (EDA) | | conan-io/conan | MIT | Software Development::Build Tools | | deepset-ai/haystack | Apache-2.0 | Scientific/Engineering::Artificial Intelligence | | docker/docker-py | Apache-2.0 | Software Development::Libraries | | dpkp/kafka-python | Apache-2.0 | Software Development::Libraries | | embeddings-benchmark/mteb | Apache-2.0 | Scientific/Engineering::Artificial Intelligence | | facebookresearch/hydra | MIT | Software Development::Libraries | | fairlearn/fairlearn | MIT | Scientific/Engineering::Artificial Intelligence | | falconry/falcon | Apache-2.0 | Internet::WWW/HTTP | | google-deepmind/optax | Apache-2.0 | Scientific/Engineering::Artificial Intelligence | | googleapis/python-aiplatform | Apache-2.0 | Scientific/Engineering::Artificial Intelligence | | googleapis/python-bigquery | Apache-2.0 | Internet::WWW/HTTP | | gradio-app/gradio | Apache-2.0 | Scientific/Engineering::Human Machine Interfaces | | graphql-python/graphene | MIT | Software Development::Libraries | | huggingface/accelerate | Apache-2.0 | Scientific/Engineering::Artificial Intelligence | | huggingface/datasets | Apache-2.0 | Scientific/Engineering::Artificial Intelligence | | huggingface/huggingface_hub | Apache-2.0 | Scientific/Engineering::Artificial Intelligence | | huggingface/pytorch-image-models | Apache-2.0 | Software Development::Libraries | | huggingface/trl | Apache-2.0 | Scientific/Engineering::Artificial Intelligence | | joblib/joblib | BSD-3-Clause | Software Development::Libraries | | joke2k/faker | MIT | Software Development::Testing | | lark-parser/lark | MIT | Text Processing::Linguistic | | minio/minio-py | Apache-2.0 | Software Development::Libraries | | open-mmlab/mmengine | Apache-2.0 | Utilities | | openvinotoolkit/datumaro | MIT | Scientific/Engineering::Image Processing | | pgmpy/pgmpy | MIT | Scientific/Engineering::Artificial Intelligence | | pre-commit/pre-commit | MIT | Software Development::Quality Assurance | | prometheus/client_python | Apache-2.0 | System::Monitoring | | prompt-toolkit/python-prompt-toolkit | BSD-3-Clause | Software Development::User Interfaces | | pygments/pygments | BSD-2-Clause | Software Development::Documentation | | pyocd/pyOCD | Apache-2.0 | Software Development::Debuggers | | pypa/hatch | MIT | Software Development::Build Tools | | pyro-ppl/pyro | Apache-2.0 | Scientific/Engineering::Artificial Intelligence | | python-hyper/h2 | MIT | Internet::WWW/HTTP | | roboflow/supervision | MIT | Scientific/Engineering::Image Processing | | rytilahti/python-miio | GPL-3.0 | Home Automation | | saleweaver/python-amazon-sp-api | MIT | Internet::WWW/HTTP | | scrapy/scrapy | BSD-3-Clause | Software Development::Libraries | | scverse/scanpy | BSD-3-Clause | Scientific/Engineering::Bio-Informatics | | slackapi/bolt-python | MIT | Communications::Chat | | slackapi/python-slack-sdk | MIT | Communications::Chat | | snowflakedb/snowflake-connector-python | Apache-2.0 | Software Development::Libraries | | softlayer/softlayer-python | MIT | Software Development::Libraries | | spec-first/connexion | Apache-2.0 | Internet::WWW/HTTP | | statsmodels/statsmodels | BSD-3-Clause | Scientific/Engineering::Information Analysis | | tfranzel/drf-spectacular | BSD-3-Clause | Software Development::Documentation | | tobymao/sqlglot | MIT | Database::Database Engines/Servers | | tornadoweb/tornado | Apache-2.0 | Internet::WWW/HTTP | | tortoise/tortoise-orm | Apache-2.0 | Database::Front-Ends | | wagtail/wagtail | BSD-3-Clause | Internet::WWW/HTTP |# 参考数据集卡片元数据规范，请参阅：https://github.com/huggingface/hub-docs/blob/main/datasetcard.md?plain=1 # 文档/使用指南：https://huggingface.co/docs/hub/datasets-cards license: other license_name: other license_link: LICENSE dataset_info: features: - name: instance_id dtype: string - name: pull_number dtype: int64 - name: repo dtype: string - name: version dtype: string - name: base_commit dtype: string - name: environment_setup_commit dtype: string - name: created_at dtype: string - name: FAIL_TO_PASS sequence: string - name: PASS_TO_PASS sequence: string splits: - name: test num_bytes: 9002054 num_examples: 1285 download_size: 1588592 dataset_size: 9002054 configs: - config_name: default data_files: - split: test path: data/test-* --- # FEA-Bench 数据集卡片 <!-- 请提供数据集的简要概述。 --> 用于评估仓库级代码生成以实现功能开发的基准测试集。 ## 数据集详情 ### 数据集描述 <!-- 请提供关于该数据集的更详细说明。 --> FEA-Bench是一款基准测试集，其测试集包含来自83个GitHub仓库的1401个任务实例。本基准旨在评估模型在仓库级增量代码开发方面的能力。任务实例均采集自以实现新功能为目的的GitHub拉取请求（Pull Request, PR），每个任务实例包含仓库信息、基础提交SHA256哈希值、PR编号以及单元测试状态。 - **整理方：** FEA-Bench论文作者：李伟、张鑫、郭忠鑫、毛少光及其合作者。 - **语言（自然语言处理）：** 英语 - **许可协议：** 其他相关许可；本数据集涉及的所有GitHub仓库的许可协议已在文末列出。 <!-- - **资助方 [可选]：** {{ funded_by | default("[更多信息需补充]", true)}} - **共享方 [可选]：** {{ shared_by | default("[更多信息需补充]", true)}} --> <!-- ### 数据集来源 [可选] --> <!-- 请提供数据集的基本链接。 --> <!-- - **仓库：** {{ repo | default("[更多信息需补充]", true)}} - **论文 [可选]：** {{ paper | default("[更多信息需补充]", true)}} - **演示 [可选]：** {{ demo | default("[更多信息需补充]", true)}} --> ## 数据集用途 <!-- 请说明本数据集的预期使用场景。 --> 本数据集旨在评估大语言模型（Large Language Model, LLM）在仓库级代码开发这一复杂软件工程任务上的性能表现。 - 仓库级增量代码开发：FEA-Bench可用于评估模型的仓库级增量代码开发能力。该任务的完成效果通常以高/低解决率进行衡量。相关排行榜将以网站形式近期发布。 ### 直接使用场景 可通过FEA-Bench仓库提供的脚本获取任务实例信息，并将其整理为提示词（Prompt）用于大语言模型的推理。此外，也可直接获取相关信息或借助AI智能体（AI Agent）直接解决包含代码变更的PR任务。 ### 不适配使用场景 本数据集并非为大语言模型训练设计，请勿将FEA-Bench用作训练数据集，以免出现数据污染问题。 ## 数据集结构 <!-- 请说明数据集的字段信息，以及额外的数据集结构相关信息，例如划分数据集的标准、数据点之间的关系等。 --> 示例如下： { "instance_id": "huggingface__accelerate-270", "pull_number": 270, "repo": "huggingface/accelerate", "version": null, "base_commit": "515fcca9ed2b36c274c595dbdff75f1c2da635de", "environment_setup_commit": "08101b9dde2b1a9658c2e363e3e9f5663ba06073", "FAIL_TO_PASS": [ "tests/test_state_checkpointing.py::CheckpointTest::test_can_resume_training", "tests/test_state_checkpointing.py::CheckpointTest::test_invalid_registration", "tests/test_state_checkpointing.py::CheckpointTest::test_with_scheduler" ], "PASS_TO_PASS": [] } ## 数据集构建 ### 构建初衷 在仓库级代码库中实现新功能是代码生成模型的重要应用场景之一，但当前基准测试集尚未针对该能力设置专门的评估框架。为填补这一空白，我们推出FEA-Bench——一款旨在评估大语言模型在代码仓库内开展增量开发能力的基准测试集。 ### 源数据 #### 数据采集与处理流程 我们从83个GitHub仓库中采集拉取请求，并基于规则与意图过滤构建以新功能开发为目标的任务实例。每个包含代码变更的任务实例均搭配相关单元测试文件，以确保解决方案可被验证。 #### 源数据生产者是谁？ 本文文末列出的83个GitHub仓库的原作者。 <!-- ### 标注 [可选] --> <!-- 如果数据集包含并非初始数据采集阶段的标注信息，请使用本节描述标注过程。 --> <!-- #### 标注流程 --> <!-- 请说明标注流程，例如过程中使用的标注工具、标注的数据量、提供给标注人员的标注指南、标注者间统计数据、标注验证等。 --> <!-- {{ annotation_process_section | default("[更多信息需补充]", true)}} --> <!-- #### 标注者是谁？ --> <!-- 请说明创建标注的人员或系统。 --> <!-- {{ who_are_annotators_section | default("[更多信息需补充]", true)}} --> #### 个人与敏感信息说明 本数据集未包含任何个人或敏感信息。 ## 偏差、风险与局限性 <!-- 本节用于说明技术与社会技术层面的局限性。 --> - 适用于仓库级增量开发的高质量数据数量有限，面向新功能开发的高质量可用拉取请求相对稀缺。许多用于实现新功能的仓库级代码开发工作发生在仓库初创阶段，未经过开源社区典型的严格代码审查流程，导致数据质量较低无法使用。 - 此外，软件初创阶段的开发工作甚至可能未使用GitHub平台开展，这为数据采集与利用带来了挑战。 - 仓库级增量代码开发的任务范畴并不局限于新功能实现。 - FEA-Bench仅涵盖Python语言仓库。 - 本基准测试集的任务实例推理结果可能包含对计算机系统有害的代码。 ### 建议 建议采用Docker容器进行评估，与SWE-bench的评估方式一致。我们还将发布针对SWE-bench的补丁，使其可兼容本数据集的任务评估。 <!-- ## 引用 [可选] --> <!-- 如果有介绍该数据集的论文或博客文章，请在此处列出其APA和BibTeX格式的引用信息。 --> **BibTeX格式引用：** 待ArXiv论文发表后公布。 **APA格式引用：** 待ArXiv论文发表后公布。 ## 数据集卡片联系人 如需获取更多信息或咨询问题，请联系张鑫（邮箱：[e-mail]）。 ## FEA-Bench涉及的所有GitHub仓库 | 仓库名称 | 许可协议 | 主题 | |-----------------------------------|-------------------|--------------------------------------------| | astropy/astropy | BSD-3-Clause | 科学/工程::天文学 | | django/django | BSD-3-Clause | 互联网::万维网/超文本传输协议 | | matplotlib/matplotlib | Other | 科学/工程::可视化 | | mwaskom/seaborn | BSD-3-Clause | 科学/工程::可视化 | | pallets/flask | BSD-3-Clause | 互联网::万维网/超文本传输协议 | | pvlib/pvlib-python | BSD-3-Clause | 科学/工程::物理学 | | pydata/xarray | Apache-2.0 | 科学/工程::信息分析 | | pydicom/pydicom | Others | 科学/工程::医学应用 | | pylint-dev/astroid | LGPL-2.1 | 软件开发::库文件 | | pylint-dev/pylint | GPL-2.0 | 软件开发::质量保证 | | pyvista/pyvista | MIT | 科学/工程::信息分析 | | scikit-learn/scikit-learn | BSD-3-Clause | 科学/工程::人工智能 | | sphinx-doc/sphinx | BSD-2-Clause | 文本处理::标记语言 | | sqlfluff/sqlfluff | MIT | 软件开发::质量保证 | | sympy/sympy | Others | 科学/工程::数学 | | Aider-AI/aider | Apache-2.0 | 软件开发::代码生成器 | | Cog-Creators/Red-DiscordBot | GPL-3.0 | 通信::聊天工具 | | DLR-RM/stable-baselines3 | MIT | 科学/工程::人工智能 | | EleutherAI/lm-evaluation-harness | MIT | 科学/工程::人工智能 | | Project-MONAI/MONAI | Apache-2.0 | 科学/工程::医学应用 | | PyThaiNLP/pythainlp | Apache-2.0 | 文本处理::语言学 | | RDFLib/rdflib | BSD-3-Clause | 软件开发::库文件 | | Textualize/rich | MIT | 软件开发::库文件 | | Textualize/textual | MIT | 软件开发::用户界面 | | TileDB-Inc/TileDB-Py | MIT | 软件开发::库文件 | | astronomer/astronomer-cosmos | Apache-2.0 | 软件开发::构建工具 | | atlassian-api/atlassian-python-api| Apache-2.0 | 互联网::万维网/超文本传输协议 | | aws-cloudformation/cfn-lint | MIT-0 | 软件开发::质量保证 | | aws-powertools/powertools-lambda-python | MIT-0 | 软件开发::库文件 | | aws/sagemaker-python-sdk | Apache-2.0 | 科学/工程::人工智能 | | biopragmatics/bioregistry | MIT | 科学/工程::生物信息学 | | boto/boto3 | Apache-2.0 | 软件开发::库文件 | | boto/botocore | Apache-2.0 | 软件开发::库文件 | | cocotb/cocotb | BSD-3-Clause | 科学/工程::电子设计自动化(EDA) | | conan-io/conan | MIT | 软件开发::构建工具 | | deepset-ai/haystack | Apache-2.0 | 科学/工程::人工智能 | | docker/docker-py | Apache-2.0 | 软件开发::库文件 | | dpkp/kafka-python | Apache-2.0 | 软件开发::库文件 | | embeddings-benchmark/mteb | Apache-2.0 | 科学/工程::人工智能 | | facebookresearch/hydra | MIT | 软件开发::库文件 | | fairlearn/fairlearn | MIT | 科学/工程::人工智能 | | falconry/falcon | Apache-2.0 | 互联网::万维网/超文本传输协议 | | google-deepmind/optax | Apache-2.0 | 科学/工程::人工智能 | | googleapis/python-aiplatform | Apache-2.0 | 科学/工程::人工智能 | | googleapis/python-bigquery | Apache-2.0 | 互联网::万维网/超文本传输协议 | | gradio-app/gradio | Apache-2.0 | 科学/工程::人机交互 | | graphql-python/graphene | MIT | 软件开发::库文件 | | huggingface/accelerate | Apache-2.0 | 科学/工程::人工智能 | | huggingface/datasets | Apache-2.0 | 科学/工程::人工智能 | | huggingface/huggingface_hub | Apache-2.0 | 科学/工程::人工智能 | | huggingface/pytorch-image-models | Apache-2.0 | 软件开发::库文件 | | huggingface/trl | Apache-2.0 | 科学/工程::人工智能 | | joblib/joblib | BSD-3-Clause | 软件开发::库文件 | | joke2k/faker | MIT | 软件开发::测试工具 | | lark-parser/lark | MIT | 文本处理::语言学 | | minio/minio-py | Apache-2.0 | 软件开发::库文件 | | open-mmlab/mmengine | Apache-2.0 | 实用工具 | | openvinotoolkit/datumaro | MIT | 科学/工程::图像处理 | | pgmpy/pgmpy | MIT | 科学/工程::人工智能 | | pre-commit/pre-commit | MIT | 软件开发::质量保证 | | prometheus/client_python | Apache-2.0 | 系统::监控 | | prompt-toolkit/python-prompt-toolkit | BSD-3-Clause | 软件开发::用户界面 | | pygments/pygments | BSD-2-Clause | 软件开发::文档工具 | | pyocd/pyOCD | Apache-2.0 | 软件开发::调试器 | | pypa/hatch | MIT | 软件开发::构建工具 | | pyro-ppl/pyro | Apache-2.0 | 科学/工程::人工智能 | | python-hyper/h2 | MIT | 互联网::万维网/超文本传输协议 | | roboflow/supervision | MIT | 科学/工程::图像处理 | | rytilahti/python-miio | GPL-3.0 | 家庭自动化 | | saleweaver/python-amazon-sp-api | MIT | 互联网::万维网/超文本传输协议 | | scrapy/scrapy | BSD-3-Clause | 软件开发::库文件 | | scverse/scanpy | BSD-3-Clause | 科学/工程::生物信息学 | | slackapi/bolt-python | MIT | 通信::聊天工具 | | slackapi/python-slack-sdk | MIT | 通信::聊天工具 | | snowflakedb/snowflake-connector-python | Apache-2.0 | 软件开发::库文件 | | softlayer/softlayer-python | MIT | 软件开发::库文件 | | spec-first/connexion | Apache-2.0 | 互联网::万维网/超文本传输协议 | | statsmodels/statsmodels | BSD-3-Clause | 科学/工程::信息分析 | | tfranzel/drf-spectacular | BSD-3-Clause | 软件开发::文档工具 | | tobymao/sqlglot | MIT | 数据库::数据库引擎/服务器 | | tornadoweb/tornado | Apache-2.0 | 互联网::万维网/超文本传输协议 | | tortoise/tortoise-orm | Apache-2.0 | 数据库::前端框架 | | wagtail/wagtail | BSD-3-Clause | 互联网::万维网/超文本传输协议 |