CompanyKG Dataset V2.0: A Large-Scale Heterogeneous Graph for Company Similarity Quantification

CompanyKG is a heterogeneous graph consisting of 1,169,931 nodes and 50,815,503 undirected edges, with each node representing a real-world company and each edge signifying a relationship between the connected pair of companies. Edges: We model 15 different inter-company relations as undirected edges, each of which corresponds to a unique edge type. These edge types capture various forms of similarity between connected company pairs. Associated with each edge of a certain type, we calculate a real-numbered weight as an approximation of the similarity level of that type. It is important to note that the constructed edges do not represent an exhaustive list of all possible edges due to incomplete information. Consequently, this leads to a sparse and occasionally skewed distribution of edges for individual relation/edge types. Such characteristics pose additional challenges for downstream learning tasks. Please refer to our paper for a detailed definition of edge types and weight calculations. Nodes: The graph includes all companies connected by edges defined previously. Each node represents a company and is associated with a descriptive text, such as "Klarna is a fintech company that provides support for direct and post-purchase payments ...". To comply with privacy and confidentiality requirements, we encoded the text into numerical embeddings using four different pre-trained text embedding models: mSBERT (multilingual Sentence BERT), ADA2, SimCSE (fine-tuned on the raw company descriptions) and PAUSE. Evaluation Tasks. The primary goal of CompanyKG is to develop algorithms and models for quantifying the similarity between pairs of companies. In order to evaluate the effectiveness of these methods, we have carefully curated three evaluation tasks: Similarity Prediction (SP). To assess the accuracy of pairwise company similarity, we constructed the SP evaluation set comprising 3,219 pairs of companies that are labeled either as positive (similar, denoted by "1") or negative (dissimilar, denoted by "0"). Of these pairs, 1,522 are positive and 1,697 are negative. Competitor Retrieval (CR). Each sample contains one target company and one of its direct competitors. It contains 76 distinct target companies, each of which has 5.3 competitors annotated in average. For a given target company A with N direct competitors in this CR evaluation set, we expect a competent method to retrieve all N competitors when searching for similar companies to A. Similarity Ranking (SR) is designed to assess the ability of any method to rank candidate companies (numbered 0 and 1) based on their similarity to a query company. Paid human annotators, with backgrounds in engineering, science, and investment, were tasked with determining which candidate company is more similar to the query company. It resulted in an evaluation set comprising 1,856 rigorously labeled ranking questions. We retained 20% (368 samples) of this set as a validation set for model development. Edge Prediction (EP) evaluates a model's ability to predict future or missing relationships between companies, providing forward-looking insights for investment professionals. The EP dataset, derived (and sampled) from new edges collected between April 6, 2023, and May 25, 2024, includes 40,000 samples, with edges not present in the pre-existing CompanyKG (a snapshot up until April 5, 2023). Background and Motivation In the investment industry, it is often essential to identify similar companies for a variety of purposes, such as market/competitor mapping and Mergers & Acquisitions (M&A). Identifying comparable companies is a critical task, as it can inform investment decisions, help identify potential synergies, and reveal areas for growth and improvement. The accurate quantification of inter-company similarity, also referred to as company similarity quantification, is the cornerstone to successfully executing such tasks. However, company similarity quantification is often a challenging and time-consuming process, given the vast amount of data available on each company, and the complex and diversified relationships among them. While there is no universally agreed definition of company similarity, researchers and practitioners in PE industry have adopted various criteria to measure similarity, typically reflecting the companies' operations and relationships. These criteria can embody one or more dimensions such as industry sectors, employee profiles, keywords/tags, customers' review, financial performance, co-appearance in news, and so on. Investment professionals usually begin with a limited number of companies of interest (a.k.a. seed companies) and require an algorithmic approach to expand their search to a larger list of companies for potential investment. In recent years, transformer-based Language Models (LMs) have become the preferred method for encoding textual company descriptions into vector-space embeddings. Then companies that are similar to the seed companies can be searched in the embedding space using distance metrics like cosine similarity. The rapid advancements in Large LMs (LLMs), such as GPT-3/4 and LLaMA, have significantly enhanced the performance of general-purpose conversational models. These models, such as ChatGPT, can be employed to answer questions related to similar company discovery and quantification in a Q&A format. However, graph is still the most natural choice for representing and learning diverse company relations due to its ability to model complex relationships between a large number of entities. By representing companies as nodes and their relationships as edges, we can form a Knowledge Graph (KG). Utilizing this KG allows us to efficiently capture and analyze the network structure of the business landscape. Moreover, KG-based approaches allow us to leverage powerful tools from network science, graph theory, and graph-based machine learning, such as Graph Neural Networks (GNNs), to extract insights and patterns to facilitate similar company analysis. While there are various company datasets (mostly commercial/proprietary and non-relational) and graph datasets available (mostly for single link/node/graph-level predictions), there is a scarcity of datasets and benchmarks that combine both to create a large-scale KG dataset expressing rich pairwise company relations. Source Code and Tutorial:https://github.com/llcresearch/CompanyKG2 Paper: to be publishedCompanyKG（企业知识图谱）是一个异质图，包含1169931个节点与50815503条无向边，其中每个节点代表一家真实企业，每条边则表示相连两家企业之间的关联关系。 **边结构**：我们将15种不同的企业间关联建模为无向边，每种关联对应唯一的边类型。这些边类型刻画了相连企业对之间的多种相似性维度。对于每种类型的边，我们会计算一个实数值权重，用以近似该类型下的相似程度。需要说明的是，由于信息不完备，本次构建的边并非所有潜在关联的完整集合，这导致单个关联/边类型的边分布呈现稀疏性，且部分类型存在偏态分布，这为下游学习任务带来了额外挑战。边类型的详细定义与权重计算方式，请参阅我们的论文。 **节点结构**：该图谱包含前述所有通过边连接的企业。每个节点代表一家企业，并附带一段描述性文本，例如“Klarna是一家为直接支付及售后付款提供支持的金融科技公司……”。为遵守隐私与保密要求，我们使用四种不同的预训练文本嵌入模型将文本编码为数值嵌入：mSBERT（多语言Sentence BERT）、ADA2、SimCSE（基于原始企业描述微调）以及PAUSE。 **评估任务**：CompanyKG的核心目标是开发用于量化企业间相似性的算法与模型。为评估这些方法的有效性，我们精心设计了四项评估任务： 1. 相似性预测（Similarity Prediction, SP）：为评估成对企业相似性的预测准确性，我们构建了SP评估集，包含3219对企业样本，样本标签分为正例（相似，标记为“1”）与负例（不相似，标记为“0”），其中正例样本1522个，负例样本1697个。 2. 竞品检索（Competitor Retrieval, CR）：每个样本包含一家目标企业及其一家直接竞品，共涵盖76家不同的目标企业，每家企业平均标注5.3个竞品。对于该CR评估集中拥有N个直接竞品的目标企业A，我们期望优秀的模型能够在检索与A相似的企业时，返回全部N个竞品。 3. 相似性排序（Similarity Ranking, SR）：该任务用于评估模型基于候选企业（编号0与1）与查询企业的相似性进行排序的能力。我们邀请了具备工程、科学及投资背景的付费人类标注人员，判断哪一家候选企业与查询企业的相似性更高。最终构建的评估集包含1856个经过严格标注的排序问题，我们保留其中20%（368个样本）作为模型开发用的验证集。 4. 边预测（Edge Prediction, EP）：该任务评估模型预测企业间未来或缺失关联的能力，可为投资专业人士提供前瞻性洞察。EP数据集源自2023年4月6日至2024年5月25日期间收集的新边（并经采样处理），共包含40000个样本，且这些边均未出现在此前的CompanyKG快照（截至2023年4月5日）中。 **研究背景与动机**：在投资行业中，识别相似企业通常是一项核心需求，可应用于市场/竞品映射、并购（Mergers & Acquisitions, M&A）等诸多场景。识别可比企业是一项关键任务，可为投资决策提供参考、助力发掘潜在协同效应，并揭示增长与优化方向。准确量化企业间相似性（又称企业相似性量化）是完成上述任务的基石。然而，由于每家企业相关的数据体量庞大、企业间关系复杂多样，企业相似性量化往往是一项极具挑战性且耗时的工作。 尽管目前尚无关于企业相似性的统一定义，但私募股权（PE）行业的研究人员与从业者已采用多种标准衡量相似性，这些标准通常反映企业的运营模式与关联关系，可涵盖行业板块、员工画像、关键词/标签、客户评价、财务表现、新闻共现等一个或多个维度。投资专业人士通常从少量目标企业（又称种子企业）出发，需要借助算法手段扩展搜索范围，以获取更多潜在投资标的企业。 近年来，基于Transformer的语言模型（LMs）已成为将企业文本描述编码为向量空间嵌入的主流方法。随后，可通过余弦相似度等距离度量在嵌入空间中检索与种子企业相似的企业。随着大语言模型（Large Language Model, LLM）的快速发展，例如GPT-3/4与LLaMA，通用对话模型的性能得到了显著提升。诸如ChatGPT这类模型，可通过问答形式解答与相似企业发现及量化相关的问题。 然而，图谱仍是表示与学习多样企业关联的最自然选择，因其能够建模大量实体间的复杂关系。将企业表示为节点、关联关系表示为边，即可构建知识图谱（Knowledge Graph, KG）。利用该知识图谱，我们能够高效捕捉并分析商业格局的网络结构。此外，基于知识图谱的方法可让我们借助网络科学、图论以及图神经网络（Graph Neural Networks, GNNs）等领域的强大工具，提取洞察与模式，以助力相似企业分析。尽管目前已有多种企业数据集（多为商用/专有且非关系型）与图谱数据集（多针对单链路/节点/图级预测任务），但兼具大规模性与丰富企业成对关联表达能力的知识图谱数据集与基准测试集仍较为稀缺。 **源代码与教程**：https://github.com/llcresearch/CompanyKG2 **论文**：待发表