FlyWire Whole-brain Connectome Connectivity Data

This repository contains the connectivity data for the FlyWire Connectome release. Currently, the latest release is version 783 (see also codex.flywire.ai). The synapses represent a combination of four different data releases, which are combined by the FlyWire whole-brain connectome release. The synapses (as points in space) were detected and published by Buhmann et al., 2021 who made use of a cleft segmentation produced by Heinrich et al., 2018. The neurotransmitter for these synapses was then predicted and released by Eckstein, Bates et al., 2024. The segmentation and neuron IDs (= root IDs) were proofread by the FlyWire consortium and are released by Dorkenwald et al., 2024 as part of the FlyWire connectome paper package. Because multiple methods were involved in the production of this resource, the description of the methods is distributed across these manuscripts. We provide a summary in Dorkenwald et al., 2024 (Methods->Synaptic connections). Some of the files made available use feather as a file format. See a code example for reading these files, including a chunk-wise streaming to handle the large file. flywire_synapses_783.feather a pandas dataframe with all ~130 million synapses, their locations, neurotransmitter predictions and pre and postsynaptic partners (=root ids). This table contains all synapses that passed the thresholds (see methods section in Dorkenwald et al., 2024), but not all synapses were associated with proofread neurons (e.g., see Discussion->Limitations of our reconstruction in Dorkenwald et al., 2024). Hence, not all root IDs in this table will have a match in the proofread root IDs array. This table is provided for completeness, and to allow calculations about total synaptic input and output of neurons independent of other limitations. Columns: id: synapse ID pre_pt_root_id: presynaptic neuron ID post_pt_root_id: postsynaptic neuron ID connection_score: score assigned by Buhmann et al.; higher is better. We did not use this score to threshold synapses in any analysis cleft_score: score derived from the cleft segmentation by Heinrich et al.; higher is better. We used a threshold of 50 for all analyses and the released dataset. Synapses with lower score are not made available but can be made available on demand. gaba: probability for neurotransmitter=GABA ach: probability for neurotransmitter=Acetylcholine glut: probability for neurotransmitter=Glutamate oct: probability for neurotransmitter=Octopamine ser: probability for neurotransmitter=Serotonin da: probability for neurotransmitter=Dopamine neuropil: the name of the neuropil associated with this synapse. Symmetric neuropils contain a hemisphere annotation after '_'. E.g., ME_L is the medulla in the left hemisphere. For mapping long names, see Ext. Data Fig. 1 or https://codex.flywire.ai/app/neuropils post_pt_position_{x,y,z}: Coordinate within the postsynaptic neuron (synapses were identified with two points, one in each neuron). Coordinates are in nanometers. pre_pt_position_{x,y,z}: Coordinate within the presynaptic neuron (synapses were identified with two points, one in each neuron). Coordinates are in nanometers. per_neuron_neuropil_count_post_783.feather a pandas dataframe containing the number of postsynapses per neuropil and segment id, i.e. this is a summarized version of flywire_synapses_783.feather Columns: post_pt_root_id: segment ID neuropil: neuropil name. Symmetric neuropils contain a hemisphere annotation after '_'. E.g., ME_L is the medulla in the left hemisphere. For a mapping to long names see Ext. Data Fig. 1 or https://codex.flywire.ai/app/neuropils Count: number of synapses for this segment ID and neuropil per_neuron_neuropil_count_pre_783.feather a pandas dataframe containing the number of presynapses per neuropil and segment id, i.e. this is a summarized version of flywire_synapses_783.feather Columns: pre_pt_root_id: segment ID neuropil: neuropil name. Symmetric neuropils contain a hemisphere annotation after '_'. E.g., ME_L is the medulla in the left hemisphere. For mapping long names, see Ext. Data Fig. 1 or https://codex.flywire.ai/app/neuropils Count: number of synapses for this segment ID and neuropil proofread_root_ids_783.npy an array of all proofread neuron ids (=root ids) proofread_connections_783.feather a pandas dataframe containing the proofread subset from flywire_synapses_783.feather and summarized per neuron-neuron pair and neuropil, i.e. this table contains one entry per neuron-neuron pair and neuropil if there is 1 or more synapses for a given combination Columns: pre_pt_root_id: presynaptic neuron ID post_pt_root_id: postsynaptic neuron ID neuropil: neuropil name. Symmetric neuropils contain a hemisphere annotation after '_'. E.g., ME_L is the medulla in the left hemisphere. For mapping long names, see Ext. Data Fig. 1 or https://codex.flywire.ai/app/neuropils syn_count: number of synapses between these two neurons in this neuropil gaba_avg: average probability across the synapses for neurotransmitter=GABA ach_avg: average probability across the synapses for neurotransmitter=Acetylcholine glut_avg: average probability across the synapses for neurotransmitter=Glutamate oct_avg: average probability across the synapses for neurotransmitter=Octopamine ser_avg: average probability across the synapses for neurotransmitter=Serotonin da_avg: average probability across the synapses for neurotransmitter=Dopamine Code for reading and streaming feather files Read feather files with pandas: import pandas as pd df = pd.read_feather(path) Stream large feather files in chunks: import pyarrow.feather as feather table = feather.read_table(path) # Total number of rows in the Feather filenum_rows = table.num_rows # Define chunk sizechunk_size = 1000 # Read and process the data in chunksfor i in range(0, num_rows, chunk_size): end_row = min(i + chunk_size, num_rows) chunk = table.slice(i, end_row - i) # Slice the table from i to end_row # Convert to pandas DataFrame if needed df_chunk = chunk.to_pandas() # Now you can process each chunk DataFrame as needed print(df_chunk.head())本仓库包含FlyWire连接组（FlyWire Connectome）发布的全脑连接组数据。当前最新发布版本为783版（亦可参考codex.flywire.ai）。 本数据集的突触整合了四项不同的数据发布成果，由FlyWire全脑连接组发布项目进行整合。这些突触以空间点位形式呈现，由Buhmann等人于2021年检测并发布，其研究依托Heinrich等人2018年提出的裂隙分割结果。随后，Eckstein、Bates等人于2024年完成了这些突触的神经递质预测并对外发布。神经元分割结果与神经元ID（即根ID，root IDs）由FlyWire联盟完成校对，并由Dorkenwald等人2024年作为FlyWire连接组论文配套资料的一部分发布。 由于该资源的制备涉及多种方法，相关方法描述分散于各研究手稿中。我们在Dorkenwald等人2024年的研究（方法部分→突触连接）中提供了方法总结。 部分文件采用feather格式（feather）存储，可参考下述代码示例读取此类文件，包括分块流式读取以处理大型文件。 --- #### flywire_synapses_783.feather 该文件为包含约1.3亿条突触的pandas数据框，记录了突触位置、神经递质预测结果以及突触前（presynaptic）、突触后（postsynaptic）神经元的根ID。本表格包含所有通过阈值筛选的突触（详见Dorkenwald等人2024年的方法部分），但并非所有突触都关联了已校对的神经元（详见Dorkenwald等人2024年的讨论→本重建的局限性部分）。因此，该表格中的部分根ID无法在已校对根ID数组中找到匹配项。本表格旨在提供完整的数据视图，支持独立于其他限制条件的神经元总突触输入、输出量计算。 列说明： - id：突触ID - pre_pt_root_id：突触前（presynaptic）神经元ID - post_pt_root_id：突触后（postsynaptic）神经元ID - connection_score：Buhmann等人赋予的连接评分，分值越高效果越好。本研究在任何分析中均未使用该评分进行突触筛选 - cleft_score：源自Heinrich等人裂隙分割结果的裂隙评分，分值越高效果越好。本研究在所有分析与发布的数据集中均采用50作为阈值，得分低于50的突触未被纳入本数据集，但可按需申请获取 - gaba：神经递质为γ-氨基丁酸（GABA）的概率 - ach：神经递质为乙酰胆碱（Acetylcholine）的概率 - glut：神经递质为谷氨酸（Glutamate）的概率 - oct：神经递质为章鱼胺（Octopamine）的概率 - ser：神经递质为5-羟色胺（Serotonin）的概率 - da：神经递质为多巴胺（Dopamine）的概率 - neuropil：该突触所属的神经纤维网（neuropil）名称。具有双侧对称性的神经纤维网名称后会带有半球标注，例如ME_L代表左侧半球的髓质（medulla）。如需获取全称映射关系，可参考扩展数据图1或https://codex.flywire.ai/app/neuropils - post_pt_position_{x,y,z}：突触后（postsynaptic）神经元内的坐标（突触通过两个点位识别，分别位于两个神经元内）。坐标单位为纳米 - pre_pt_position_{x,y,z}：突触前（presynaptic）神经元内的坐标（突触通过两个点位识别，分别位于两个神经元内）。坐标单位为纳米 --- #### per_neuron_neuropil_count_post_783.feather 该文件为pandas数据框，记录了每个神经纤维网（neuropil）与神经元片段ID对应的突触后突触数量，是flywire_synapses_783.feather的汇总版本。 列说明： - post_pt_root_id：神经元片段ID - neuropil：神经纤维网（neuropil）名称。具有双侧对称性的神经纤维网名称后会带有半球标注，例如ME_L代表左侧半球的髓质。如需获取全称映射关系，可参考扩展数据图1或https://codex.flywire.ai/app/neuropils - Count：该神经元片段ID与神经纤维网对应的突触数量 --- #### per_neuron_neuropil_count_pre_783.feather 该文件为pandas数据框，记录了每个神经纤维网（neuropil）与神经元片段ID对应的突触前突触数量，是flywire_synapses_783.feather的汇总版本。 列说明： - pre_pt_root_id：神经元片段ID - neuropil：神经纤维网（neuropil）名称。具有双侧对称性的神经纤维网名称后会带有半球标注，例如ME_L代表左侧半球的髓质。如需获取全称映射关系，可参考扩展数据图1或https://codex.flywire.ai/app/neuropils - Count：该神经元片段ID与神经纤维网对应的突触数量 --- #### proofread_root_ids_783.npy 该文件为包含所有已校对神经元ID（即根ID）的数组。 --- #### proofread_connections_783.feather 该文件为pandas数据框，包含flywire_synapses_783.feather中的已校对子集，并按神经元-神经元对与神经纤维网（neuropil）进行汇总。即，若给定神经元-神经元对与神经纤维网的组合存在至少1个突触，则该表格中会存在对应条目。 列说明： - pre_pt_root_id：突触前（presynaptic）神经元ID - post_pt_root_id：突触后（postsynaptic）神经元ID - neuropil：神经纤维网（neuropil）名称。具有双侧对称性的神经纤维网名称后会带有半球标注，例如ME_L代表左侧半球的髓质。如需获取全称映射关系，可参考扩展数据图1或https://codex.flywire.ai/app/neuropils - syn_count：该神经纤维网中，这两个神经元之间的突触数量 - gaba_avg：该组突触中神经递质为GABA的平均概率 - ach_avg：该组突触中神经递质为乙酰胆碱的平均概率 - glut_avg：该组突触中神经递质为谷氨酸的平均概率 - oct_avg：该组突触中神经递质为章鱼胺的平均概率 - ser_avg：该组突触中神经递质为5-羟色胺的平均概率 - da_avg：该组突触中神经递质为多巴胺的平均概率 --- ### 读取与流式读取feather文件的代码示例 #### 使用pandas读取feather文件 python import pandas as pd df = pd.read_feather(path) #### 分块流式读取大型feather文件 python import pyarrow.feather as feather table = feather.read_table(path) # 获取Feather文件的总行数 num_rows = table.num_rows # 定义分块大小 chunk_size = 1000 # 分块读取并处理数据 for i in range(0, num_rows, chunk_size): end_row = min(i + chunk_size, num_rows) # 从表格中截取分块数据 chunk = table.slice(i, end_row - i) # 如需转换为pandas数据框 df_chunk = chunk.to_pandas() # 在此处对分块数据框进行处理 print(df_chunk.head())