Medical Imaging Data Resource Center (MIDRC) - RSNA International COVID-19 Open Radiology Database (RICORD) Release 1c - Chest x-ray Covid+

<h4>Background</h4><p>The COVID-19 pandemic is a global healthcare emergency. Prediction models for COVID-19 imaging are rapidly being developed to support medical decision making in imaging. However, inadequate availability of a diverse annotated dataset has limited the performance and generalizability of existing models.</p><h4>Purpose</h4><p>To create the first multi-institutional, multi-national expert annotated COVID-19 imaging dataset made freely available to the machine learning community as a research and educational resource for COVID-19 chest imaging. The Radiological Society of North America (RSNA) assembled the <a href="https://www.rsna.org/en/covid-19/COVID-19-RICORD">RSNA International COVID-19 Open Radiology Database (RICORD)</a> collection of COVID-related imaging datasets and expert annotations to support research and education. RICORD data will be incorporated in the <a href="https://www.midrc.org/">Medical Imaging and Data Resource Center (MIDRC)</a>, a multi-institutional research data repository funded by the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health.</p><h4>Materials and Methods</h4><p>This dataset was created through a collaboration between the RSNA and Society of Thoracic Radiology (STR). Clinical annotation by thoracic radiology subspecialists was performed for all COVID positive chest radiography (CXR) imaging studies using a labeling schema based upon guidelines for reporting classification of COVID-19 findings in CXRs (see <a href="https://doi.org/10.1097/rti.0000000000000541">Review of Chest Radiograph Findings of COVID-19 Pneumonia and Suggested Reporting Language</a>, Journal of Thoracic Imaging).</p><h4>Results</h4><p>The RSNA International COVID-19 Open Annotated Radiology Database (RICORD) consists of 998 chest x-rays from 361 patients at four international sites annotated with diagnostic labels.</p><p>Patient Selection: Patients at least 18 years in age receiving positive diagnosis for COVID-19.</p><h4>Data Abstract</h4><ol><li><p>998 Chest x-ray examinations from 361 patients.</p></li><li><p>Annotations with labels:</p><ol><li><p>Classification</p><ul><li><p><u>Typical Appearance</u><br />Multifocal bilateral, peripheral opacities, and/or Opacities with rounded morphology<br />Lower lung-predominant distribution (Required Feature - must be present with either or both of the first two opacity patterns)</p></li><li><p><u>Indeterminate Appearance</u><br />Absence of typical findings AND Unilateral, central or upper lung predominant distribution of airspace disease</p></li><li><u>Atypical Appearance</u><br />Pneumothorax or pleural effusion, Pulmonary Edema, Lobar Consolidation, Solitary lung nodule or mass, Diffuse tiny nodules, Cavity</li><li><p><u>Negative for Pneumonia</u><br />No lung opacities</p></li></ul></li><li><p>Airspace Disease Grading<br />Lungs are divided on frontal chest xray into 3 zones per lung (6 zones total). The upper zone extends from the apices to the superior hilum. The mid zone spans between the superior and inferior hilar margins. The lower zone extends from the inferior hilar margins to the costophrenic sulci.</p><ul><li><p><u>Mild</u> - Required if not negative for pneumonia<br />Opacities in 1-2 lung zones</p></li><li><p><u>Moderate</u> - Required if not negative for pneumonia<br />Opacities in 3-4 lung zones</p></li><li><p><u>Severe</u> - Required if not negative for pneumonia<br />Opacities in >4 lung zones</p></li></ul></li></ol></li><li><p> Supporting clinical variables: MRN*, Age, Study Date*, Exam Description, Sex, Study UID*, Image Count, Modality, Testing Result, Specimen Source (* pseudonymous values).</p></li></ol><div><p>How to use the JSON annotations</p><div><p>More information about how the JSON annotations are organized can be found on <a href="https://docs.md.ai/data/json/">https://docs.md.ai/data/json/</a>.  Steps 2 & 3 in this <a href="https://docs.md.ai/libraries/python/guides-convert-json/">example code</a> demonstrate how to to load the JSON into a Dataframe. The JSON file can be downloaded via the data access table below; it is not available via <a href="http://MD.ai">MD.ai</a>. This <a href="https://github.com/mdai/ml-lessons/blob/master/lesson2-lung-xrays-segmentation.ipynb">Jupyter Notebook</a> may also be helpful.</p></div></div><h4>Research Benefits</h4><p>RICORD is available for non-commercial use (and further enrichment) by the research and education communities which may include development of educational resources for COVID-19, use of RICORD to create AI systems for diagnosis and quantification, benchmarking performance for existing solutions, exploration of distributed/federated learning, further annotation or data augmentation efforts, and evaluation of the examinations for disease entities beyond COVID-19 pneumonia. Deliberate consideration of the detailed annotation schema, demographics, and other included meta-data will be critical when generating cohorts with RICORD, particularly as more public COVID-19 imaging datasets are made available via complementary and parallel efforts. It is important to emphasize that there are limitations to the clinical “ground truth” as the SARS-CoV-2 RT-PCR tests have widely documented limitations and are subject to both false-negative and false-positive results which impact the distribution of the included imaging data, and may have led to an unknown epidemiologic distortion of patients based on the inclusion criteria. These limitations notwithstanding, RICORD has achieved the stated objectives for data complexity, heterogeneity, and high-quality expert annotations as a comprehensive COVID-19 thoracic imaging data resource.</p>

<h4>背景</h4><p>COVID-19疫情是一场全球性的公共卫生紧急事件。针对COVID-19影像的预测模型正在迅速开发，以支持影像学领域内的医疗决策。然而，多样化的标注数据集供应不足，限制了现有模型的性能和泛化能力。</p><h4>目的</h4><p>构建首个由多机构、多国专家标注的COVID-19影像数据集，并将其免费提供给机器学习社区作为COVID-19胸部影像研究和教育资源。北美放射学会（RSNA）汇集了<a href="https://www.rsna.org/en/covid-19/COVID-19-RICORD">RSNA国际COVID-19开放放射学数据库（RICORD）</a>，该数据库收集了与COVID相关的影像数据集和专家标注，以支持研究和教育。RICORD数据将被纳入<a href="https://www.midrc.org/">医学影像和数据资源中心（MIDRC）</a>，这是一个由美国国立卫生研究院国家生物医学成像与生物工程研究所资助的多机构研究数据存储库。</p><h4>材料和与方法</h4><p>本数据集由RSNA和胸部放射学会（STR）合作创建。由胸部放射学亚专业医师对所有COVID阳性胸部X光摄影（CXR）影像研究进行了临床标注，使用基于报告COVID-19在CXR中发现的分类指南的标注方案（参见<a href="https://doi.org/10.1097/rti.0000000000000541">COVID-19肺炎胸部X光影像发现综述及建议报告语言</a>，胸科学杂志）。</p><h4>结果</h4><p>RSNA国际COVID-19开放标注放射学数据库（RICORD）包括来自四个国际地点的361名患者的998张胸部X光片，并附有诊断标签。</p><p>患者选择：年龄至少为18岁，且被诊断为COVID-19的患者。</p><h4>数据摘要</h4><ol><li><p>361名患者的998项胸部X光检查。</p></li><li><p>带有标签的标注：</p><ol><li><p>分类</p><ul><li><p><u>典型表现</u><br>多灶性双侧、边缘性模糊，以及/或边缘圆润的模糊<br>下肺优先分布（必备特征 - 必须与上述两种模糊模式之一或两者同时存在）</p></li><li><p><u>不确定表现</u><br>缺乏典型发现 AND 肺泡病损分布以单侧、中央或上肺为主</p></li><li><p><u>非典型表现</u><br>气胸或胸腔积液、肺水肿、肺叶实变、孤立性肺结节或肿块、弥漫性小结节、空洞</p></li><li><p><u>无肺炎</u><br>无肺部模糊</p></li></ul></li><li><p>肺泡病损分级<br>在正位胸部X光片上，将肺部分为3个区域（总共6个区域）。上区从尖部延伸至上肺门。中区位于上、下肺门之间。下区从下肺门延伸至肋膈窦。</p><ul><li><p><u>轻度</u><br>如果不为肺炎阴性，则必需<br>1-2个肺区模糊</p></li><li><p><u>中度</u><br>如果不为肺炎阴性，则必需<br>3-4个肺区模糊</p></li><li><p><u>重度</u><br>如果不为肺炎阴性，则必需<br>超过4个肺区模糊</p></li></ul></li></ol></li><li><p>支持性临床变量：MRN*、年龄、研究日期*、检查描述、性别、研究UID*、图像数量、模式、检测结果、标本来源（*为化名值）。</p></li></ol><div><p>如何使用JSON标注</p><div><p>有关如何组织JSON标注的更多信息，请参阅<a href="https://docs.md.ai/data//">https://docs.md.ai/data//</a>。此<a href="https://docs.md.ai/libraries/python/guides-convert-/">示例代码</a>中的步骤2和3演示了如何将JSON加载到数据框中。可以通过下表中的数据访问表下载JSON文件；它不通过<a href="http://MD.ai">MD.ai</a>提供。此<a href="https://github.com/mdai/ml-lessons/blob/master/lesson2-lung-xrays-segmentation.ipynb">Jupyter Notebook</a>也可能有所帮助。</p></div></div><h4>研究益处</h4><p>RICORD可供研究和教育社区非商业用途（并进一步丰富）使用，这可能包括开发COVID-19的教育资源、使用RICORD创建用于诊断和定量分析的AI系统、为现有解决方案进行基准测试、探索分布式/联邦学习、进一步标注或数据增强工作，以及评估针对COVID-19肺炎以外的疾病实体的检查。在生成包含RICORD的队列时，仔细考虑详细的标注方案、人口统计学和其他包含的元数据将是至关重要的，尤其是随着更多公共COVID-19影像数据集通过互补和并行努力公开发布时。重要的是要强调，临床“真实情况”存在局限性，因为SARS-CoV-2 RT-PCR检测广泛记录了局限性，并且存在假阴性和假阳性结果，这些结果影响了所包含影像数据的分布，并可能导致基于纳入标准的患者流行病学未知扭曲。尽管存在这些局限性，RICORD作为一个综合的COVID-19胸部影像数据资源，已实现了所声明的关于数据复杂性、异质性和高质量专家标注的目标。</p>