Model outputs for the study "Guidance in Radiology Report Summarization: An Empirical Evaluation and Error Analysis"

This resources provides pre-processed input data, model checkpoints and model outputs for experiments on the OpenI dataset in below study. 

Jan Trienes, Paul Youssef, Jörg Schlötterer, and Christin Seifert. 2023. Guidance in Radiology Report Summarization: An Empirical Evaluation and Error Analysis. In Proceedings of the 16th International Natural Language Generation Conference (INLG), Prague, Czech Republic. Association for Computational Linguistics.

For more information please refer to the accompanying paper and code repository (https://github.com/jantrienes/inlg2023-radsum).

The data is structured as follows:

data/preprocessed/ includes the dataset(s) for each model

output/ includes one folder for each experiment/model run. The first part of each output path indicates the dataset that was used at inference.

For a mapping between model IDs and results in the paper, see below table. All models were also trained with the background section as input. These are available in directories with the -bg- qualifier. 

Model name in paper
Output directory

Results from Table 2

OracleExt
openi-unguided/oracle

BertExt (Liu and Lapata, 2019)
openi-unguided/bertext-default

BertAbs (Liu and Lapata, 2019)
openi-unguided/bertabs-default

GSum (Dou et al., 2021)
openi-bertext-default-clip-k1/gsum-default

GSum w/ LR-Approx
openi-bertext-default-clip-lrapprox/gsum-default

GSum w/ BERT-Approx
openi-bertext-default-clip-bertapprox/gsum-default

GSum w/ Thresholding
openi-bertext-default-clip-threshold/gsum-default

WGSum (Hu et al., 2021)
openi-wgsum/wgsum-default

WGSum+CL (Hu et al., 2022)
openi-wgsum-cl/wgsum-cl-default

Results from Table 3

Fixed (k=1)
openi-unguided/bertext-default-clip-k1

LR-Approx
openi-unguided/bertext-default-clip-lrapprox

BERT-Approx
openi-unguided/bertext-default-clip-bertapprox

Thresholding
openi-unguided/bertext-default-clip-threshold

k = |OracleExt|
openi-unguided/bertext-default-clip-oracle

Results from Table 4

Fixed (Dou et al., 2021)
openi-bertext-default-clip-k1/gsum-default

Oracle Length
openi-bertext-default-clip-oracle/gsum-default

Oracle Length + Content
openi-oracle/gsum-default

Results from Table 5

BertExt w/ k=[1,5]
openi-unguided/bertext-default-clip-k{1,2,3,4,5}

GSum w/ k=[1,5]
openi-bertext-default-clip-k{1,2,3,4,5}/gsum-default