MedFSN-Bench: A Benchmark Corpus for Medical Device Safety

MedFSN-Bench is a benchmark corpus constructed from Field Safety Notices (FSNs) issued within medical device vigilance systems. FSNs are manufacturer-issued regulatory communications describing device malfunctions, associated risks, health hazards, and corrective actions identified during post-market surveillance. The corpus was developed to support research on Natural Language Processing (NLP) and Large Language Models (LLMs) for medical device safety analysis. MedFSN-Bench contains three benchmark tasks: Task 1: Medical Device Classification Input: Intended-use descriptions and candidate label space Output: Device category and subcategory Task 2: Health Effect Prediction Input: Device problem descriptions, risk descriptions, and candidate label space derived from IMDRF Annex F terminology Output: Appropriate IMDRF Annex F health effect terms Task 3: Device Problem Classification Input: Device problem descriptions, risk descriptions, corrective actions, and candidate label space derived from IMDRF Annex A terminology Output: Appropriate IMDRF Annex A device problem terms The corpus was constructed from publicly available Field Safety Notices collected from the German medical device vigilance system between 2013 and 2025. A hybrid human–LLM annotation workflow was employed, followed by LLM-as-a-Judge verification and additional human quality checks on a subset of the data. Corpus Statistics: Total instances: 874 Task 1: 612 instances Task 2: 192 instances Task 3: 70 instances The dataset is intended for benchmarking NLP and LLM systems in medical device safety surveillance, regulatory document understanding, post-market surveillance, and medical device vigilance research. This resource is released to support academic research, education, and reproducible evaluation of AI systems in the medical device safety domain. The benchmark was derived from publicly available regulatory safety communications and is intended solely for scientific, educational, and non-commercial research purposes. Ownership and copyright of the original source documents remain with their respective rights holders. Users are responsible for complying with any applicable licensing, copyright, and usage conditions associated with the original source materials. Citation If you use MedFSN-Bench in your research, please cite the associated publication: Md Moin Uddin and Mouzhi Ge. 2026. MedFSN-Bench: A Benchmark Corpus for Medical Device Safety. In Proceedings of the 17th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics (BCB '26), June 30–July 03, 2026, Rende (CS), Italy. ACM, New York, NY, USA. DOI: https://doi.org/10.1145/3807503.3819497 BibTeX: @inproceedings{uddin2026medfsnbench,author = {Uddin, Md Moin and Ge, Mouzhi},title = {MedFSN-Bench: A Benchmark Corpus for Medical Device Safety},booktitle = {Proceedings of the 17th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics (BCB '26)},year = {2026},location = {Rende (CS), Italy},publisher = {Association for Computing Machinery},doi = {10.1145/3807503.3819497},isbn = {979-8-4007-2653-8/2026/06}}