<p>Human-only analysis procedure and person-hours.</p>

Background
The exponential growth of Big Qualitative (Big Qual) data in healthcare research presents methodological challenges for traditional analysis approaches. This study evaluates the effectiveness of machine-assisted analysis using artificial intelligence (AI) tools compared to human-only analysis for processing large-scale qualitative datasets, using the Royal College of Anaesthetists’ 7th National Audit Project (NAP7) baseline survey as a test case.
Methodology/Principal Findings
We conducted a comparative methodological study analysing 5,196 free-text responses about peri-operative cardiac arrest experiences. Three researchers established a human-coded reference standard following SRQR guidelines. We then applied machine-assisted analysis using Pulsar for exploratory analysis and Caplena for sentiment and thematic analysis, evaluating performance against the human gold standard using STARD-AI reporting standards. Performance metrics included accuracy, precision, recall, F1-scores, and Cohen’s Kappa, with confidence intervals calculated using bootstrap resampling.
Machine-assisted analysis substantially reduced analysis time, with particularly dramatic improvements in theme identification speed. The machine-assisted approach achieved good thematic and sentiment classification accuracy compared to the human reference standard, though human analysis identified an emergent ‘ambiguous’ sentiment category that current AI tools cannot accommodate, highlighting limitations in commercial platforms’ flexibility for inductive analysis.
Conclusions/Significance
Machine-assisted analysis offers substantial efficiency gains with acceptable accuracy trade-offs for large-scale qualitative data analysis. However, human expertise remains essential for capturing nuanced meanings, identifying emergent categories, and providing domain-specific interpretation. This hybrid approach represents a viable methodology for Big Qual research, though current AI tools’ constraints in accommodating emergent classification schemes remain a limitation. Our findings establish benchmarks for future development of more flexible AI systems adapted to qualitative research paradigms.

研究背景
医疗保健研究领域中大质性（Big Qualitative，简称Big Qual）数据的指数级增长，给传统分析方法带来了方法论层面的挑战。本研究以英国皇家麻醉医师学院第七届全国审计项目（NAP7）基线调查为测试案例，对比评估了仅依靠人工分析与使用人工智能（AI）工具辅助的机器分析在处理大规模质性数据集时的有效性。

研究方法与主要结果
本研究开展了一项对比性方法学研究，共分析了5196份关于围手术期心脏骤停经历的自由文本回复。三名研究人员遵循质性研究报告标准（SRQR）建立了人工编码参考标准。随后，我们采用机器辅助分析方案：使用Pulsar进行探索性分析，使用Caplena开展情感与主题分析，并依据STARD-AI报告标准，以人工金标准为参照评估模型性能。性能评估指标包括准确率、精确率、召回率、F1值以及科恩卡帕系数（Cohen’s Kappa），置信区间通过自助重采样法计算得出。
机器辅助分析大幅缩短了分析时长，在主题识别速度上的提升尤为显著。相较于人工参考标准，机器辅助分析在主题与情感分类任务中取得了良好的准确率，但人工分析仍识别出了当前AI工具无法覆盖的涌现式"模糊"情感类别，这凸显了商用平台在归纳式分析灵活性方面的局限性。

结论与研究意义
机器辅助分析可在大规模质性数据分析中实现显著的效率提升，同时在准确率方面的权衡处于可接受范围。但在捕捉细微语义、识别涌现式类别以及开展领域专属解读方面，人工专业知识仍不可或缺。尽管当前AI工具在适配涌现式分类框架方面仍存在约束，但这种混合分析方案为大质性研究提供了一种可行的方法论路径。本研究结果为未来适配质性研究范式的更灵活AI系统开发建立了基准。