Physician characteristics associated with proper assessment of overstated conclusions in research abstracts: A secondary analysis of a randomized controlled trial

Objectives
Little is known about the physician characteristics associated with appraisal skills of research evidence, especially the assessment of the validity of study methodology. This study aims to explore physician characteristics associated with proper assessment of overstated conclusions in research abstracts.
Design
A secondary analysis of a randomized controlled trial.
Setting and participants
We recruited 567 volunteers from the Japan Primary Care Association.
Methods
Participants were randomly assigned to read the abstract of a research paper, with or without an overstatement, and to rate its validity. Our primary outcome was proper assessment of the validity of its conclusions. We investigated the association of physician characteristics and proper assessment using logistic regression models and evaluated the interaction between the associated characteristics and overstatement.
Results
We found significant associations between proper assessment and post-graduate year (odds ratio [OR] = 0.67, 95% confidence interval [CI] 0.49 to 0.91, for every 10-year increase) and research experience as a primary investigator (PI; OR = 2.97, 95% CI 1.65 to 5.34). Post-graduate year and PI had significant interaction with overstatement (P = 0.015 and < 0.001, respectively). Among participants who read abstracts without an overstatement, post-graduate year was not associated with proper assessment (OR = 1.04, 95% CI 0.82 to 1.33), and PI experience was associated with lower scores of the validity (OR = 0.58, 95% CI 0.35 to 0.96).
Conclusion
Physicians who have been in practice longer should be trained in distinguishing overstatements in abstract conclusions. Physicians with research experience might be informed that they tend to rate the validity of research lower regardless of the presence or absence of overstatements.
Trial registration
UMIN000026269.

研究目标
目前学界对与科研证据评价能力（尤其是研究方法学有效性评估能力）相关的医师特征所知甚少。本研究旨在探索与科研摘要中夸大性结论的正确识别相关的医师特征。
研究设计
本研究为一项随机对照试验（randomized controlled trial, RCT）的二次分析。
研究场景与研究对象
本研究从日本基层医疗协会招募了567名志愿者作为研究对象。
研究方法
将参与者随机分组，令其阅读带有或不带有夸大性结论的科研论文摘要，并对摘要的有效性进行评分。本研究的主要结局指标为对研究结论有效性的正确评估。我们采用logistic回归模型分析医师特征与正确评估之间的关联，并评估了相关特征与夸大性结论之间的交互作用。
研究结果
本研究发现，正确评估能力与医师毕业后执业年限（每增加10年，比值比（odds ratio, OR）=0.67，95%置信区间（confidence interval, CI）：0.49~0.91）及作为主要研究者（primary investigator, PI）的科研经历（OR=2.97，95%CI：1.65~5.34）存在显著关联。毕业后执业年限与PI经历均与夸大性结论存在显著交互作用（P值分别为0.015和<0.001）。在阅读无夸大性结论的摘要的参与者中，毕业后执业年限与正确评估能力无显著关联（OR=1.04，95%CI：0.82~1.33），而PI经历则与更低的有效性评分显著相关（OR=0.58，95%CI：0.35~0.96）。
研究结论
对于行医年限较长的医师，应开展针对其识别摘要结论中夸大性内容能力的培训。有科研经历的医师可能需要知晓：无论摘要是否存在夸大性结论，他们往往会对研究有效性给出更低的评分。
试验注册
UMIN000026269.