Dynamic Decision Making With Individualized Variable Selection

Physicians today have access to a variety of tests for diagnosing and prognosticating medical conditions. Ideally, they would apply a high-quality prediction model utilizing all relevant features to facilitate appropriate decision-making (e.g., treatment selection; risk assessment). However, some of these features incur additional costs and are not readily available to patients and physicians. In practice, predictors are typically gathered sequentially, i.e., physicians continually evaluate information dynamically until sufficient information is acquired to make a reasonable confidence decision. More importantly, the prospective information to collect may differ for each patient and depend on the predictor values already known. In this paper, we design a novel adaptive prediction rule to determine the optimal order of acquiring features in predicting a clinical outcome of interest. The objective is to maximize prediction accuracy while minimizing the cost associated with measuring features for individual subjects. To achieve this, we employ reinforcement learning, where the agent decides the best action at each step: either making a final clinical decision or continuing to collect new predictors based on the current state of knowledge. Extensive simulation studies have been conducted to evaluate the efficacy of the proposed strategy. Additionally, real examples are presented to illustrate the practical utility.

当前临床医师可借助多种检测手段开展病症的诊断与预后评估。理想情况下，医师应采用纳入全部相关特征的高质量预测模型，以辅助制定合理决策（如治疗方案选择、风险评估）。但部分特征的检测会产生额外成本，且患者与医师未必能轻易获取这些特征。实际操作中，预测特征往往按序采集：医师会动态持续评估已有信息，直至获取足够信息以做出置信度合理的临床决策。更为关键的是，每位患者需采集的前瞻信息可能存在差异，且取决于已获知的预测特征取值。本文设计了一种全新的自适应预测规则，用于确定针对目标临床结局开展预测时的最优特征采集顺序。该方法的优化目标为在最大化预测准确率的同时，最小化针对个体受试者测量特征所需承担的相关成本。为实现这一目标，我们采用强化学习（reinforcement learning）方法，由智能体在每一步决策中选择最优动作：要么直接做出最终临床决策，要么基于当前已掌握的知识状态继续采集新的预测特征。我们通过大量仿真实验评估了所提策略的有效性，此外还结合实际案例展示了该方法的实际应用价值。