Dynamic Decision Making With Individualized Variable Selection

Physicians today have access to a variety of tests for diagnosing and prognosticating medical conditions. Ideally, they would apply a high-quality prediction model utilizing all relevant features to facilitate appropriate decision-making (e.g., treatment selection; risk assessment). However, some of these features incur additional costs and are not readily available to patients and physicians. In practice, predictors are typically gathered sequentially, i.e., physicians continually evaluate information dynamically until sufficient information is acquired to make a reasonable confidence decision. More importantly, the prospective information to collect may differ for each patient and depend on the predictor values already known. In this paper, we design a novel adaptive prediction rule to determine the optimal order of acquiring features in predicting a clinical outcome of interest. The objective is to maximize prediction accuracy while minimizing the cost associated with measuring features for individual subjects. To achieve this, we employ reinforcement learning, where the agent decides the best action at each step: either making a final clinical decision or continuing to collect new predictors based on the current state of knowledge. Extensive simulation studies have been conducted to evaluate the efficacy of the proposed strategy. Additionally, real examples are presented to illustrate the practical utility.