Novel aggregate deletion/substitution/addition learning algorithms for recursive partitioning

<b>Motivation</b>: Many complex diseases are caused by a variety of both genetic and environmental factors acting in conjunction. To help understand these relationships, non-parametric methods that utilize aggregate learning have been developed such as <i>Random Forests</i> and <i>Conditional Forests</i>. Molinaro et al. (2010) described a powerful, single model approach called <i>partDSA</i> that has the advantage of producing interpretable models. <b>Methods</b>: We propose two extensions to the <i>partDSA</i> algorithm called <i>Bagged partDSA</i> and <i>Boosted partDSA</i>. These algorithms achieve higher prediction accuracies than individual <i>partDSA</i> objects through aggregating over a set of independent <i>partDSA</i> objects. Further, by utilizing <i>partDSA</i> objects in the ensemble, each base learner creates decision rules using both ‘and’ and ‘or’ statements, which allows for natural logical constructs. We also provide four variable ranking techniques that aid in identifying the most important individual factors in the models. <b>Results</b>: In the regression context, we compared <i>Bagged partDSA</i> and <i>Boosted partDSA</i> to <i>Random Forests</i> and <i>Conditional Forests</i>. Using simulated and real data, we found that <i>Bagged partDSA</i> had lower prediction error than the other methods if the data were generated by a simple logic model, and that it performed similarly for other generating mechanisms. We also found that <i>Boosted partDSA</i> was effective for a particularly complex case. Taken together these results suggest that the new methods are useful additions to the ensemble learning toolbox. We implement these algorithms as part of the partDSA R package.