Ensemble Learning with Bayesian Additive Regression Trees (Day 1 of 2)

Presented by Professor Robert E McCulloch from the School of Mathematical and Statistical Sciences at Arizona State University and Associate Professor Rodney Sparapani from the Division of Biostatistics at the Medical College of Wisconsin.

Abstract: Modern computing power has led to breakthroughs in our ability to learn high-dimensional, complex relationships from data.  Recently, the two key modeling approaches in this arena are deep learning with neural networks and ensemble learning with binary trees.  Deep learning is the best currently-known method of prediction where all of the covariates are of the same type, i.e., they are all pixels or words or audio waves, etc.  Ensemble learning is the best currently-known method with respect to out-of-sample predictive performance for tabular data where all of the covariates are of different types, i.e., age, sex, weight, etc.  A collection of machines (in our case trees) are fit simultaneously that form the basis of an ensemble's aggregate prediction with superior performance to any single machine's fit.  In this workshop, you will learn a Bayesian approach to modeling with ensembles of trees called Bayesian Additive Regression Trees (BART).  The Bayesian approach allows for a Markov chain Monte Carlo stochastic exploration of the model space, uncertainty quantification, and Bayesian posterior inference.  BART is a modern nonparametric approach that exploits the elegance and convenience of the Bayesian conceptual toolkit. We can employ BART for outcomes of different types: continuous, dichotomous, categorical and time-to-event.  Furthermore, we will demonstrate BART's effectiveness in a wide range of regression applications: marginal effects, variable selection, monotonicity, outlier detection and time-to-event extensions like competing risks or recurrent events. This is a lot of material to cover in two days.  Therefore, we will be providing an overview with a wealth of materials for the attendees to self-explore afterward.  You can find the slides themselves online at the following links.  Rob’s slides can be found at https://www.rob-mcculloch.org/BART-wo.... And Rodney’s: https://community.amstat.org/wisconsi....