Bayesian Additive Regression Trees ( BART )
Bayesian approach to nonparametric function estimation using Regression trees. Regression Decision trees rely on recursive binary partitioning of predictor space into a set of hyper-rectangles to approximate some unknown function. BART is defined by Two Major concepts : Sum-Of-Trees Model Regularized Priors Sum-Of-Trees Model Drawbacks of Single Decision Tree: Over-emphasize the interactions between the Variables. Have Difficulty finding Linear relationships. Alternative: Fit many small models ( sequential weak learners ) using Back-Fitting Algorithm where, T → Binary Regression Tree M → Terminal Node parameters Back-Fitting Algorithm : Fit a small regression tree Get fitted values from that tree Estimate the Residuals = True values - Observed Values Fit next Regression tree on above residuals Process continues till m number of Trees are fitted. Regularized Priors Any Tree based Model can easily overfit the data. Boosted Regression Trees limit the tree depth ...