Dragging: Density-Ratio Bagging

Abstract

We propose density-ratio bagging (dragging), a semi-supervised extension of bootstrap aggregation (bagging) method. Additional unlabeled training data are used to calculate the weight on each labeled training point by a density-ratio estimator. The weight is then used to construct a weighted labeled empirical distribution, from which bags of bootstrap samples are drawn. Asymptotically, dragging is proved to be no worse than bagging and requires no semi-supervised learning assumptions other than $iid$-ness. We show that compared to bagging, the dragging predictor achieves less asymptotic variance, which leads to a smaller MSE. We conduct real experiments on several regression and classification tasks. The performance of dragging, bagging, semi-supervised learning with density-ratio estimator, and basic supervised learning is compared and discussed.