Uncertainty estimation for gradient boosting models based on Gaussian graph model and natural gradient

Muyi Chen; Daling Wang

doi:10.1117/12.2675209

25 May 2023 Uncertainty estimation for gradient boosting models based on Gaussian graph model and natural gradient

Muyi Chen, Daling Wang

Proceedings Volume 12636, Third International Conference on Machine Learning and Computer Application (ICMLCA 2022); 1263633 (2023) https://doi.org/10.1117/12.2675209
Event: Third International Conference on Machine Learning and Computer Application (ICMLCA 2022), 2022, Shenyang, China

Abstract

We study the multi-output probability regression prediction and uncertainty quantification problem in multi-output regression prediction with gradient boosting models, and propose an uncertainty estimation method based on Gaussian Graph Model (GMM) constraint and natural gradient. Under the assumption of Gaussian distribution, we employ GMM to model the conditional independence relationship of the outputs, and propose a mixed parameterization to represent the subfamily of Gaussian distributions defined by the graph model. We set this mixed parameters as the learning target of the gradient boosting model, and use natural gradient in Riemann space that based on Fisher information matrix (FIM) instead of the vanilla gradient to be the steepest descent direction. The formula to compute FIM and natural gradient for the subfamily defined by GMM is deduced. Experiments show that the proposed method is able to incorporate the priors on the outputs to optimize the search space, therefore, lead to better uncertainty estimation results on multi-output regression for gradient boosting models.

Citation Download Citation

Muyi Chen and Daling Wang "Uncertainty estimation for gradient boosting models based on Gaussian graph model and natural gradient", Proc. SPIE 12636, Third International Conference on Machine Learning and Computer Application (ICMLCA 2022), 1263633 (25 May 2023); https://doi.org/10.1117/12.2675209

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available