Chebyshev-Cantelli PAC-Bayes-Bennett Inequality for the Weighted Majority Vote ---- Yi Shan Wu
- https://wsc.project.cwi.nl/ml-reading-group/events/tba-yi-shan-wu
- Chebyshev-Cantelli PAC-Bayes-Bennett Inequality for the Weighted Majority Vote ---- Yi Shan Wu
- 2021-10-14T14:00:00+02:00
- 2021-10-14T15:00:00+02:00
- When Oct 14, 2021 from 02:00 PM to 03:00 PM (Europe/Amsterdam / UTC200)
- Where L016
-
Add event to calendar
iCal
Joint work with Andrés R. Masegosa, Stephan S. Lorenzen, Christian Igel and Yevgeny Seldin
Abstract:
We present a new second-order oracle bound for the expected risk of a weighted majority vote. The bound is based on a novel parametric form of the Chebyshev-Cantelli inequality (a.k.a. one-sided Chebyshev’s), which is amenable to efficient minimization. The new form resolves the optimization challenge faced by prior oracle bounds based on the Chebyshev-Cantelli inequality, the C-bounds [Germain et al., 2015], and, at the same time, it improves on the oracle bound based on second order Markov’s inequality introduced by Masegosa et al. [2020]. We also derive the PAC-Bayes-Bennett inequality, which we use for empirical estimation of the oracle bound. The PAC-Bayes-Bennett inequality improves on the PAC-Bayes-Bernstein inequality by Seldin et al. [2012]. We provide an empirical evaluation demonstrating that the new bounds can improve on the work by Masegosa et al. [2020]. Both the parametric form of the Chebyshev-Cantelli inequality and the PAC-Bayes-Bennett inequality may be of independent interest for the study of concentration of measure in other domains.