Valid sequential inference on probability forecast performance
- https://wsc.project.cwi.nl/ml-reading-group/events/valid-sequential-inference-on-probability-forecast-performance
- Valid sequential inference on probability forecast performance
- 2021-09-10T14:00:00+02:00
- 2021-09-10T15:00:00+02:00
- When Sep 10, 2021 from 02:00 PM to 03:00 PM (Europe/Amsterdam / UTC200)
- Where L016
-
Add event to calendar
iCal
Johanna Ziegel is professor of statistics at the University of Bern
Abstract
Probability forecasts for binary events play a central role in many applications. Their quality is commonly assessed with proper scoring rules, which assign forecasts a numerical score such that a correct forecast achieves a minimal expected score. In this paper, we construct e-values for testing the statistical significance of score differences of competing forecasts in sequential settings. E-values have been proposed as an alternative to p-values for hypothesis testing, and they can easily be transformed into conservative p-values by taking the multiplicative inverse. The e-values proposed in this article are valid in finite samples without any assumptions on the data generating processes. They also allow optional stopping, so a forecast user may decide to interrupt evaluation taking into account the available data at any time and still draw statistically valid inference, which is generally not true for classical p-value based tests. In a case study on postprocessing of precipitation forecasts, state-of-the-art forecasts dominance tests and e-values lead to the same conclusions.
To attend the meeting remotely, join us on:
https://cwi-nl.zoom.us/j/83113639352?pwd=U2NtVC9kWmJkY3NOOGdTMUVSVHorUT09
[Meeting ID: 831 1363 9352 Passcode: 551343]