ERC Advanced Grant for Peter Grünwald for research on a revolutionary statistical theory

Peter Grünwald, senior researcher in the Machine Learning group of CWI and part-time full professor of Statistical Learning at Leiden University has been awarded an ERC Advanced Grant for his research on developing a new, revolutionary theory of statistics.

Publication date
11 Apr 2024

Grünwald will use the prestigious top grant of 2.5 million euros for the development of a new, more robust and flexible theory of statistics based on the novel concept of e-value. This will lead, for example, to more reliable methods for determining whether or not scientific results are statistically significant. In short: the e-value is the new p-value.

Replication crisis, p-values and confidence intervals

Whether experimental results are significant and did not arise by chance is traditionally determined by p-values, a methodology largely developed in the 1930s. Later on, this was extended with confidence intervals – error bars around graphs that were, for example, widely used by the RIVM during the COVID pandemic briefings to express uncertainty about the number of infected people.

“However, p-values and confidence intervals are inappropriate for the way science is done today”, says Grünwald. “Both were invented for one-shot research. It is incorrect to add new data and recalculate the p-value or interval, but that is what happens all the time in modern research. For example, if an experimental outcome of a drug trial is promising but not significant, scientists often repeat the experiment by adding more participants. If one repeats this process, then, by pure chance, the result eventually will have a small p-value and look significant.” In practice, this leads to overly many false-positive scientific conclusions - a phenomenon that has become known as the ‘replication crisis’.

Peter Grunwald in front of whiteboard
Peter Grünwald in his office at CWI.


According to Grünwald, in practice we need much more flexibility than traditional methods can offer. Essentially, all such methods – p-value-based but also for example the popular “Bayesian” techniques – require the researcher to specify in advance things that one doesn’t want to specify: the total number of participants in a study, the number of studies, the costs and benefits of making certain decisions, what one would do in case things don’t go as planned (e.g. one unexpectedly runs out of money), and so on. This makes them highly inflexible and susceptible to improper use.

Grünwald’s research proposal aims for a revolutionary new statistical theory -based on the e-value- in which all such data–collection and decision-aspects may be unknown in advance. Grünwald: “The E-value calculates how much evidence you have against any given hypothesis. It is a value between zero and infinity. The higher the value, the higher the evidence that the outcomes are significant (“the medication works”, “the phenomenon did not arise by chance”). In practice, in one-shot situations, significance is usually associated with a p-value smaller than 0.05. This corresponds to an E-value larger than 20, so you can say that if the E-value is larger than 20, the outcome can be accepted as significant. But you may now add data as long as you like, stop whenever you like and re-calculate the E-value, and still maintain the interpretation that an E-value larger than 20 means that the results are significant. Similarly, you can make e-value based confidence intervals that are anytime-valid: they are valid irrespective of when or how often you look at them.”

E-value-based methods originated only in 2019. By now, they can be used to solve several simple yet important statistical problems. Grünwald played an essential role in their rapid development. Grünwald: `Part of the proposal is to develop e-value methods for more complex applications. But mainly I will establish a general mathematical theory of flexible statistics. I am extremely happy that the EU has decided to fund this radical proposal. It will make statistics both safer and more flexible – allowing us to get more reliable conclusions based on less data.

About Peter Grünwald

Peter Grünwald is a senior researcher in the Machine Learning group at CWI in Amsterdam, which he headed from 2016 to 2023. At the moment he is a member of CWI’s Management Team. He is also part-time (0.2 fte) full professor of statistical learning at the Mathematical Institute of Leiden University. He has received numerous grants (including NWO VIDI and VICI), has co-chaired major international machine learning conferences and is co-recipient of the Van Dantzig prize (2010), the highest Dutch award in statistics and operations research. He holds a deep interest in the foundations of statistics, and regularly gives talks to both expert and non-expert audiences about the problems and difficulties surrounding traditional statistical methods.

About the ERC Advanced Grant

The ERC Advanced Grant funding is amongst the most prestigious and competitive EU funding schemes (in the 2023 round, only 10 grants were funded in the field of mathematics in all of the EU on a total of 255 awarded grants). It provides researchers with the opportunity to pursue ambitious, curiosity-driven projects that could lead to major scientific breakthroughs. They are awarded to established, leading researchers with a proven track-record of significant research achievements over the past decade. The funding will enable these researchers to explore their most innovative and ambitious ideas.

More information