The perils of p-hacking and the promise of pre-analysis plans

Add to your list(s) Download to your calendar using vCal

• Jacob Stegenga (with Zoë Hitzig) (Department of History and Philosophy of Science)
• Wednesday 18 October 2017, 13:00-14:30
• Seminar Room 2, Department of History and Philosophy of Science.

If you have a question about this talk, please contact .

P-hacking involves the manipulation of data to find a statistically significant result. Many claim that p-hacking is a problem in science, especially in the medical and social sciences, while others deny this. The problem with p-hacking is usually articulated from a frequentist perspective. In this paper we articulate the epistemic peril of p-hacking using Bayesian confirmation theory and model selection theory, which we then draw on to explain the arguments on both sides of the debate. This requires a novel understanding of Bayesianism, since a standard criticism of Bayesian confirmation theory is that it cannot accommodate the influence of biased methods. A methodological device widely used to mitigate the peril of p-hacking is a pre-analysis plan. Some say that following a pre-analysis plan is epistemically meritorious while others deny this, and in practice pre-analysis plans are often violated. We use the formal groundwork developed earlier in the paper to resolve this debate, offering a modest defence of the use of pre-analysis plans. In the longer run our ambition is to use this approach to make sense of scenarios in which scientists depart from pre-analysis plans.

This talk is part of the CamPoS (Cambridge Philosophy of Science) seminar series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• CamPoS (Cambridge Philosophy of Science) seminar
• Cambridge talks
• Department of History and Philosophy of Science
• Featured lists
• History and Philosophy of Science long list
• Philosophy Events
• Seminar Room 2, Department of History and Philosophy of Science
• Trust & Technology Initiative - interesting events
• hc446
• jer64's list

Note that ex-directory lists are not shown.