What's the next frontier for improving psychological research? | BPS

Join us<br>Search<br>Search

Sign in

My BPS

Search

Join us

Research, Research Ethics<br>What's the next frontier for improving psychological research?

Isaac Handley-Miner, PhD, on the work of ‘Transparent Replications’.<br>07 May 2026

Share this page

Login as a British Psychological Society member , at the top of this page, for access to author-narrated audio.<br>I was forged in the crucible of psychology's replication crisis.

I took my first psychology course in 2012, a year after a paper earnestly claimed evidence of extrasensory perception (ESP) and another paper not-so-earnestly demonstrated how to make people younger by playing them a Beatles tune.

I worked in my first psychology lab in 2013, a year after Daniel Kahneman penned a letter to social priming researchers in which he stated: 'I see a train wreck looming.'

I began my first full-time position in psychology in 2016, a year after the Open Science Collaboration found that less than half of 100 papers drawn from three top psychology journals successfully replicated.<br>My formative years in the field were shaped by watching psychology's skeletons dragged from the closet and paraded across the pages of the Guardian, the New York Times, and the New Yorker.

The skeletons go by many names. Data mining. Data dredging. Data fishing. Fishing for significance. Significance chasing. Undisclosed analytic flexibility. Or, the preferred shibboleth of the replication crisis: p-hacking.

In brief, these terms refer to the process of selecting or reporting statistical tests based on the desirability of the results they produce. To get published, psychologists are incentivised to find statistically significant results (p

As a researcher forged during the replication turmoil, it was obvious how p-hacking could damage the scientific enterprise. Moreover, p-hacking seemed not only prevalent but sometimes hard to avoid, manifesting as an unconscious bias luring researchers through a garden of forking paths.

Was this bias toward 'statistical significance' something our field could overcome? And if p-hacking were curbed, what would that leave as the next frontier?

I've recently had the opportunity to explore these questions.

Evaluating newly published papers

Fast-forward a decade from my first full-time position in psychology. I now direct Transparent Replications, a project designed to celebrate high-quality psychological research and shift researchers' incentives toward replicable, reliable methods.

Our team has embarked on a new kind of replication project. We randomly select recently published articles from five of the most influential journals that publish psychology research: Science, Nature, Proceedings of the National Academy of Sciences, Psychological Science, and Journal of Personality and Social Psychology. We then complete direct replications of a pivotal study in the paper and evaluate the study on three dimensions. (Given resource constraints, we currently select only articles whose studies can be run online with an adult population.)

First, we assess transparency. Are the study's data, materials, and analysis code available?<br>Was the study preregistered? Was the preregistration followed? Next, we assess replicability. Do our results match those reported in the original study? Finally, we assess clarity, a new evaluation criterion we created to assess the extent to which the paper's claims match what the studies actually demonstrate. Papers receive a score out of five stars on each of these three dimensions, and we write a report on each study we replicate. You can check out our reports here.

Unlike many replication projects, we target new papers. We see a few advantages to this approach. First, if a finding isn't reliable, we correct the scientific record quickly, and if it is reliable, we boost its credibility. Second, if psychologists know their work might be evaluated soon after publication, they have a greater incentive to follow good practices – ensuring transparency, checking for errors, appropriately calibrating claims, and so forth. Finally, targeting new papers reveals the current state of the field and lets us investigate a crucial question: what exactly is going right and what exactly is going wrong, right now?

We're in the early stages of this endeavour, but we already have some surprising findings.

So far, we've attempted to replicate 15 studies. How many of those do you think replicated?

For context, a large-scale replication project published in early April, 2026 found that 49 per cent of the 58 psychology papers it targeted successfully replicated. So, if you guessed somewhere around 50 per cent, that would be justified.

Actually, we were pleasantly surprised to find that 12 of the 15 studies (80 per cent) fully or mostly replicated (of the 70 findings in these studies we tested, 84 per cent replicated). If the replicability rate of new articles in top psychology...