Papers with open peer-review reports are less likely to be retracted
Skip to main content
Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain<br>the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in<br>Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles<br>and JavaScript.
Advertisement
Bluesky
Credit: Javier Zayas/Getty<br>Papers with publicly available peer-review reports are less likely to be retracted than are those without public reports, a study has found. The authors say this could, in some cases, reflect greater confidence among researchers who choose to make the review process transparent.<br>The analysis, posted on preprint server MetaArXiv earlier this year1, examined 116,359 papers that have been published since May 2019 in open-access journals published by Public Library of Science (PLOS), based in San Francisco, California. PLOS journals give authors the choice to publish peer-review reports alongside their paper once the manuscript is accepted. The authors of 46,920 of the analysed papers — or slightly more than 40% — opted in.<br>One hundred and five of the papers with open referee reports were later retracted, compared with 321 of the papers for which the peer-review process remained closed.<br>The study does not attempt to explain the association between open peer review and lower retraction rates, but in unpublished follow-up work described to Nature Index, study co-author Matt Spick, a research-integrity specialist at the University of Surrey in Guildford, UK, explored whether reviewer sentiment might offer clues.<br>One possibility was that PLOS authors, who choose whether to publish reviewer reports only after acceptance, might be more likely to opt in when reviews are favourable. To test this, Spick compared reviewer sentiments in PLOS journals with publicly available reports for papers in journals published by BMC, based in London. (The BMC portfolio is owned by Springer Nature, which publishes Nature.) He found no difference between the two data sets, suggesting that the lower retraction rate is unlikely to be explained by authors selectively publishing more-positive reviews.<br>Instead, Spick and Adrian Barnett, a statistician at the Queensland University of Technology in Brisbane, Australia, and co-author of the MetaArXiv preprint, hypothesize that researchers who are confident in their work and who broadly embrace open-science practices might be more likely to choose open peer review. As a result, the decision to opt for open peer review could itself serve as an indicator of article quality.<br>Jessica Polka, who develops biology teaching materials at MIT Open Learning, a research and education initiative in Boston, Massachusetts, agrees with this interpretation, saying that opting in to making referee reports public could be “a very useful signal to the reader that the authors are confident in the findings”.<br>Open or closed?<br>There are many variables at play that make it difficult to explain the results of the study one way or the other.<br>For example, the study does not investigate why some of the papers in the PLOS data set were retracted. Other than for misconduct, retractions can occur owing to honest mistakes and a lack of reproducibility, which would not affect an author’s confidence in their work when they are deciding whether to opt in to open peer review.<br>The study also did not evaluate whether the peer-review models of the PLOS journals, such as ‘single-blind’ (referees know authors’ identities but not vice versa) or ‘double-blind’ (neither authors nor reviewers know each other’s identities), affected the proportion of authors opting in to open reviews.<br>However, the study did find that researchers from certain countries are more likely to opt in, which could provide some insight into where open-review practices are most favoured.<br>Authors from Saudi Arabia, South Korea, Pakistan, Poland and China, for example, were the least likely to opt in, whereas those in the United Kingdom, the Netherlands, France and Ethiopia were among the most likely to do so.<br>Cultural norms — including how strongly open science and transparency are encouraged by institutions and funders — can shape these decisions. National publishing mandates can also affect how likely researchers are to opt in, Barnett says.<br>Publishing practices can also be heavily influenced by the policies of major research funders and charities, such as Wellcome in London and the Gates Foundation in Seattle, Washington, which actively promote open science and the sharing of peer-review reports.<br>Author mobility could be another factor. The study found that authors who used an institutional e-mail address ending in .edu were less likely to choose open peer review than were those who used a personal e-mail account. Barnett told Nature Index that he...