Dr. Jon Tennant on an Introduction to Open Peer Review Process

The Open Access Week is a yearly event which raises awareness and promotes the free, immediate, and online access to scientific research through Open Access (OA) publishing. For Open Access Week 2018, running from 22-28 October 2018, Eurodoc will publish an article each day on various aspects of OA from international experts.

In this fourth article of the series, Jon Tennant, Paleontologist and Founder of the Open Science MOOC, discusses opening up research via the scholarly peer review system.

Peer review is one of the main pillars of modern science. It is part of the critical process that decides what knowledge enters the scholarly record, and ultimately the collective knowledge of mankind. It also decides the fate and the future of researchers, as publishing of peer reviewed research articles is often crucial for career advancement.

However, what is becoming increasingly clear is that often the common ideal of peer review as a method of filtering out ‘bad’ research is distinct from its actual implementation. Peer review has often been heavily criticised as a result of this, for unreliability and inconsistency, often causing unnecessary delays to communicating research, being expensive to manage and biased, often without accountability, and often also being wasteful in that much of the valuable discussion within is typically lost. Furthermore, it is often criticised for failing to stop ‘bad’ research from being published, and the proliferation of scientific misinformation.

Now, these are not systemic issues across the entire publishing system by any means. However, enough evidence exists to suggest that these problems pervade peer review differently across journals, communities, and depending on the management of the process. So, what is being done to help resolve some of these issues?

Open Peer Review Takes Many Forms

Open Peer Review (OPR) is part of an ongoing transformation happening in the world of science and scholarly publishing, more widely known as Open Science or Open Scholarship or Open Research. However, OPR is not a simple concept to understand, and often it is used to reflect a myriad of different aspects of the evolving peer review process.

In a systematic review of ‘Open Peer Review’, it was found that 122 different definitions existed! From this complexity, it was possible to distil 7 core traits that define OPR:

1. Open identities, where authors and reviewers are reciprocally identified

2. Open reports, where the review reports themselves are published

3. Open participation, where the wider community can contribute to the process

4. Open interaction, where review takes the form of a discussion

5. Open pre-review manuscripts, where manuscripts are available prior to any formal review process

6. Open final-version commenting, where the final published manuscript can be reviewed further

7. Open platforms, where reviews are facilitated by a service external to the journal itself

What this means is that when we are discussing OPR, we have to be very clear about which aspect we are talking about to make sure that we are all having the same discussion!

It is worth being aware that things are also changing very rapidly with OPR. For example, recently Jessica Polka and ASAPbio launched an open letter, now signed by 100s of journals, calling for the widespread publication of peer review reports as a way of injecting transparency into the process. There are now dozens of tools, platforms, and services that exist around journals and peer review to help improve the process in a variety of ways.

Open Peer Review is Not a Solution to All Our Problems

Now, OPR is not some magic bullet that is going to solve all issues within the overall system of peer review overnight. For example, there is little informed consensus on the impact of double-blind reviews (reciprocal concealment of identities) versus fully revealed identities, and often the limited available evidence is conflicting or overlapping.

One problem with this is that getting data on the functionality of peer review, and the differential impact that it has across communities, is often very difficult. This is because, historically, it has been a closed and secretive process, which actually makes any sort of detailed or systemic understanding of peer review very difficult.

The Future of (Open) Peer Review

The hope is that with increased transparency in the peer review process, we can gain a greater understanding of how it works (or does not work well), and use this to improve and streamline processes across journals and communities, with bespoke solutions as needed. At the moment, accountability within the present mostly closed system is almost completely absent, and we should all be seeking ways to improve that, based on the technologies and systems that we have readily available to us now.

We also need to stop perpetuating the fallacy that peer review is some magical process that differentiates scholarly research in ‘true’ and ‘false’, as this creates all sorts of issues further down the line with the media and wider public engagement with science. What is clear is that there is no single unified process of ‘peer review’, and it is a diverse, inherently subjective, and well, human activity. Openness is a way of injecting some much needed experimentation into the process in the hope that it can be improved for those engaged in peer review, as well as the wider society who benefits from the use of peer reviewed research.

As scholars, we should not be afraid to innovate around peer review, but let us make sure that when we do, it is based on an honest and transparent evaluation of present processes. And we cannot have this without exposing the closed, black box of peer review.

Further reading: see Survey on Open Peer Review and The State of the Art in Peer Review!

This article is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) License.