Do you check restaurant reviews before you reserve a table? Many people don’t trust a restaurant or product unless it has favorable reviews and comments online. This doesn’t mean that products without reviews are bad, it’s more that we want to know what other people think of something first. A robust version of this system exists in academia: peer review.
Peer-review aims to identify the strengths and potential flaws in the reported research, and provide feedback to the authors. This process has been important in promoting rigorous and reproducible research. However, the current system of peer review is creaking under the pressure of an exponentially increasing number of manuscripts, combined with a shortage of qualified reviewers. We, a group of early career researchers (ECRs) involved in the eLife Community Ambassadors programme, are passionate about improving the peer review process. Here are our thoughts on how peer-review can be made more robust, fair, and inclusive.Here, we will first describe how the current peer review process works, and then look at what we believe can be improved. Our aim is to highlight potential solutions to make peer review more robust, fair and inclusive.
The traditional peer review system is under pressure. Editors have their own time constraints, and for some journals, editors are working scientists. As such, they often tend to invite established experts, who frequently have a long track-record of publishing in the research field of the submitted manuscript, and are therefore well-known to the editor. While this sounds reasonable, this small pool of "trusted experts" are often overwhelmed by their own scientific commitments. On average, just 20% of scientists perform 69% to 94% of all journal-solicited peer reviews. Since reviewers are not compensated for their efforts, reviewing a paper is not at the top of their priority list when they are trying to keep their lab funded. This can lead to de-prioritising manuscript reviewing, thus delaying communication of research. The pool of peer reviewers also suffers from a lack of gender and geographical diversity, with women and researchers outside of North America and Europe being highly under-represented.
This state of affairs leads to numerous problems and missed opportunities.
- Delayed reviews: As there are only a limited number of experts, combined with an increasing demand for their work, manuscript reviews are often delayed. This hinders the progress of science and could also negatively impact the careers of ECRs.
- Inadequate training opportunities for ECRs: ECRs are at the front lines of science and can benefit from manuscript reviewing by learning to provide professional critique. They should be given more opportunities to peer review, and this could start by reviewing together with their supervisor.
- Ghost reviewing: The time constraint on established expert scientists sometimes leads them to outsource their reviewer tasks to graduate and postdoctoral trainees in their lab. Seeing the previous point, this is not a bad thing, were it not for the fact that ECRs often do not receive credit for their efforts. This process, termed “ghost-peer review”, is driven by a lack of guidelines and a suitable reporting system at scientific journals.
To bridge existing shortages of expert peer reviewers, we propose involving more ECRs in peer review.
Currently, it is not standard practice to include ECRs as peer reviewers. This is most likely driven by the perception that ECRs lack adequate experience in peer review and might therefore not be rigorous in their reports. However, a recent study by eLife found that ECRs peer-reviewed articles at a similar (and even marginally better) standard than established experts. Crucially, ECRs were also slightly quicker (~2.5 days) in submitting their reports and editors valued the high quality of their peer reviews. As ECRs are more likely to be actively performing research, they are potentially more knowledgeable in techniques and methodology. We believe that this would positively impact reproducibility and robustness of research papers in the long term. These findings and ideas need to be emphasized and communicated to editors to correct their existing notions that ECRs may produce peer reviews of lower quality.
We recognize that it can be challenging to involve more ECRs in peer review. It can be hard to identify ECRs with sufficient experience, and there is also limited information for journal publishers and editors to start involving ECRs in their peer review system. Here are three simple suggestions to increase involvement of ECRs as peer reviewers:
- Require editors to include at least one ECR reviewer per manuscript, as successfully trialed by eLife. Our next two points should make it easier for publishers and editors to increase the pool of reviewing ECRs.
- Encourage (or even require) reviewers to report the names of any ECR that helped with the manuscript review. The journal can then include these individuals in their databases for the future.
- Create a system for the nomination of ECRs as peer reviewers from fellow ECRs and expert researchers. A group of eLife Community Ambassadors led by Dr. Steven Burgess has created such a database with the goal to circulate this among journal editors receptive to this initiative (access to this database can be requested here). Furthermore, these two interactive databases includes female and LGBTQ+ scientists that voluntarily signed up to be peer reviewer. Another option is to encourage scientists to nominate at least one ECR reviewer upon submission of their manuscript to a journal.
We believe that the rigour, speed, and inclusivity of existing peer review processes can be significantly improved by including (more) ECRs. In the long term, this would also boost the quality of scientific reviews and publications as ECR peer reviewers would have been trained in providing critical scientific evaluations early in their career.
Confucious said that “moving a mountain begins by carrying away small stones". We believe that increased inclusion and better recognition of ECRs will significantly improve the current state of peer review.
Written by Alexander van Vliet (@Alex_van_Vliet), Renuka Kudva (@biologigal3), Ewoud Compeer (@ewoudcompeer), Asmaa Elkenawi (@asmaakenawy), Nele Haelterman (@HaeltermanNele), Patricia Resa-Infante, eLife Community Ambassadors, Sarvenaz Sarabipour (@SSarabipour) & Lotte de Winde (@lotte_dewinde), Early Career Advisors to eLife.
