Daniela Saderi and Sam Hindle have a shared passion for pushing the boundaries of scientific publishing in order to make research openly available. While pursuing a PhD and postdoc in neuroscience, they came together to establish PREreview, to encourage the use of preprints, speed up dissemination of research, help authors get feedback on their work and democratize the peer review process. We caught up with them to find out how the idea came about and what you can do to join the revolution.
Daniela Saderi (left) Sam Hindle (right)
Can you tell us a bit about PREreview?
PREreview (Post, Read, and Engage with preprint reviews) provides a centralized hub in which participants of scientific journal clubs can share their feedback about preprints with the whole community. Preprints are freely available scientific manuscripts that have not yet undergone editorial peer review, and represent an invaluable wealth of data and knowledge made available at a point in time in which community feedback is useful.
What’s great is you can directly write your preprint review on the website, as it runs on the collaborative writing infrastructure developed by Authorea. PREreview also hosts a series of practical resources to help scientists start preprint journal clubs at their institutions, as well as guidelines on how to write a scientific peer review. We support early-career researchers, as we believe they deserve a chance to a) be trained on how to perform peer review and b) be heard as valuable contributors to scientific progress and evaluation.
How did the idea come about?
We met in April 2017 at a Mozilla Working Open Workshop in Portland where we had the first conversation that inspired us to work on what then became PREreview.
Inspired by the work of ASAPbio and, in particular, Dr. Prachee Avasthi, we decided to tackle the lack of awareness and scarce engagement with preprints in biological sciences by developing resources to make it easy for researchers to provide feedback to preprint authors and share it with the community.
Journal clubs were the perfect, already established way to put scientists in the same room and get them to discuss a scientific article. So, instead of wasting that precious energy used to critically evaluate an already published, peer-reviewed, and static article, why not do it with preprints? Using preprints sets the stage for a more constructive rather than destructive discussion, and therefore leads to useful feedback for the authors.
What has been the biggest challenge so far?
Central to the success of this project is getting researchers to engage with preprints and dedicate time to writing the reviews. While almost everyone seems to recognize the advantages of discussing and reviewing preprints at journal clubs, many see time and motivation as major obstacles.
To start addressing the time concern, we developed guidelines to produce a short and a long version of review: the former gives preprint reviewers a more time-efficient option whilst still producing useful feedback to the authors; the latter leads to a perhaps more complete review and can be used as training for those who wish to gain experience writing a formal scientific peer review.
To address motivation, at least in this initial phase, we applied and were awarded a mini grant from the Mozilla Science Lab (here is our full application if you are curious!), allowing us to provide twenty beta testers with a small amount of money to buy snacks and drinks.
Even though we are extremely grateful for the opportunity to feed our testers, and we believe that reviewing preprints is advantageous to the reviewers, we are aware that the motivation issue deserves a better and more sustainable address. We are advocating for and hopefully soon will help implement new reward schemes to encourage scientific reviews of preprints (see “future plans” below).
What is the status of the initiative at the moment?
We are currently in our beta testing phase and, as part of this process, we have been surveying both the preprint journal club participants and the preprint authors who received feedback. So far, we have been overwhelmed with the level of enthusiasm with which the community has embraced this trial run. When we asked participants if they would be likely to attend a preprint journal club in the future, we heard a resounding “Yes!” And the preprint authors? All of the authors said they were happy to receive the preprint review, that the feedback was useful, and that they would likely incorporate the feedback into their manuscripts; in fact, many did! Also, almost all of them were positive about the review being openly available online, a response we did not expect.
What are your future plans?
We believe in our mission and are committed to scale it to a level to which discussing preprints and writing collaborative reviews becomes the new way of doing journal clubs. We are aware that to achieve this goal we need to tackle non-trivial problems, such as making it worth the time of busy scientists!
In addition to working with the community to develop better and more efficient ways of writing the reviews, we hope to implement some key features that we think are necessary for the success of the model. First, we think it is important for preprint authors to be able to solicit reviews at the time of preprint submission; therefore, we are actively seeking out collaborations with preprint servers to provide this feature. Additionally, once those reviews have been signed, sealed, and delivered to PREreview, we would like to enhance their discoverability by the community and by journal editors. We are working on ways to implement this, so watch this space!
As young scientists ourselves, we appreciate that peer review contributions by ECRs often go unrecognised. By integrating ORCiD authentication, it will allow the reviewing experience to be easily tracked, making reviewers’ contributions more visible. PREreview will therefore provide an opportunity for ECRs to learn how to constructively write a peer review, as well as gain recognition for their role in scholarly publishing. While this idea is still at its early stages, we are encouraged by the enthusiasm and interest that Chief Editors and journal Directors have expressed in integrating and acknowledging PREreviews within their peer review process. We are excited about these new developments, and feel this will provide concrete incentives to motivate ECRs to compile their preprint journal club reviews.
Other features on the horizon are ways for preprint authors to reply to the reviews, and the implementation of a system to directly link the reviews and author’s responses to the preprints themselves as Linked Data.
How can others get involved?
Become a member of PREreview and start writing collaborative reviews today (here is how to get started). By joining PREreview, you can be part of a growing community of preprint reviewers who are contributing to making the scientific publishing system more transparent and making science more reproducible.
We need your feedback to make PREreview as beneficial as possible! If you are a scientist, you can help us by becoming one of our beta testers (here is how). Also, if you know of traditional post-publication journal clubs at your institution, reaching out to the organizers/participants to suggest the integration of preprints would be really useful – bonus points if you direct them to PREreview.org! We have written some tips and email templates to help you with this initial process. Please, don’t hesitate to reach out to us if you would like more info or support.
Follow us on Twitter @PREreview_ and tweet at us using the #prereview and #preprintjc hashtags!
If you are interested in learning more, helping us with developing our project, or want to establish a partnership, please email us at firstname.lastname@example.org.