by Sarvenaz Sarabipour
Imagine spending months trouble-shooting a technique when another researcher has already done so and recorded in a lab book or a private computer in the lab. Lenny Teytelman was frustrated by this via first-hand experience in the lab as a postdoc. Upon exploring the web, he realized that the scientific community needs an effective way to share detailed protocols and improved techniques with the research community at large. During completing his postdoctoral work at Massachusetts Institute for Technology, he co-founded protocols.io in 2012 to create a free, central, up-to-date, crowd-sourced protocol repository [1,2].
The motivations for his impressive campaign were as he puts it: 1) Method development is important but hard: Tweaking and adopting existing methods is laborious but often unrewarded. Knowledge around protocol optimization is routinely lost in paper notebooks and poorly reported in published papers. 2) Organizing and sharing protocols saves time: By sharing methods on protocols.io in your company or lab, you make sure that knowledge stays with the team, even if the scientist leaves. It also helps to keep the protocols up-to-date and make sure everyone is using the most recent version. 3) Publishing protocols increase trust. When you provide a citable detailed protocol with your publication, you get credit for the method development and increase the reproducibility and reliability of your paper.
Protocols.io submissions have since grown exponentially [3]. Today in excess of 450 Journals recommend prior/concurrent submission of protocols to Lenny’s platform. The platform accepts protocols from all research methods in all disciplines in areas as diverse as Molecular Biology, Clinical trials, Physiology, Device setup, Computational workflows, Microscopy, Teaching and Geosciences. Using their impactful platform, Lenny hopes to change the culture of performing and curating research within labs, within specific fields and in the global research community at large.
Having myself spent a long time refining molecular biology workflows and finessing fluorescence microscopy techniques as a doctoral student, I reached out to Lenny Teytelman to learn more about his increasingly popular platform, here’s what I learned:
SS: When did the platform launch online?
LT: protocols.io launched in 2014.
SS: What types of protocols are hosted?
LT: Our platform hosts two types of protocols: private and public protocols. Every protocol is originally a private one, upon publication of the research article corresponding to the protocol, the protocol becomes public. Only the public protocols receive a Digital Object Identification (DOI). Over 90% of all protocols are academic protocols, but companies also submit protocols and they are almost all private.
SS: What has the reception been so far among scientists across the globe?
LT: Our platform used to have 5-20 protocol submissions per month. Currently has ~100-150 new public submission per month. Approximately 800-850 private protocols are submitted now each month, many of the private protocols are in waiting to become public after journals accept the corresponding research paper, so journals have a lot of control over the timing of protocols going public. March 2018 saw 100 public and 400 private protocol submissions. October 2018 saw 956 new protocols, of which 834 were private. These were created by 549 different people. Currently has ~ 3,998 (3328 unique ones plus 670 versions) total public protocols, with some protocols having multiple version uploaded. Total protocols: 15,814 (11,819 private). Protocols.io currently has ~50,000 views each month.
SS: How has the platform been greeted by journals and editors?
LT: Originally only 10 journals were recommending our platform to their authors, now >450 journals recommend (added to author guidelines) protocols.io submission to authors, including eLife, all the PLOS Journals, AACR journals, F1000, genetics from GSA, and many more.
SS: As the CEO of protocols.io, what is the most important metric for you Lenny?
LT: How many people are creating protocols every month.
SS: What are some of the coolest features of protocols.io?
LT: 1) The protocol and reagent analytics tools similar to Google analytics that we offer. When going through our analysis one can make a list: e.g. CRISPR list to see how many protocols are submitted and checking other stats. 2) The stats also provide information on “forking”: which is: users creating their own version of a protocol. For this reason, forks are more valuable than “views”. protocols.io can not track all citations of their protocols: since subscription-only journals are not accessible. Hence only Open access journals are assessable. 3) Apart from detailed protocols, protocols.io features Q & A with the authors which can be very useful for fellow researchers! 4) There is also an App we have developed where a researcher can run the protocol, this way protocols.io can track the number of researchers that take the App to the lab bench! But as some researchers may not want to contaminate their phone in the lab, protocols.io tracks protocol’s PDF downloads as well.
SS: How does protocols.io compare to existing open platforms?
LT: Other journals/platforms that have in the past or still host protocols are: Methods in Enzymology, Methods in molecular biology, bioprotocols.org (which is also peer-reviewed), Method X(by Elsevier), Openwetware, Protocol Exchange. Protocol Exchange (by Nature Springer, PE is 12 years old) has always had and now has 5-10 protocols submission per month (see below compared to protocols.io). Protocols Exchange holds 800 protocols, while protocols.io currently holds 2200. “there is only one other protocols repository, Nature’s Protocol Exchange, but the adoption of protocols.io is significantly stronger.”
SS: Do PLOS papers for instance that have cited a protocols.io protocol, get cited more often?
LT: As you are aware, citations can be subjective and not even 5 years have passed so there are not enough citations to assess reproducibility. protocols.io tracks most popular protocols, how many unique users each protocol has, the percentage of researchers that click on a protocol, article level metric on PLOS papers that cite protocols.io DOIs.
SS: So the question is how do researchers publish on protocols.io?
LT: Researchers start detailing a protocol on protocols.io well before they publish. When a journal publishes their research paper, protocols.io assigns a DOI to their protocol on protocols.io which is then implemented in their published research paper. It is not compulsory to submit your paper’s methods protocols to protocols.io. I have deliberately asked editors to make it optional so that the authors do it themselves as lab cultures change.
Further Reading:
[1] Teytelman L, Stoliartchouk A, Kindler L, Hurwitz BL. Protocols.io: Virtual Communities for Protocol Development and Discussion. PLoS Biol. (2016);14(8):e1002538. doi: 10.1371/journal.pbio.1002538. eCollection 2016 Aug.
[2] Kindler L, Stoliartchouk A, Teytelman L, Hurwitz BL. Method-centered digital communities on protocols.io for fast-paced scientific innovation. F1000Res. (2016);5:2271. Doi: 10.12688/f1000research.9453.2. eCollection 2016.
[3] Waller RF, Cleves PA, Rubio-Brotons M, Woods A, Bender SJ, Edgcomb V, Gann ER, Jones AC, Teytelman L, von Dassow P, Wilhelm SW, Collier JL. Strength in numbers: Collaborative science for new experimental model systems. PLoS Biol. (2018);16(7):e2006333. doi: 10.1371/journal.pbio.2006333. eCollection 2018 Jul.
[4] Teytelman L. No more excuses for non-reproducible methods. Nature (2018) 560, 411.
doi: 10.1038/d41586-018-06008-w
