Computational methods are widely used in different scientific fields, and they have revolutionized the way research is conducted. For instance, computational methods can predict the binding affinity of potential drug molecules to target proteins, helping in the designing of new drugs; geoscientists can simulate complex systems, such as the behavior of the atmosphere and oceans, to investigate the impacts of climate change; and linguistic data, such as speech sounds and grammatical structures, can be analyzed with computational tools to better comprehend the structure and evolution of languages. In recent years, computational tools have become particularly important in the Biological Sciences as experimental techniques have led to an exponential growth in data, particularly in the comprehensive, large-scale analysis of various biological molecules, such as genes, proteins, and metabolites. This leads to many researchers using and even creating their own software to analyze their data. However, many of them do not have formal training in software development, and many third-party software applications lack adequate instructions, descriptions, or explanations, a fact that can affect scientific development, as computational reproducibility is harder to reach.
Ensuring reproducibility of results allows for the confirmation that the results of a study or experiment are valid and reliable, as it enables other researchers to verify and build upon previous findings. Furthermore, it promotes transparency in research by making it possible for others to access and evaluate the data and code used in a study, ensuring the integrity and credibility of the study's findings. To achieve reproducibility, adequate documentation of code and methods is crucial. This documentation can help researchers avoid redundant work, identify and correct errors in the code, and increase research efficiency. Additionally, some journals mandate authors to provide data and code used in the analysis, ensuring transparency and enabling others to reproduce the results.
If you talk to any researcher in the Biological Sciences, you will have no difficulty finding examples of situations where computational reproducibility was a problem.
Researchers in the Biological Sciences often face challenges related to computational reproducibility. These can include compatibility issues between computational tools and computer systems, papers that lack proper documentation of scripts and make data unavailable, and a lack of proper training or guidelines in good reproducibility practices. Indeed, several of us have faced challenges when trying to reproduce both our own results and those from other labs. For example, we have found studies with missing data, poorly documented software, disorganized code, and hard to install pipelines. These experiences have led us to adopt tools that enhance the reproducibility of our methods, such as Galaxy platforms, Singularity containers, and Jupyter notebooks. However, we have also realized that this is a widespread issue, so we decided to reach out to the community to raise awareness on how to improve reproducibility.
Until 2022, we all had had different types of interactions in initiatives involving programming skills for bioscientists with Renato Santos, currently an eLife Community Ambassador. During the earliest moments of his career as Bioinformatician, he had been trained to document every step of his analyses using online tools called ‘wikis’ - DokuWiki and MediaWiki were some of the most famous platforms at the time. Over time, he realized that it is frequently impossible to reproduce results other researchers have generated. Talking to colleagues he realized that lack of computational reproducibility was something not restricted to his experience and, therefore, we all joined efforts in the Ambassadors programme to build an activism project on Computational Reproducibility for Life Sciences building an initial team in Brazil including colleagues co-authoring this text and others interested in this topic.
The project, entitled “Reprodutibilidade Computacional Bio” (translated as “Computational Reproducibility Bio”), raises awareness about the importance of computational reproducibility in life sciences. The fields related to Bioinformatics, programming, and Data Science in Life Sciences are essential and play a critical role in science, however, the fact that a significant majority of the manuals and resources are available exclusively in English creates a barrier to entry for individuals who speak different languages. This is particularly problematic in regions such as Latin America, which is in the process of developing its scientific landscape and encompasses a multitude of languages. It is therefore crucial to increase the availability of materials in different languages, in order to provide access to crucial information and to promote diversity and inclusion within the scientific community. By doing so, we can ensure that everyone has access to the latest tools and information required to advance the field, and ultimately contribute to progress in science and medicine. Based on our initial plan, we have devoted most of our efforts to spreading and promoting our materials in Portuguese and shared on our Instagram profile, with the aim of being as inclusive as possible.
We wanted to produce funny cartoons reflecting expectations of early career scientists and researchers in different levels, including undergraduate and graduate students, on topics related to computational reproducibility, but facing several disappointments when trying to reproduce results of others or even their own. So far, we were able to produce approximately one post every other week, reaching around 500 followers, with an average of 30 likes/ posts. Recently, we started a collaboration with Dr. Rachel Paterman, an anthropologist and artist who coordinates “Desorientanda”, an Instagram profile dedicated to producing and sharing comics about the daily life of scientists. Our first collaborative post illustrates how difficult software installation can be, which could be due to poorly detailed installation instructions.
In our original eLife’s Ambassadors Initiative Proposal, we had outlined the expectation to expand our impact to other countries, particularly to other developing countries that may have similar challenges compared to Brazil. Currently, we have expanded our collaborators and volunteer network to other researchers in the Latin American region. Concretely, we have researchers distributed among two countries in Latin America, Brazil and Colombia. However, we are willing to expand our efforts to include other countries or parts of the world.
Initially, we had also planned to produce a website hosting an ebook, which includes a chapter on open science practices and an introduction about the importance of computational reproducibility, and other two chapters presenting to the scientific community how to use several tools that would certainly help them do more reproducible computational research, from the perspectives of both experienced bioscientists and computer scientists doing research in life sciences. In the end, the plan is to include in the ebook all arts and narratives produced for Instagram, even though we are still looking for voluntary collaboration of cartoonists and artists to improve the quality and originality of our work. Currently, at least two members of our team in Brazil are involved in the ebook production (Renato and Alicia), and recently two Colombian colleagues (Luisa and Maryam) also joined efforts on this project.
In particular, our colleagues in Colombia are eager to share their experiences in the computational reproducibility field that applied to bioscientists analyzing biological data and to raise awareness on the need to adopt more reproducible practices in computational research. However, a second and more lengthy effort is to help these researchers move toward adopting those practices and making them embedded in their research culture. Toward that end, they have developed a tutorial that covers a series of manageable and realistic steps for researchers to adopt reproducible practices in computational research, namely, basics of project management, data storage, file structures, naming conventions, usage of shared resources, conventions to document and share code through scripts and implementing version control.
A second part of the future activities of our project will be centered around spreading and promoting our material, encouraging the use, and supporting collaborators to use, translate, or improve the material.
By the way, we are recruiting new members!! Contact us by email or directly on our Instagram!
About the authors:
Renato Augusto Corrêa dos Santos is a postdoctoral researcher at the Computational, Evolutionary and Systems Biology Laboratory (LabBCES), Center of Nuclear Energy in Agriculture at the University of São Paulo (CENA/USP). He has been working with Bioinformatics, especially “omics” of microorganisms and plants, applied to medicine and bioenergy. He is interested in changes of research culture that impact computational reproducibility and mental health. Connect with Renato on Twitter or LinkedIn.
Alicia Lie de Melo is a PhD student in the Bioinformatics program of the Universidade de São Paulo. She has experience in metagenomics analysis of soil, but is currently focused on studying the promoter regions of sugarcane genes, with a special interest in characterizing condition-specific promoters. She believes ensuring reproducibility in all areas of science is crucial to maintaining the integrity and credibility of research findings.
Maryam Chaib has been an assistant professor at Universidad Nacional de Colombia since February 2023. She is interested in understanding the dynamics and molecular mechanisms driving transitions to pathogenicity in host-microbe associations. She would like to contribute to a culture of open science across academic, private and public sectors. Connect with Maryam on Twitter.
Luis Augusto Eijy Nagai, an assistant professor at the Laboratory of Computational Genomics at the University of Tokyo, specializes in gene expression regulation via multi-omics approaches. He harnesses Graph Theory for deeper insights into single-cell transcriptomic data, revealing intricate cellular states and transitions. Connect with Eijy on Twitter.
Luisa Matiz is a PhD student in Life Science at Hokkaido University, Sapporo - Japan. She has experience in metagenomics analysis, plant genomics and developmental biology. She is interested in understanding the mechanisms behind unusual sex determination and how it could be used to predict chromosome structure in animals. She believes simple communication that affects the listener is the key to foster interest in science for anyone.
Tulio Campos is a Public Health Technologist at the Oswaldo Cruz Foundation - Aggeu Magalhães Institute (IAM) / Brazil, where he participated in the implementation and consolidation of a Bioinformatics Core Facility, and of an institutional E-learning platform. He has extensive experience in Information Technology and data analysis, focusing on systems engineering, machine-learning and genomics of pathogens, parasites, and vectors of disease. He is involved in several projects involving “omics” technologies, and is also a member of the Fiocruz Genomic Surveillance Network that is responsible for sequencing and analysis of viral pathogens in Brazil. Connect with Tulio on LinkedIn.
Raissa Melo de Sousa is a PhD student in Genetics and Molecular Biology at the Federal University of Pará – Brazil. Currently, she is interested in omics applications in comparative oncology, focusing on the proteomics profile of triple-negative canine mammary tumors to find potential targets and biomarkers. She believes in the potential of open science for collaborative and accessible scientific discoveries.