Dear PNAS editorial board,
We would like to bring to your attention the attached manuscript entitled
Sustainable computational science: the ReScience initiative
and ask you to consider it for publication in PNAS.
The article explains our motivation for launching ReScience, an online journal dedicated to publishing replication work in computational science, and reports on its first two years. The author list contains everybody who has actively contributed to this venture, in the role of author, reviewer, or editor.
ReScience aims to solve the problem of scientific models disappearing from the scientific literature. More and more scientific models take the form of complex algorithms, which are impossible to describe in detail in a traditional journal article. The full model exists only in the form of computer software, which is traditionally not published. The peer-reviewed article contains only a summary of the method, making it very difficult for reviewers and readers to understand the method and the results obtained with its help.
Publishing the software is not a sufficient solution either. Much research software is hard to understand and hard to use. Moreover, it is very difficult to judge if it actually corresponds to the published method summary. Finally, given today's state of the art, software ceases to work after a few years. Encoding scientific models in software is therefore not sustainable.
The best technique we have today to validate computational methods is an independent reimplementation by a different team, based only on the published method summary. If this independently developed software replicates the results of the original one, many mistakes commonly made in computational science can be excluded. The trust in the correctness of the results can be increased further by subjecting the independent reimplementation to peer review. From the point of view of sustainability, the success of an independent reimplementation proves that the original method summary is correct and sufficient.
ReScience makes two important contributions that traditional scientific journals cannot offer. First, it provides a venue for publishing replication work, which traditional journals exclude for lack of novelty, and thereby provides an incentive for contributing to scientific quality assurance. Second, it has a peer reviewing process adapted to the specific needs of computational replication work, borrowing from the techniques of both scientific peer review and collaborative software development.
Considering the ever increasing importance of computational methods in all scientific disciplines, we believe that our approach to replication is of interest to a broad audience of researchers, and therefore to the readership of PNAS. Moreover, we feel that our initiative is fully in line with the National Academy's recent efforts to improve reproducibility and replicability in science.
Signed,
Dr. Konrad Hinsen for the authors.