An
interesting article over at the
Guardian that includes the following quote from
Darrel Ince of the O.U.:
...if you are publishing research articles that use computer programs, if you want to claim that you are engaging in science, the programs are in your possession and you will not release them then I would not regard you as a scientist; I would also regard any papers based on the software as null and void.
My immediate response was broad agreement but on reflection I am not so sure that it is so straightforward and that quote needs to be unpacked a little because the argument, as stated, is inflammatory, definitely divisive, and possibly also a little offensive because it not only abuses the original scientist through an
ad hominem (although one weasily couched in terms of opinion) but suggests that the peer reviewers are also failing in their job by approving papers that shouldn't be published.
My position is that you need not always publish your sourcecode, and in some cases it might actually be unhelpful to do so, for example:
- You need to release your data if your findings depend upon that data as an input. A corollary to this is that you also need to detail your methods for deriving your conclusion, but that just releasing sourcecode is not a sufficient substitute for that discussion of methods. If you don't release your data then nobody else can reproduced your results, hence you are NOT doing science. However, as a rule releasing the code will not affect whether or not your findings are valid or not, only reproducibility from first principles can do that, i.e. Another group reproducing your findings from the original dataset. Even better is another group reproducing findings from an independent dataset but we must accept that in domains like that of Anthropomorphic Climate Change the scale and nature of data collection makes it difficult to build such datasets.
- If the software merely implements a formal model so that the results come from the execution of the software then there is no need to release the code, only the formal model, and possibly, but not necessarily the methods used to implement, it need to be released. However this must be done in sufficient detail so that a reimplementation can be achieved and any results can be reproduced.
By not releasing the original code, new implementation by different groups can strengthen the scientific basis for a conclusion. For example, by taking a different approach or implementing in a different language to that of the original theoretical model can uncover any bias or errors in the original implementation. This is potentially strengthening the original finding, or conversely uncovering weaknesses in the implementation. Either way, not having the code in this case will lead to greater insight into the original model. By having the original code we are tempted to reuse, or possibly merely tainted by having read a solution that colours any new approach.
So, broadly, share your data, share your (formal) theoretical basis, but you don't necessarily need to share your code. If you have a formal model that underpins your findings then your peer-reviewers and editor should be picking up on whether that is sufficiently well presented to allow reuse. Similarly it is the venues that publish research who should be ensuring that the datasets which underpin it are available to the wider community. They have failed in their role if there is not enough detail, as also has the scientist who has not provided the detail. It is for the scientific community to then take up the burden of reproducing results. The peer review process should not be expected to ensure correctness of findings but merely to ensure that well written, well argued, and important research is made widely available. Just because an article has gone through this process does not mean that the matter is in any way settled.
To return to the original quote, the argument that "if you are publishing research articles that use computer programs, ..., the programs are in your possession and you will not release them then I would not regard you as a scientist" does not really hold water and makes an emotive argument that polarises the debate. I am unaware of any findings based purely off of a piece of software that is not based in either an input dataset or a formal theoretical model (of which the software is merely a realisation) and it is these that must be publicised not the code that is based upon them.
