Replicability is not Reproducibility: Nor is it Good Science

Drummond, Dr. Chris (2009) Replicability is not Reproducibility: Nor is it Good Science. [Conference Paper]

Full text available as:



At various machine learning conferences, at various times, there have been discussions arising from the inability to replicate the experimental results published in a paper. There seems to be a wide spread view that we need to do something to address this problem, as it is essential to the advancement of our field. The most compelling argument would seem to be that reproducibility of experimental results is the hallmark of science. Therefore, given that most of us regard machine learning as a scientific discipline, being able to replicate experiments is paramount. I want to challenge this view by separating the notion of reproducibility, a generally desirable property, from replicability, its poor cousin. I claim there are important differences between the two. Reproducibility requires changes; replicability avoids them. Although reproducibility is desirable, I contend that the impoverished version, replicability, is one not worth having.

Item Type:Conference Paper
Subjects:Computer Science > Artificial Intelligence
ID Code:7691
Deposited By: Drummond, Dr Christopher G.
Deposited On:27 Oct 2011 01:28
Last Modified:27 Oct 2011 01:28


Repository Staff Only: item control page