Introduction

Overview

Teaching: 5 min
Exercises: 10 min

Questions

• What is reproducibility?

• What is replicability?

• Why are replicability and reproducibility important?

Objectives

• To differentiate between replicability and reproducibility.

• To describe the role of replicability and reproducibility in science.

Science aims to construct knowledge that is reliable and robust, and that provides a stable foundation for building new knowledge. Science self-corrects when the scientific community actively engages in examining and appraising ideas. In order to critically evaluate the evidence behind ideas, the scientific community might reproduce experimental results using the same materials and methods as in the original investigation. If the same data and the same analyses produce the same results, the original results are considered reproducible.

Reproducibility is a minimum necessary condition for a finding to be believable and informative. — National Science Foundation Subcommittee on Replicability in Science

Replicability can determine whether the relationship between variables in a study still holds under different conditions. In replicating a study, the scientific community duplicates the results of the original investigation using the same methods but generates new data. Replication requires detailed information about methods and materials from the original investigation.

Many publishers and funding organizations require access to data or software in order to promote transparency and reproducibility. When designing a study, looking forward toward how the scientific community will use the resulting data is wise. This is especially important for high profile studies or those with surprising results, since these are most likely to be deeply scrutinized by the scientific community. The National Institutes of Health proposed a set of actions to address issues with reproducibility, including better training. They also address replicability by expecting detailed experimental protocols and grant reviewing that requires appropriate experimental design.

Discussion

Before we go any further here are some important questions to consider. Please think about these questions, then pair up with your neighbor to discuss your responses. Finally, share your responses with all in the etherpad.

1). What makes a study reproducible?
2). What are the general characteristics of a well-designed study?

Key Points

• Reproducibility expects the same results using the same data and methods from an original study.

• Replicability corroborates the results of an experiment using the same methods as the original.

• Funding agencies and publishers increasingly require evidence of reproducibility in grant applications and manuscript submissions.