There are multiple kinds of biological experiments, which may involve generating hypotheses, testing the feasibility of research procedures, and testing hypotheses.
Pilot studies test the feasibility and efficiency of research procedures on a small scale, with few animals.
Experimental studies are characterized by randomization, replication, and control.
Exploratory studies are observational or correlational studies which identify patterns that inform hypotheses.
A good experimental question is one that is worthwhile answering and that raises one or more testable hypotheses given constraints such as time, resources, etc. (Fry, p460)
Identify and categorize the experimental units and the other variables in the study (e.g., background, constant, primary, uncontrollable) [Should there be discussion of measurement?]
Good experimental design is strategic in its sampling methods by considering randomization, blocking or stratification methods, and sample size
Good design planning also identifies the statistical tests and analysis while considering other study aspects (e.g., hypotheses, variables, design structures, and samples).
Glossary
blocking
Samples of similar structure grouped together from both treatment and control.
biological replicate
Measurement(s) from biologically distinct samples (preferably taken at the same time) that convey the random biological variation that exists within a population. Biological replicates should not be confused with technical replicates.
block
:
blocking
used to reduce unexplained variability by grouping together samples of similar structure from both treatment and control. For example,
a new drug is tested on both male and female subjects. Sex of the patient is a blocking factor that accounts for treatment variability between males and females.
confounder
An unaccounted for variable that exerts either a small or large effect on a dependent (response) variable. Such variables increase variance and bias in the study.
control
An experimental subject that does not receive the treatment, and that is used as a baseline to evaluate the effect of the treatment on another group of subjects.
controlled experiment
an experiment done in parallel on a treatment and a control group that differ in one way (the independent or explanatory variable). Investigators determine which subjects go in the treatment group and which in the control group. For contrast, see observational experiment.
deviation
:
effects
:
experimental error
:
experimental unit
:
factorial design
A design that permits testing of multiple variables at once.
fixed effects
:
observational experiment
an experiment done in parallel on a treatment (or exposure) and a control group that differ in one way (the independent or explanatory variable). The subjects, not the investigators, determine whether they are in the treatment group or the control group (i.e. smokers and non-smokers). For contrast, see controlled experiment.
random effects
:
random error
:
randomization
A method to reduce bias and minimize the likelihood of chance altering the results of an experiment.
replicability
Other researchers obtain corroborating results using experimental methods similar to those of the original study,
generating their own data independently.
reproducibility
Other researchers duplicate results and can draw the same conclusions as the original study did using
the same materials and methods (i.e. specific measurement devices, original data, software, statistical method).
sensitivity
:
specificity
:
systematic error
:
technical replicate
Repeated measurements of the same sample that represent independent measures of the random noise
associated with protocols or equipment. For contrast, see biological replicate.
treatment
:
variability
the extent to which a distribution is spread out or squeezed in; also known as dispersion or spread
variance
a measure of statistical variability or dispersion
National Academies of Sciences, Engineering, and Medicine. 2019. Reproducibility and Replicability in Science. Washington, DC: The National Academies Press. https://doi.org/10.17226/25303.
Kilkenny C, Parsons N, Kadyszewski MF, Cuthill IC, Fry D, Hutton J, Altman DG. Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PloS one. 2009;4(11).
Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLoS biology. 2010 Jun;8(6).
Optimal experimental design
B. Smucker and M. Krzywinski and N. Altman
Nature Methods 15 559–560 (2018)
https://doi.org/10.1038/s41592-018-0083-2
Customize the experiment for the setting instead of adjusting the setting to fit a classical design.
Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, Robson R, Thabane M, Giangregorio L, Goldsmith CH. A tutorial on pilot studies: the what, why and how. BMC medical research methodology. 2010 Dec;10(1):1.