Quantcast

How to Minimize Variation and Achieve Reproducibility

Ever wonder why your data isn’t the same after repeating an experiment? Well part of science’s beauty lies in the difficulty of achieving reproducibility.

Heraclitus first said that no mans steps in the same river twice and the same can be applied to experiments. It is literally impossible to control for everything because the second time you do your experiment, you will be doing it at a different point in time. The good news is that most laboratory science is performed on such a small scale that a few days, weeks, even years between repeating an experiment wouldn’t affect your results. Let’s take a moment though to think about things that could.

Consistency is Key

Every step of an experiment, from set-up to collecting, processing, and running assays, requires dozens of steps, all of which are potential sources of variation. In thinking about at an experiment holistically, consistency is key to minimizing variation. When you repeat an experiment, doing everything exactly the same as the first time is crucial. This doesn’t mean you need to wear the same clothes and have the same tunes playing when repeating an experiment, but it does mean you need to take good notes in your lab notebook regarding how many micrograms of protein lysate was in how much total volume of PBS for that last immunoprecipitation. It’s impossible to know what kind of difference every little detail makes, but always assuming that everything and anything could affect your results will help you be consistent, minimize variation, and achieve reproducibility.

Everything’s Always Changing: Why Consistency isn’t as Easy as it Sounds!

It’s really hard to control for variables when by nature they are constantly changing! Take oxidation for example – it’s always occurring, so unless you have vacuum-sealed containers, beware. If you are treating cells with a solution that you pre-make by mixing a powder from the stock chemical cabinet with water, using the same solution you made up last time might seem like you are being consistent but due to oxidation(1-3) you might be working with a very different solution. Therefore make it up fresh each time! Ever leave a tube of water in the fridge or on the counter top untouched for a while? You’ll notice droplets accumulate inside the walls of the tube. This is because the universe is constantly in motion and every molecule on earth is subject to that motion. We use –80°C and –20°C freezers to slow things down but “stasis” on the nano-scale is fleeting. As a scientist, you’re constantly battling the eternal motion of the cosmos (deep right?) so the more consistently you deal with and control these battles, the more accurate and reproducible your results will be.

Good science is extremely difficult because minimizing variation means acting borderline-OCD. Once you get used to it however, it will make a world of difference and you’ll have data ready to publish. Here is a short list to get you started on thinking about ways to minimize variation. Use it to complement your own knowledge/experience in the lab, and when repeating experiments always remember, consistency is key.

Variables to Control for Consistent Results:

Re-used vs. Fresh ingredients

Let’s say you’ve extracted RNA, used some of it to make cDNA, froze the rest, and used all of the cDNA for q-PCR. Now you want to repeat the q-PCR run, so you go back to your frozen RNA. This adds variation because the first time around you made your cDNA from fresh RNA. Freezing and thawing your RNA may seem trivial but on the molecular scale it will likely have changed your RNA. Repeating the experiment from start to finish, performing each step as you did the first time will add a little more work in the short term, but will save you significant time and worries long term.

Centrifugation

Speed and duration of spins can affect the final yield of your sample, which can subsequently have an effect on downstream applications due to variations in the concentration of reactants. Make sure that you are spinning at the same speeds – and remember RPM does not equal RCF so if you are switching between centrifuges make sure you calculate the correct speed!

Volumes/Concentrations

The amount of “stuff” in your experiment, whether it’s for a treatment or a specific reaction, has a direct influence on your results, and will probably be the most significant contributor to variation. Keep volumes and concentrations the same for everything.

Incubation Time

Three hours compared to two hours is a big difference. It’s mathematically 50% more time and in terms of molecules bathed in solution, it will allow many more interactions to take place. So before you run off for a long lunch think about whether it’s worth it if it messes up your results.

Washes

Frequency and duration of washes are both variables to consider. Not enough could decrease specificity and lead to artifacts in your data, while too many could wash away your molecule of interest.

Ratios

For example, in molecular cloning, your insert:Vector ratio during ligation is critical to optimize. Once you’ve found the one that works, keep it the same.

Identical Products from Different Companies

Different manufacturers sell the same products, but they often differ in minor ways that can impact your data. For example, one cDNA synthesis kit uses random primers while another uses a mix of oligo dT and random primers. They both make cDNA, but the difference in specificity may have a significant impact on your data. If you need to switch companies for any reason check that the products are identical or as close to it as you can get. This include chemicals too, as purities and grades may differ between companies, so consider doing a side by side test before switching completely over to ensure there are no differences.

Temperature

Room temperature is not an exact unit and can vary dramatically day-to-day depending on the season, whether the heating was turned up, or time of the day. Therefore instead of incubating reactions at room temperature, incubate them at a fixed temperature in a water bath or heat block.

Equipment

Although your PCR machines all do PCR, they probably haven’t been used equally, and were manufactured at different times. These are factors that can eventually alter the efficiencies of machines. Using the same machine when repeating an experiment will add an additional level of control that could help tighten up your data.

Handling

One of the most common sources of variation is processing too many samples at one time. Not only could this affect results among independent experiments, but it could also affect results among samples within an experiment. so instead of trying to get too much done at once, take a breath and handle fewer samples. It may take you more time in the short run, but it could save you a lot of time in the future be preventing you from having to repeat a failed experiment.

 

These are my tips for ensuring consistency. Do you have any? If so please leave them in comments below.

 

References:

  1. Calligaris, S., Manzocco, L., and Nicoli, M. C. (2007b). Modelling the temperature dependence of oxidation rate in water-in-oil emulsions stored at sub-zero temperatures. Food Chem. 101 : 1019-1024.
  1. Champion, D., Blond, G., and Simatos, D. (1997). Reaction rates at sub-zero temperatures in frozen sucrose solution: a diffusion controlled reaction. Cryo-Letters. 18: 251-260.
  1. Sudareva, NN. and Chubarova, EV., (2006). Time-dependent conversion of benzyl alcohol to benzaldehyde and bezoic acid in aqueous solutions. J Farm Biomed Anal. 41 (4) : 1380-1385

Leave a Comment





This site uses Akismet to reduce spam. Learn how your comment data is processed.