How to Write a Flawless Methodology Section

Excellent research takes time and effort, and a publication is your chance to showcase your hard work. While your main motivation might be to share and discuss your results, your methodology section is key to the reproducibility of your work, acting as a foundation for other researchers to repeat and build upon your findings.

In this article, we will guide you towards writing an exceptional methodology section, including how to design and document your experiments, and how to test your results, all while building a trustworthy reputation as a scientist.

Experimental Design: Factors That Affect Reproducibility

      1. Controls

It may be tempting to skip controls when running routine experiments. However, by including controls in each experiment, you have a better chance of troubleshooting experiments that don’t go as planned, and you can circumvent the risk of thinking you’ve made a significant observation when you haven’t. An excellent example is a flow cytometry experiment. You may be tempted to use gates established in previous experiments instead of running fresh controls. However, even daily fluctuations in atmospheric pressure can affect how a flow cytometry instrument runs your samples.

Bear in mind that depending on your experiment, you may need both positive and negative controls, and it is wise to include control samples in duplicate as a minimum. In this way, you’ll always have a backup if something happens to one of your control samples. A good rule of thumb is that your control sample should differ from your experimental samples by only one component/characteristic/trait.

2. Blinding

Blinding helps reduce the influence of external factors on the outcome of an experiment, and is most widely used during the testing of new drugs in clinical trials. During blinded clinical trials, half of the trial participants are given the test treatment (drug), and the other half are a given a placebo (mock) treatment, and the study participants don’t know which treatment they have been given. The placebo acts as a control and eliminates the possibility that the participant’s expectations may affect the outcome of the test treatment. In double blinding, neither the subject nor the investigator knows which treatment the trial participants have received.

Unfortunately, blinding is not always possible. This might be the case when the test treatment is a completely novel type of intervention, while the control group receives the standard of care intervention or treatment. Clinical trials in which participants and investigators know who is getting which treatment are referred to as open label studies.

3. Gender, Age, and Race Balancing

For treatments destined for eventual use in men and women, animal testing, and clinical trials should include a similar number of participants from each gender. Ideally, research subjects or clinical trial participants should represent as diverse a population as possible, within the scope of the experiment/trial. The same applies to balancing with race and age. When the tested population is limited to a certain age range and/or race, the results cannot be said to be universally applicable. If you cannot balance for gender, age, or race, state the limitations in your paper’s discussion section.

4. Adequate Sample Size

The size of a sample must reflect the degree of statistical power required for your experiment. For example, a pilot clinical study usually consists of 10 to 30 participants. The appropriate sample size depends on at least two important factors: the margin of error (i.e. the confidence intervals) and the confidence level.

The margin of error: If you set your margin of error to 5% and your test treatment positively affects 90% of your participants, you should expect 85-95% of your future population, i.e. participants in a subsequent trial or patients, to respond positively to the test treatment.

The confidence level: Following the above example, with a 5% margin of error and a 90% rate of positive effects on your study population, the confidence level tells you how often the percentage of your population who responded positively actually lies within the boundaries of your margin of error. In simple terms, if you choose a confidence interval of 95%, you can be sure that 95% of the time, 85-95% of your population will respond positively to the treatment.

Taken together, the margin of error and the confidence level tell you how confident you can be that your sample size is broadly applicable to the population you’re testing. For extra guidance, there are a few freely available softwares to help you determine your ideal sample size, such as this online sample size and margin of error calculator.

5. Statistical Tests

Broadly speaking, there are two types of statistical tests: planned and exploratory. Planned tests are those that are chosen in advance of experimentation. Exploratory tests are those you run after the experiments have been carried out. You should always aim to identify the types of statistical tests you will perform ahead of time. Make sure you understand the statistical program you’re using, and that you are using the appropriate statistical test(s) for your dataset. If you’re not sure, seek advice.

Exploratory analysis carries less weight as it can be a “fishing expedition” to find significant results. However, when there is adequate justification, you can include the results of an exploratory analysis in a paper. Importantly, always state clearly which statistical tests were planned and which were exploratory, and any conclusion(s) made from the exploratory analysis should be taken with a grain of salt.

6. Reagents

Choose commercially available reagents and ingredients where possible. Commercially available reagents are validated after production and should be trustworthy. If you’re preparing a reagent yourself, e.g. a buffer, based on previously published work, cite the original authors and make sure to contact them if you have any questions concerning their method. If the buffer recipe is unpublished, clearly describe every single step of the process involved in making a batch, including the order in which you dissolved the individual components.

7. Clarity and Honesty

It should be very clear from your methodology section how you got from A to B to C and so on. Your methodology is essentially a road map that allows others to do exactly what you have done. So, when designing your experiments, make sure to clearly document every single step for reproducibility. Accurately record the quantities used for each component, and note the manufacturer and lot number of each reagent or specimen you use. Make sure that you describe each step in adequate detail in simple but precise language.

Do a Run-Through

Once you’ve documented a procedure, ask at least one colleague to perform the steps you’ve written down. Shadow them and note down any comments they make throughout. In doing so, you should get a good feeling for which steps are easily understood and which ones need clarification. Resist the temptation to help them understand a step until it is clear they are stuck. Such an exercise allows both parties to spot mistakes and omissions, allowing you to check your protocol’s inter-observer reliability.

Citing and Explaining

Make sure you cite other researchers when you mention their methods, results, ideas and conclusions. If you used a previously published method but modified it, you must still cite the original author(s) and clearly state the modifications you made, no matter how small or insignificant they seem. Explain and justify any deviations from the original protocol.

Transparency

Be willing to answer questions on your methodology. Unfortunately, specific details are sometimes held back by researchers, especially when it comes to newly developed protocols. This not only slows down scientific research, but also calls into question the credibility of a researcher’s work. Open and honest collaborations allow cross-functional and interdisciplinary teams to work together to make discoveries and advance science. By sharing your experiences, i.e. your methodology, you are setting a good example for others.

Further Inspiration for Your Methodology Section

The guidance provided in this article should help you create an excellent methodology section. For more inspiration, pay close attention to the methodology sections published by experts in your field. Another useful resource is your target journal, which should contain a set of submission guidelines for authors. Finally, it is worth taking a look at journals that only publish protocols and method-based articles for even more ideas to help you write a killer methodology section. For example, check out Nature Protocol’s excellent guide for authors here.

Further Reading:

1. https://www.elsevier.com/connect/scientific-credibility-and-reproducibility
2. https://www.nih.gov/research-training/rigor-reproducibility/principles-guidelines-reporting-preclinical-research