There are six little words that can instill both excitement and trepidation in the heart of a graduate student: “No one’s ever done this before.”
What those words really mean, of course, is “No one’s ever published this before,” and you are either standing at the edge of a great discovery or a chasm of frustrating years in the lab generating a lot of negative data.
After about your third night of staying at the lab until 2 am (because that’s the only time available on the fluorimeter/microscope/insert your favorite instrument here), you can’t help but wonder if some other poor sap graduate student halfway around the world has already attempted to accomplish that same goal that “no one’s ever done before”.
But unless all your troubleshooting leads you to an experiment with some interesting, positive data, the 99 ways you discovered that two proteins DON’T interact will never see the light of day. And there is nothing to stop that poor graduate student from repeating your “failed” experiments.
We are all too well aware of the fact that a year’s worth of work is often summarized in one power point slide, and that all the failed experiments that led you to the “right” result are unlikely to make it out of your lab notebook.
Places to share the bad results
What if there was a place where you could publish all the techniques that didn’t work or all the experiments that weren’t statistically significant?
Well, thanks to a series of new journals, you can. The All Results Journal, the Journal of Negative Results, and the Journal of Errology are just a sampling of journals based on the concept of publishing negative results. All of these journals are peer reviewed, so you can’t just publish a binder full of fuzzy Westerns (no matter how reproducible they may be).
So what kind of research do these journals publish? According to the All Results Journal, experiments must be rigorously conducted and reproducible, and the discussion must contain a reasonable explanation for why the results were observed.
The Journal of Negative Results encourages articles that explain why certain methods are not suitable for studying a particular phenomenon. Finally, the Journal of Errology, which has yet to be launched, appears to be attempting to break away from the structure of traditional journals by creating a repository “where [researchers] can share experiences mostly gained via negative results, futile hypothesis, mistakes, false starts, errors and other stumbles”. Some of the stated goals of these journals are to show how methods or experiments were fine-tuned or optimized, to reveal flaws in commonly used methods or reagents, and to save resources by preventing different labs from testing hypotheses that have been disproven or using methods that have been found lacking.
Positive publication bias
So, are these journals true noble endeavors for catalyzing scientific development or simply the brainchildren of dejected former graduate students and post-docs, determined to have their research published somewhere, anywhere?
The notion, and importance, of publishing negative results, is not entirely new. Concern about a “positive publication bias” has prompted studies into whether the scientific literature accurately reflects scientific truth. It appears that it may not: a 2008 study that evaluated studies used by the FDA to make approval decisions on antidepressants found that, all together, 51% of the studies were positive; however, of the studies that were published in journals, 94% were positive.
Do scientists owe it to the community to publish negative results to avoid a positive publication bias? There are a lot of factors at play, of course. In its truest form, science is about exploring the unknown, and if we are truly investigating the unknown, then a negative result seems just about as likely as a positive result. So why not put both the good and the bad out there?
The importance of being earnest funded
Well, a hard reality of a career in science is the need to find money to support your experiments and convince people that your research is important and interesting. It’s a lot easier to get people on board when you can declare, “Hey, I’ve found a gene that causes cancer”, while, “Hey, I found a gene that doesn’t cause cancer” doesn’t have quite the same effect. And, as a grad student or post-doc, you definitely don’t want to get stuck on the non-cancer-causing gene project.
Let’s face it, somewhere in the middle of your third year of grad school, you abandon the delusional idealistic quest for advancing knowledge and uncovering the truth and realize that you just need to get some experiments to work and convince a journal that they’re worth publishing. Will journals such as these promote more risky science since results can be published even if a hypothesis proves to be false?
We have yet to see whether journals that focus on negative results will have an impact on science or whether they’ll even be taken seriously. What do you think? Would you be willing to publish in one of these journals, and what would you do if you came across a manuscript of negative results that directly related to your project?
Would you continue on with your project anyway, trying to salvage a good hypothesis, or do you accept the fact that someone’s already tried it and move on?
Tell us your thoughts in the comments section!