One of the thoughts that keep scientists up at night is the fear that another scientist will try to repeat your experiment – and find different results.
The point of publishing scientific discoveries is to allow other researchers to repeat exactly what you did in your lab, and (hopefully) find the same result. However, recently it has become uncomfortably apparent that many peer-reviewed scientific findings simply can’t be reproduced.
Whole fields of psychology are poised to tumble.
This may seem mundane compared to the climate crisis that us scientists have been tasked with solving, but the 'replication crisis' challenges our ability to trust scientific results and can leave whole fields of psychology poised to tumble like a house of cards.
Some academics have built entire careers on these ‘discoveries’, so the growing evidence that they were false positives all along has many scientists questioning everything, like Michael Inzlicht at the University of Toronto who wrote the devastating blog post, “Have I been chasing puffs of smoke for all these years?”.
Much of the media coverage of the replication crisis has focused on high-profile individuals accused of scientific fraud, like beleaguered poster boy Brian Wansink. The reality is that there is a lot of grey area between fastidious monk-like experimentation and outright scientific misconduct, and the current system of science funding encourages researchers to work in that grey area.
Scientists are trained in a system with deeply conflicting messages: replicate your own experiments to avoid false positives, but publish early and often. Science is a slow, incremental process, but to secure funding you must constantly produce definitive results. Collaborate with other scientists, but don’t give away so much that you get scooped.
Ironically, being scooped independently validates your results, but negatively affects your ability to use them to advance your career – high profile journals simply don’t publish straight replications of other experiments. It goes without saying that the lack of enthusiasm for replications is a major part of the replication crisis.
We need to publish failures so we don't repeat them.
Knowing if another scientist has tried and failed at the thing you want to do can save you a lot of time and money – because you then know not to do the same thing they did. But, you’ll never know – because journals don’t publish ‘null’ results. And the media don’t want to report on an experiment that didn’t prove the hypothesis.
It’s particularly disheartening as a scientist to discover that you’ve been wasting time and money attempting an experiment that has already failed in someone else’s lab. This happens often enough that a common joke is that we need a Journal of Failed Experiments. As a PhD student and postdoc, every experiment that ended without a definitive result (as most experiments do) felt like a failure, and like a waste of months of hard work.
In most other professions, months or years of hard work and good ideas would not go unrewarded, but science isn’t treated like a normal job. It’s romanticised as a passion, not a profession, and many scientists working within the public grant system internalise the notion of science’s nobility to cope with career instability and the demands of publication and grant writing. In my postdoc at UCLA I even heard researchers argue that it would be unethical to increase their paltry salaries because they believed it would divert money away from science.
The 'publish or perish' pressure leads to premature results.
Scientists want to change the system that has led to the replication crisis, but those in power are academics that have directly benefited from that system, and may not perceive it as broken. A researcher’s publication record is often argued to be an objective standard of scientific success, but obtaining publishable results fundamentally relies on luck. Many researchers work just as rigorously as their successful colleagues, but don’t happen to be working in a fruitful part of their field.
Solving the replication crisis requires fundamentally changing the role of publications in science. Scientists need long-term employment based on knowledge and skill, not short-term grants awarded to those with the most publications. Scientific discovery thrives in an open and collaborative environment but is stifled by the competition bred by the pressure to publish. The more scientists are pressured to ‘publish or perish’, the more premature results will be circulated.
When considering solutions to the replication crisis, it’s easy to focus on the personal responsibility of scientists. Instead, we should be asking, what compels researchers to publish results they are concerned other scientists will not be able to replicate?
Dr Ashleigh Morse is a specialist in the science of decision making.
