Science is stuck in bad patterns – Time to evolve

Giant pandas. Self-fertilizing plants. People.

Lots of species have been proposed as “evolutionary dead ends” (though whether they actually are is another question). And now added to those ranks might be science itself.

A new paper has argued that science appears to be in something of an evolutionary cul-de-sac, mired in poor methodology and misguided objectives that have changed only for the worse over the past 60 years. Indeed, science is doing pretty much what it has evolved to do — produce a steady stream of plausible and attention-grabbing but frequently false findings.

At least, that’s what a pair of researchers at the University of California, Davis, said in their new paper, titled “The Natural Selection of Bad Science.” The authors, Paul Smaldino and Richard McElreath, argued that “some of the most powerful incentives in contemporary science actively encourage, reward, and propagate poor research methods and abuse of statistical procedures.”

Smaldino and McElreath, who posted their article on the preprint server arXiv, meaning it has not yet been through peer review, noted that the behavioral sciences in particular fail to produce statistically rigorous studies. The term of art here is “power” — an indication of the ability of an analysis to detect the outcomes it purports to find. And, according to Smaldino and McElreath, “despite over 50 years of reviews of low statistical power and its consequences, there has been no detectable increase” in the average power of studies.

Why the lack of improvement? The UC Davis researchers, like many others, believe that science is simply behaving the way it has evolved to behave: as a publishing factory. “An incentive structure that rewards publication quantity will, in the absence of countervailing forces, select for methods that produce the greatest number of publishable results. This in turn will lead to the natural selection of poor methods and increasingly high false discovery rates.” Sticking with the Darwinian metaphor, Smaldino and McElreath said that successful labs churn out “progeny” — grad students, junior faculty, future lab heads — that mimic their approach.

And the evolutionary metaphor is telling in another way. Evolutionary theory talks about the presence of “cheaters” in any population, but Smaldino and McElreath pointed out that the influence of such cheaters is even more insidious than one would think. For example, cheaters may invent ways to game the system, but others are quickly forced to mimic their methods, lest they be left behind.

Learn more: Science is stuck in bad patterns – Time to evolve

See Also

To Teach Scientific Reproducibility, Start Young