Findings from studies that cannot be verified when the experiments are repeated are cited 153 times more because the research is interesting, according to a new UC San Diego paper.
Photo credit: The Academic Times
Papers that cannot be replicated are cited 153 times more because their findings are interesting, according to a new UC San Diego study
Papers in leading psychology, economic and science journals that fail to replicate and therefore are less likely to be true are often the most cited papers in academic research, according to a new study by the University of California San Diego’s Rady School of Management.
Published in Science Advances, the paper explores the ongoing “replication crisis” in which researchers have discovered that many findings in the fields of social sciences and medicine don’t hold up when other researchers try to repeat the experiments.
The paper reveals that findings from studies that cannot be verified when the experiments are repeated have a bigger influence over time. The unreliable research tends to be cited as if the results were true long after the publication failed to replicate.
“We also know that experts can predict well which papers will be replicated,” write the authors Marta Serra-Garcia, assistant professor of economics and strategy at the Rady School and Uri Gneezy, professor of behavioral economics also at the Rady School. “Given this prediction, we ask ‘why are non-replicable papers accepted for publication in the first place?’”
Their possible answer is that review teams of academic journals face a trade-off. When the results are more “interesting,” they apply lower standards regarding their reproducibility.
The link between interesting findings and nonreplicable research also can explain why it is cited at a much higher rate—the authors found that papers that successfully replicate are cited 153 times less than those that failed.
“Interesting or appealing findings are also covered more by media or shared on platforms like Twitter, generating a lot of attention, but that does not make them true,” Gneezy said.
Serra-Garcia and Gneezy analyzed data from three influential replication projects which tried to systematically replicate the findings in top psychology, economic and general science journals (Nature and Science). In psychology, only 39 percent of the 100 experiments successfully replicated. In economics, 61 percent of the 18 studies replicated as did 62 percent of the 21 studies published in Nature/Science.
With the findings from these three replication projects, the authors used Google Scholar to test whether papers that failed to replicate are cited significantly more often than those that were successfully replicated, both before and after the replication projects were published. The largest gap was in papers published in Nature/Science: non-replicable papers were cited 300 times more than replicable ones.
When the authors took into account several characteristics of the studies replicated—such as the number of authors, the rate of male authors, the details of the experiment (location, language and online implementation) and the field in which the paper was published—the relationship between replicability and citations was unchanged.
“Remarkably, only 12 percent of post-replication citations of non-replicable findings acknowledge the replication failure,” the authors write.
The influence of an inaccurate paper published in a prestigious journal can have repercussions for decades. For example, the study Andrew Wakefield published in The Lancet in 1998 turned tens of thousands of parents around the world against the measles, mumps and rubella vaccine because of an implied link between vaccinations and autism. The incorrect findings were retracted by The Lancet 12 years later, but the claims that autism is linked to vaccines continue.
The authors added that journals may feel pressure to publish interesting findings, and so do academics. For example, in promotion decisions, most academic institutions use citations as an important metric in the decision of whether to promote a faculty member.
This may be the source of the “replication crisis,” first discovered the early 2010s.
“We hope our research encourages readers to be cautious if they read something that is interesting and appealing,” Serra-Garcia said. “Whenever researchers cite work that is more interesting or has been cited a lot, we hope they will check if replication data is available and what those findings suggest.”
Gneezy added, “We care about the field and producing quality research and we want to it to be true.”
More from: University of California San Diego
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
Science replication crisis
- The problem with academic funding, in 1 ironic scandalon August 23, 2021 at 12:25 pm
But the problem goes deeper than possible misconduct by one high-profile researcher. Social science is already in the grip of a so-called replication crisis. In other words, researchers don't get the ...
- Colby Cosh: Yet another famous psychology study turns out to be pure bunkon August 23, 2021 at 3:00 am
Psychological science has been suffering a “replication crisis” for a decade now , but this week there was a particularly newspaper-friendly example of a famous finding failing to hold up on ...
- Psychology is in a crisis. But not the one you’re thinking ofon August 12, 2021 at 1:42 am
Can we still have faith in psychology in the face of the ‘replication crisis’? Perhaps, but the field has a much larger problem to tackle.
- Hepatitis, one of the top killers: Experton July 27, 2021 at 10:25 pm
Despite its non-living status, once within a host cell, it is very active inducing cell replication and assembles to form many viruses. Covid is transmissible through air, breath and hand contact ...
- India, Russia, Brazil & S Africa to repurpose SARS CoV2 replicating enzyme to develop covid-19 medicineson July 26, 2021 at 7:44 am
Scientists from India, Russia, Brazil, and South Africa will work together to repurpose, validate and synthesise lead compounds against main protease and RNA replicas, the enzyme that catalyzes ...
Go deeper with Google Headlines on:
Science replication crisis
Go deeper with Bing News on:
Science’s reproducibility crisis
- John Boylanon August 23, 2021 at 4:54 am
EconoTimes is a fast growing non-partisan source of news and intelligence on global economy and financial markets, providing timely, relevant, and critical insights for market professionals and those ...
- Michele Trott, PhDon August 18, 2021 at 12:05 pm
Michele Trott (nee Wilson) is a New Zealand-based science writer who pursues freelance writing projects alongside her in-house role as a scientific content writer at Izon Science. After completing her ...
- Ultromics raises $33M to build out its AI-enabled cardiovascular imaging solutionon August 18, 2021 at 10:55 am
Both EchoGo Core and EchoGo Pro are already being used and validated by several organizations, including Mayo Clinic and the Oregon Health and Science University ... eliminates the variability and ...
- Experimental Design for Laboratory Biologistson August 17, 2021 at 2:54 am
Highly recommended.' Dorothy Bishop, University of Oxford 'Worldwide there is a salient discussion about deficiencies in the validity and predictiveness of research in the life sciences. Indeed, a ...
- Sigmoid CEO, Lokesh Anand, Recognized as One of India's Best Leaders in Times of Crisis 2021 by Great Place to WorkÂ® Indiaon August 16, 2021 at 1:36 pm
BENGALURU, India, Aug. 16, 2021 /PRNewswire/ -- Great Place to Work ® India has recognized Lokesh Anand, the CEO of Sigmoid as one of India's Best Leaders in Times of Crisis 2021 in the mid ...