International scientists are challenging their colleagues to make artificial intelligence (AI) research more transparent and reproducible to accelerate the impact of their findings for cancer patients.
In an article published in Nature on Oct. 14, 2020, scientists at Princess Margaret Cancer Centre, University of Toronto, Stanford University, Johns Hopkins University, Harvard University School of Public Health, Massachusetts Institute of Technology, and others, challenge scientific journals to hold computational researchers to higher standards of transparency, and call for their colleagues to share their code, models and computational environments in publications.
“Scientific progress depends on the ability of researchers to scrutinize the results of a study and reproduce the main finding to learn from,” says Dr. Benjamin Haibe-Kains, Senior Scientist at the Princess Margaret and first author of the article. “But in computational research, it’s not yet a widespread criterion for the details of an AI study to be fully accessible. This is detrimental to our progress.”
The authors voiced their concern about the lack of transparency and reproducibility in AI research after a Google Health study by McKinney et al., published in a prominent scientific journal in January 2020, claimed an artificial intelligence (AI) system could outperform human radiologists in both robustness and speed for breast cancer screening. The study made waves in the scientific community and created a buzz with the public, with headlines appearing in BBC News, CBC and CNBC.
A closer examination raised some concerns: the study lacked a sufficient description of the methods used, including their code and models. The lack of transparency prohibited researchers from learning exactly how the model works and how they could apply it to their own institutions.
“On paper and in theory, the McKinney et al. study is beautiful,” says Dr. Haibe-Kains, “But if we can’t learn from it then it has little to no scientific value.”
According to Dr. Haibe-Kains, who is jointly appointed as Associate Professor in Medical Biophysics at the University of Toronto and affiliate at the Vector Institute for Artificial Intelligence, this is just one example of a problematic pattern in computational research.
“Researchers are more incentivized to publish their finding rather than spend time and resources ensuring their study can be replicated,” explains Dr. Haibe-Kains. “Journals are vulnerable to the ‘hype’ of AI and may lower the standards for accepting papers that don’t include all the materials required to make the study reproducible – often in contradiction to their own guidelines.”
This can actually slow down the translation of AI models into clinical settings. Researchers are not able to learn how the model works and replicate it in a thoughtful way. In some cases, it could lead to unwarranted clinical trials, because a model that works on one group of patients or in one institution, may not be appropriate for another.
In the article titled Transparency and reproducibility in artificial intelligence, the authors offer numerous frameworks and platforms that allow safe and effective sharing to uphold the three pillars of open science to make AI research more transparent and reproducible: sharing data, sharing computer code and sharing predictive models.
“We have high hopes for the utility of AI for our cancer patients,” says Dr. Haibe-Kains. “Sharing and building upon our discoveries – that’s real scientific impact.”
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
Artificial intelligence research
- IIT Madras Offering Two Year Research Fellowship In Artificial Intelligence, Apply Nowon June 12, 2021 at 6:04 am
The candidates who are interested in the IIT Madras Research Fellowship in AI and Data Science must submit their applications before June 30, 2021.
- Artificial Intelligence for Edge Devices Market Is Booming Worldwide | Arm, Cambricon, Horizon Roboticson June 11, 2021 at 10:24 am
Global Artificial Intelligence for Edge Devices Market (Thousands Units) and Revenue (Million USD) Market Split by Product Type such as [Type]. Further the research study is segmented by Application ...
- Artificial Intelligence (AI) in BFSI Market will Witness Huge Growth till 2027 & Covid-19 Analysison June 11, 2021 at 8:09 am
This Artificial Intelligence (AI) in BFSI market report provides a comprehensive overview of the major aspects that will drive market growth, such as market drivers, constraints, prospects, ...
- How artificial intelligence is changing the future of air transportationon June 11, 2021 at 7:29 am
A George Washington University School of Engineering and Applied Science professor is working on an interdisciplinary research project funded by NASA that aims to design and develop a safety ...
- White House launches artificial intelligence task forceon June 11, 2021 at 6:07 am
The White House Office of Science and Technology Policy (OSTP) and the National Science Foundation (NSF) on Thursday announced the launch of the National Artificial Intelligence Research Resource Task ...
Go deeper with Google Headlines on:
Artificial intelligence research
Go deeper with Bing News on:
Artificial intelligence research transparency
- The impacts of artificial intelligence on the workplaceon June 10, 2021 at 5:18 pm
As part of this programme of work, the OECD is conducting firm-level case studies to understand what happens when a specific AI-based technology is implemented in a workplace from the perspectives of ...
- Forward Thinking on artificial intelligence with Microsoft CTO Kevin Scotton June 10, 2021 at 5:00 am
How could AI help create jobs even in rural areas, and what would it take? In this podcast episode, Kevin Scott shares his ideas with James Manyika.
- Mastering artificial intelligence and machine learningon June 6, 2021 at 2:02 am
Control Engineering - Just a few decades ago, artificial intelligence (AI) was the stuff of science fiction, but has since become part of our daily lives. In manufacturing, it ...
- Artificial Intelligence in Education Market Insights by 2030 – Cumulative Impact of COVID-19on May 26, 2021 at 3:12 am
Education is one of the fastest growing industry verticals adopting artificial intelligence ... the-way-tmr-301180473.html About Transparency Market Research Transparency Market Research is ...
- Artificial Intelligence In Food and Beverages Market By New Business Developments, Innovations, And Top Companies – Forecast To 2027on May 26, 2021 at 1:20 am
May 26, 2021 (AmericaNewsHour) -- The Global Artificial Intelligence (AI ... AI can also help improve transparency in the working of enterprises by proper management of supply chain.