International scientists are challenging their colleagues to make artificial intelligence (AI) research more transparent and reproducible to accelerate the impact of their findings for cancer patients.
In an article published in Nature on Oct. 14, 2020, scientists at Princess Margaret Cancer Centre, University of Toronto, Stanford University, Johns Hopkins University, Harvard University School of Public Health, Massachusetts Institute of Technology, and others, challenge scientific journals to hold computational researchers to higher standards of transparency, and call for their colleagues to share their code, models and computational environments in publications.
“Scientific progress depends on the ability of researchers to scrutinize the results of a study and reproduce the main finding to learn from,” says Dr. Benjamin Haibe-Kains, Senior Scientist at the Princess Margaret and first author of the article. “But in computational research, it’s not yet a widespread criterion for the details of an AI study to be fully accessible. This is detrimental to our progress.”
The authors voiced their concern about the lack of transparency and reproducibility in AI research after a Google Health study by McKinney et al., published in a prominent scientific journal in January 2020, claimed an artificial intelligence (AI) system could outperform human radiologists in both robustness and speed for breast cancer screening. The study made waves in the scientific community and created a buzz with the public, with headlines appearing in BBC News, CBC and CNBC.
A closer examination raised some concerns: the study lacked a sufficient description of the methods used, including their code and models. The lack of transparency prohibited researchers from learning exactly how the model works and how they could apply it to their own institutions.
“On paper and in theory, the McKinney et al. study is beautiful,” says Dr. Haibe-Kains, “But if we can’t learn from it then it has little to no scientific value.”
According to Dr. Haibe-Kains, who is jointly appointed as Associate Professor in Medical Biophysics at the University of Toronto and affiliate at the Vector Institute for Artificial Intelligence, this is just one example of a problematic pattern in computational research.
“Researchers are more incentivized to publish their finding rather than spend time and resources ensuring their study can be replicated,” explains Dr. Haibe-Kains. “Journals are vulnerable to the ‘hype’ of AI and may lower the standards for accepting papers that don’t include all the materials required to make the study reproducible – often in contradiction to their own guidelines.”
This can actually slow down the translation of AI models into clinical settings. Researchers are not able to learn how the model works and replicate it in a thoughtful way. In some cases, it could lead to unwarranted clinical trials, because a model that works on one group of patients or in one institution, may not be appropriate for another.
In the article titled Transparency and reproducibility in artificial intelligence, the authors offer numerous frameworks and platforms that allow safe and effective sharing to uphold the three pillars of open science to make AI research more transparent and reproducible: sharing data, sharing computer code and sharing predictive models.
“We have high hopes for the utility of AI for our cancer patients,” says Dr. Haibe-Kains. “Sharing and building upon our discoveries – that’s real scientific impact.”
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
Artificial intelligence research
- University of Minnesota part of Toyota Research Institute’s new initiativeon January 26, 2021 at 10:01 am
U of M researchers will use artificial intelligence to better understand drivers' behavior to increase safe driving ...
- Snap looks to enhance augmented reality features with acquisition of British research lab Ariel AIon January 26, 2021 at 9:39 am
Snap, the parent company of Snapchat, has acquired a British artificial intelligence start-up called Ariel AI which focuses on augmented reality.
- Real-world Lab 'Robotic Artificial Intelligence' to be established at KITon January 26, 2021 at 8:20 am
Artificial intelligence (AI) is subject of films and stories, but it also is reality. Superhuman capacity of AI is obvious in games as well as in image and language processing or in autonomous driving ...
- Artificial Intelligence Will Define Google’s Future. For Now, It’s a Management Challenge.on January 26, 2021 at 7:02 am
Google’s parent, Alphabet, has waded through one controversy after another involving its top researchers and executives in the field over the past 18 months.
- Five ways artificial intelligence can help space explorationon January 26, 2021 at 6:11 am
Artificial intelligence has been making waves in recent years, enabling us to solve problems faster than traditional computing could ever allow. Recently, for example, Google's artificial intelligence ...
Go deeper with Google Headlines on:
Artificial intelligence research
Go deeper with Bing News on:
Artificial intelligence research transparency
- Artificial Intelligence (AI) in Fintech Market Size, Historical Growth, Analysis, Opportunities and Forecast To 2025on January 21, 2021 at 12:21 am
Selbyville, Delaware According to the research report titled 'Global Artificial Intelligence (AI) in Fintech Market Analysis, 2020', available with Market Study Report LLC, global artificial ...
- Artificial intelligence can deepen social inequality. Here are 5 ways to help prevent thison January 12, 2021 at 11:10 am
If the historical data used to train an AI system disadvantages certain minority groups, the system can be swayed to follow these patterns in its own decision-making process.
- FDA’s Action Plan for Artificial Intelligence: Highlights and Insights for Developerson January 11, 2021 at 4:00 pm
The FDA plans to hold a public workshop to elicit input from stakeholders on end user transparency ... Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based ...
- Scientists Press AI Researchers for Transparencyon January 8, 2021 at 11:46 am
An international group of scientists is demanding scientific journals demand more transparency ... in Nature last January. The research claimed an artificial intelligence system was faster and ...
- Ping An Uses Artificial Intelligence to Detect Greenwashing and Assess Impact of Climate Risk Disclosure Transparency on Financial Performanceon December 8, 2020 at 4:06 pm
The research partnership focuses on developing methodologies using artificial intelligence (AI) and big data to assess the risks for investment assets from climate change and other ESG-related ...