
via OpenAI
An updated analysis from OpenAI shows how dramatically the need for computational resources has increased to reach each new AI breakthrough.
In 2018, OpenAI found that the amount of computational power used to train the largest AI models had doubled every 3.4 months since 2012.
The San Francisco-based for-profit AI research lab has now added new data to its analysis. This shows how the post-2012 doubling compares with the historic doubling time since the beginning of the field. From 1959 to 2012, the amount of power used doubled every two years, tracking Moore’s Law. This means the resources used today are doubling at a rate seven times faster than before.
This dramatic increase in the resources needed underscores just how costly the field’s achievements have become. Keep in mind that the above graph shows a logarithmic scale. On a linear scale (below), you can more clearly see how compute usage has increased by 300,000-fold in the last seven years.

via OpenAI
The chart also notably does not include some of the most recent breakthroughs, including Google’s large-scale language model BERT, OpenAI’s language model GPT-2, or DeepMind’s StarCraft II-playing model AlphaStar.
In the past year, more and more researchers have sounded the alarm on the exploding costs of deep learning. In June, an analysis from researchers at the University of Massachusetts, Amherst, showed how these increasing computational costs directly translate into carbon emissions.
In their paper, they also noted how the trend exacerbates the privatization of AI research because it undermines the ability for academic labs to compete with much more resource-rich private ones.
In response to this growing concern, several industry groups have made recommendations. The Allen Institute for Artificial Intelligence, a nonprofit research firm in Seattle, has proposed that researchers always publish the financial and computational costs of training their models along with their performance results, for example.
In its own blog, OpenAI suggested policymakers increase funding to academic researchers to bridge the resource gap between academic and industry labs.
Learn more: The computing power needed to train AI is now rising seven times faster than ever before
The Latest Google Headlines on:
Deep learning computational resources
[google_news title=”” keyword=”deep learning computational resources” num_posts=”10″ blurb_length=”0″ show_thumb=”left”]
The Latest Bing News on:
Deep learning computational resources
- Vision Transformers Market Worth $1.2 billion by 2028, Growing At a CAGR of 34.2% Report by MarketsandMarkets™on November 29, 2023 at 6:00 am
The global Vision Transformers Market size is projected to grow from USD 0.2 billion in 2023 to USD 1.2 billion by 2028 at a growth rate of 34.2% during the forecast period, according to a new report ...
- Deep Learning and the Future of Consumer Hardwareon November 24, 2023 at 5:12 am
This subset of machine learning, which mimics the neural networks of the human brain, requires significant computational power. As a result, the future of consumer hardware is inextricably linked to ...
- Before Deep Learning: A Journey into the Pre-2000 Era of Neural Networkson November 24, 2023 at 3:33 am
Introduction The roots of deep learning stretch far beyond the current buzz. Before the advent of deep learning, the realm of neural networks had already started to take shape, laying the foundation ...
- Technique enables AI on edge devices to keep learning over timeon November 16, 2023 at 6:24 am
Personalized deep-learning models can enable artificial intelligence chatbots that adapt to understand a user's accent or smart keyboards that continuously update to better predict the next word based ...
- The Emergence Of Organoid Intelligence: Reshaping AI With Miniature Brainson November 15, 2023 at 6:00 am
This fusion could result in a new era of "hybrid intelligence" that combines the analytical power of AI with the nuanced understanding of human-like cognition.
- Machine Learning vs Deep Learning what are the differences?on November 2, 2023 at 5:00 pm
As for application scope, machine learning is suitable for problems with limited data and computational resources, whereas deep learning excels at tasks that involve massive amounts of data.
- Deep Learning in Computational Materials Scienceon April 14, 2023 at 4:38 am
Riding with the current wave of Artificial Intelligence (AI), many engineers and scientists have adopted machine learning and deep learning as powerful tools in various engineering disciplines, ...
- Computational Methods for Deep Learningon November 1, 2022 at 12:38 pm
Historia cen - trend cenowy ?Historia cen dostępna jest po zalogowaniu się. Dzięki niej możesz sprawdzić aktualny trend cenowy, wzrost lub spadek ceny oraz sezonowe obniżki cen produktów.
- Machine and Deep-learning for Computational Neuroscienceon July 5, 2022 at 3:54 am
As machine learning (ML) and Deep learning (DL) progresses from purely theoretical models to real implementations, the issue of ML safety and fairness becomes increasingly important. In neuroscience, ...
The Latest Google Headlines on:
Artificial intelligence
[google_news title=”” keyword=”artificial intelligence” num_posts=”10″ blurb_length=”0″ show_thumb=”left”]
The Latest Bing News on:
Artificial intelligence
- How artificial intelligence is changing health care in treating stroke victimson November 29, 2023 at 11:01 pm
I have witnessed many advances in the diagnosis and treatment of patients with stroke. However, with the advent of artificial intelligence (AI), we have a new powerful ally.
- OpenAI may have made a ‘dangerous’ artificial intelligence discover that led to chaos, Elon Musk sayson November 29, 2023 at 3:39 pm
Elon Musk co-founded OpenAI, as part of his response to concerns that artificial intelligence could prove dangerous to humanity. But he has been critical of its recent direction, including its turn ...
- Strictly Legal | Artificial intelligence on trialon November 29, 2023 at 2:10 pm
Netflix just announced that Rockstar’s Grand Theft Auto: The Trilogy – The Definitive Edition will hit its platform on December 14. The open-world mayhem simulator joins more than 80 other games, ...
- 'World's oldest languages' - that were carved into 5,000-year-old stones - can now be deciphered by artificial intelligence as fast as Google translateon November 29, 2023 at 10:14 am
Cuneiform tablets are up to 5,000 years old and present details of life in ancient Mesopotamia. Now scientists have trained an AI program to scan their text and detect what it says.
- Artificial Intelligence Needs A Shrinkon November 29, 2023 at 6:33 am
There are widely reported concerns about ChatGPT hallucinations. If this were a human being, we’d be assessing their psychological fitness. As artificial intelligence (AI) begins to permeate our world ...
- A Bull Market Is Coming: 2 Magnificent Artificial Intelligence (AI) Stocks I'm Buying Before 2023 Is Overon November 28, 2023 at 10:28 pm
Detailed price information for Crowdstrike Holdings Inc (CRWD-Q) from The Globe and Mail including charting and trades.
- Artificial Intelligence Needs Spiritual Intelligence | Opinionon November 28, 2023 at 3:00 pm
The future of artificial intelligence will require spiritual intelligence, or "the human capacity to ask questions about the ultimate meaning of life and the integrated relationship between us and the ...
- The US is racing ahead in its bid to control artificial intelligence – why is the EU so far behind?on November 28, 2023 at 8:09 am
Washington is laying down rules for the use of seemingly mundane AI that could, in fact, be incredibly dangerous, says professor of philosophy Seth Lazar ...
- 1 Artificial Intelligence (AI) Stock on Track to Be a Trillion-Dollar Companyon November 28, 2023 at 3:34 am
Detailed price information for Palantir Technologies Inc Cl A (PLTR-N) from The Globe and Mail including charting and trades.
- The Rise Of Artificial Intelligence In Marketingon November 27, 2023 at 4:15 am
Today, we possess new AI tools to scrutinize data sets, suggest changes and create innovative content like never before.