
via OpenAI
An updated analysis from OpenAI shows how dramatically the need for computational resources has increased to reach each new AI breakthrough.
In 2018, OpenAI found that the amount of computational power used to train the largest AI models had doubled every 3.4 months since 2012.
The San Francisco-based for-profit AI research lab has now added new data to its analysis. This shows how the post-2012 doubling compares with the historic doubling time since the beginning of the field. From 1959 to 2012, the amount of power used doubled every two years, tracking Moore’s Law. This means the resources used today are doubling at a rate seven times faster than before.
This dramatic increase in the resources needed underscores just how costly the field’s achievements have become. Keep in mind that the above graph shows a logarithmic scale. On a linear scale (below), you can more clearly see how compute usage has increased by 300,000-fold in the last seven years.

via OpenAI
The chart also notably does not include some of the most recent breakthroughs, including Google’s large-scale language model BERT, OpenAI’s language model GPT-2, or DeepMind’s StarCraft II-playing model AlphaStar.
In the past year, more and more researchers have sounded the alarm on the exploding costs of deep learning. In June, an analysis from researchers at the University of Massachusetts, Amherst, showed how these increasing computational costs directly translate into carbon emissions.
In their paper, they also noted how the trend exacerbates the privatization of AI research because it undermines the ability for academic labs to compete with much more resource-rich private ones.
In response to this growing concern, several industry groups have made recommendations. The Allen Institute for Artificial Intelligence, a nonprofit research firm in Seattle, has proposed that researchers always publish the financial and computational costs of training their models along with their performance results, for example.
In its own blog, OpenAI suggested policymakers increase funding to academic researchers to bridge the resource gap between academic and industry labs.
Learn more: The computing power needed to train AI is now rising seven times faster than ever before
The Latest Google Headlines on:
Deep learning computational resources
[google_news title=”” keyword=”deep learning computational resources” num_posts=”10″ blurb_length=”0″ show_thumb=”left”]
The Latest Bing News on:
Deep learning computational resources
- Stable Diffusion Benchmarked: Which GPU Runs AI Fastest (Updated)on January 26, 2023 at 7:33 am
We test all the modern graphics cards in Stable Diffusion and show which ones are fastest, along with a discussion of potential issues and other requirements.
- Report Outlines Creation of a $2.6B U.S. National AI Research Capabilityon January 25, 2023 at 12:18 pm
The Biden White House on Tuesday announced the release of a final report (PDF download) outlining a three-year plan to build a National Artificial Intelligence Research Resource (NAIRR).
- Large Hadron Collider Tech Might Help Self-Driving Cars "See" the Roadon January 25, 2023 at 10:00 am
A three- year study between Volvo’s autonomous software arm and the Large Hadron Collider in Europe shows promise in helping self-driving cars understand the road and make split decisions. The Large ...
- Project could enable self-driving cars to make better decisions faster and avoid collisionson January 25, 2023 at 7:34 am
In the future, autonomous or self-driving cars are expected to considerably reduce the number of road accident fatalities. Advancing developments on this revolutionary road, CERN and car-safety ...
- What Will Ediscovery Lawyers Do After ChatGPT?on January 25, 2023 at 6:06 am
A practical exercise in ChatGPT's capabilities and limitations when applied to a variety of common ediscovery tasks.
- Why TensorFlow for Python is dying a slow deathon January 13, 2023 at 6:47 am
Many developers who use Python for machine learning are now switching to PyTorch. Find out why and what the future could hold for TensorFlow.
- Deep learning-designed diffractive processor computes hundreds of transformations in parallelon January 8, 2023 at 4:00 pm
Thus, it is necessary to develop hardware resources that can perform ... through the combination of optics and deep learning to optically perform complex computational tasks such as image ...
- 3 AI trends in drug discovery that stood out in 2022on January 2, 2023 at 3:47 pm
Find out where AI made major contributions to drug discovery in 2022 — and what may be in store for AI in pharmacology in 2023.
- Conventional Wisdom Watch: Matsuoka & Co. Take on 12 Myths of HPCon December 31, 2000 at 4:00 pm
A group of HPC thinkers, including the estimable Satoshi Matsuoka of the RIKEN Center for Computational Science in Japan, have come together to challenge common lines of thought they say have become, ...
The Latest Google Headlines on:
Artificial intelligence
[google_news title=”” keyword=”artificial intelligence” num_posts=”10″ blurb_length=”0″ show_thumb=”left”]
The Latest Bing News on:
Artificial intelligence
- Researchers develop AI-based strategy for discovering high-affinity antibody drugson January 30, 2023 at 4:21 pm
Scientists at University of California San Diego School of Medicine have developed an artificial intelligence (AI)-based strategy for discovering high-affinity antibody drugs.
- Terry Savage: Artificial intelligence for money managementon January 30, 2023 at 2:56 pm
OpenAI, whose artificial intelligence tool ChatGPT has lit up the internet since its introduction in November. (Rokas Tenys/Dreamstime/TNS) Every few years, something really “new” comes along, ...
- Artificial intelligence can help patients interpret home tests for COVID-19on January 30, 2023 at 2:03 pm
New machine learning research led by George Mason University's Professor Farrokh Alemi and Professor Janusz Wojtusiak provides a way for patients and clinicians to better predict whether symptoms are ...
- Artificial intelligence predicts climate change coming faster than we recently thought, new study sayson January 30, 2023 at 12:00 pm
The paper used artificial neural networks to model how much the climate has warmed in the past and how much it's likely to warm in the future.
- BuzzFeed to use artificial intelligence for contenton January 30, 2023 at 11:48 am
AI chatbots are software applications used to have a conversation that can take on the role of a live human using technology.
- Is brain learning slower than artificial intelligence? Read on to know what this study sayson January 30, 2023 at 10:23 am
Artificial intelligence has traditionally been derived from human brain dynamics. However, as compared to deep learning, brain learning has several important limitations (DL). First, efficient DL wire ...
- Artificial Intelligence Market [New 117 Pages Report]: Foretelling Strategy to bear Anticipated Growth by 2026on January 30, 2023 at 8:37 am
Artificial Intelligence Market Size [2023 to 2026] Research Report is spread across 117 Pages and provides private data, statistics, vital statistics, Exclusive trends, and competitive landscape ...
- Artificial Intelligence comes for Hollywoodon January 30, 2023 at 8:14 am
Artificial intelligence is quickly gaining a foothold in Hollywood and is primed to be one of the hottest investment areas over the next few years. Why it matters: New forms of generative AI — it's ...
- Can the brain compete with advanced artificial intelligence systems?on January 30, 2023 at 6:55 am
Artificial intelligence stems from human brain dynamics, but brain learning is restricted in a number of significant aspects compared to deep learning.
- Kettering students training artificial intelligence models for school bus safetyon January 30, 2023 at 6:30 am
A group of undergraduate computer science students at Kettering University are developing a system to implement cheap, reliable artificial intelligence models on school buses.