Connect with us

Technology

Charted: The Exponential Growth in AI Computation

Published

on

Click to view this graphic in a higher-resolution.

A time series chart showing the creation of machine learning systems on the x-axis and the amount of AI computation they used on the y-axis measured in FLOPs.

Charted: The Exponential Growth in AI Computation

Electronic computers had barely been around for a decade in the 1940s, before experiments with AI began. Now we have AI models that can write poetry and generate images from textual prompts. But what’s led to such exponential growth in such a short time?

This chart from Our World in Data tracks the history of AI through the amount of computation power used to train an AI model, using data from Epoch AI.

The Three Eras of AI Computation

In the 1950s, American mathematician Claude Shannon trained a robotic mouse called Theseus to navigate a maze and remember its course—the first apparent artificial learning of any kind.

Theseus was built on 40 floating point operations (FLOPs), a unit of measurement used to count the number of basic arithmetic operations (addition, subtraction, multiplication, or division) that a computer or processor can perform in one second.

ℹ️ FLOPs are often used as a metric to measure the computational performance of computer hardware. The higher the FLOP count, the higher computation, the more powerful the system.

Computation power, availability of training data, and algorithms are the three main ingredients to AI progress. And for the first few decades of AI advances, compute, which is the computational power needed to train an AI model, grew according to Moore’s Law.

PeriodEraCompute Doubling
1950–2010Pre-Deep Learning18–24 months
2010–2016Deep Learning5–7 months
2016–2022Large-scale models11 months

Source: “Compute Trends Across Three Eras of Machine Learning” by Sevilla et. al, 2022.

However, at the start of the Deep Learning Era, heralded by AlexNet (an image recognition AI) in 2012, that doubling timeframe shortened considerably to six months, as researchers invested more in computation and processors.

With the emergence of AlphaGo in 2015—a computer program that beat a human professional Go player—researchers have identified a third era: that of the large-scale AI models whose computation needs dwarf all previous AI systems.

Predicting AI Computation Progress

Looking back at the only the last decade itself, compute has grown so tremendously it’s difficult to comprehend.

For example, the compute used to train Minerva, an AI which can solve complex math problems, is nearly 6 million times that which was used to train AlexNet 10 years ago.

Here’s a list of important AI models through history and the amount of compute used to train them.

AIYearFLOPs
Theseus195040
Perceptron Mark I1957–58695,000
Neocognitron1980228 million
NetTalk198781 billion
TD-Gammon199218 trillion
NPLM20031.1 petaFLOPs
AlexNet2012470 petaFLOPs
AlphaGo20161.9 million petaFLOPs
GPT-32020314 million petaFLOPs
Minerva20222.7 billion petaFLOPs

Note: One petaFLOP = one quadrillion FLOPs. Source: “Compute Trends Across Three Eras of Machine Learning” by Sevilla et. al, 2022.

The result of this growth in computation, along with the availability of massive data sets and better algorithms, has yielded a lot of AI progress in seemingly very little time. Now AI doesn’t just match, but also beats human performance in many areas.

It’s difficult to say if the same pace of computation growth will be maintained. Large-scale models require increasingly more compute power to train, and if computation doesn’t continue to ramp up it could slow down progress. Exhausting all the data currently available for training AI models could also impede the development and implementation of new models.

However with all the funding poured into AI recently, perhaps more breakthroughs are around the corner—like matching the computation power of the human brain.

Where Does This Data Come From?

Source: “Compute Trends Across Three Eras of Machine Learning” by Sevilla et. al, 2022.

Note: The time estimated to for computation to double can vary depending on different research attempts, including Amodei and Hernandez (2018) and Lyzhov (2021). This article is based on our source’s findings. Please see their full paper for further details. Furthermore, the authors are cognizant of the framing concerns with deeming an AI model “regular-sized” or “large-sized” and said further research is needed in the area.

Methodology: The authors of the paper used two methods to determine the amount of compute used to train AI Models: counting the number of operations and tracking GPU time. Both approaches have drawbacks, namely: a lack of transparency with training processes and severe complexity as ML models grow.

green check mark icon

This article was published as a part of Visual Capitalist's Creator Program, which features data-driven visuals from some of our favorite Creators around the world.

Click for Comments

Technology

Chart: The Price of Entertainment Subscription Services

From Netflix to Google Play Pass, we compare the cost of subscription services across a range of entertainment platforms.

Published

on

This bar chart shows the price of entertainment subscription services per month.

The Price of Entertainment Subscription Services

This was originally posted on our Voronoi app. Download the app for free on iOS or Android and discover incredible data-driven charts from a variety of trusted sources.

Subscription models have become ubiquitous in the entertainment sector, providing a recurring stream of revenue to a host of platforms.

While inattentive customers using subscription models can increase revenue by as much as 200%, many entertainment platforms struggle to make a profit. In fact, Netflix and Disney are the only two profitable streaming services in the market.

This graphic shows the cost of entertainment subscription services, based on data compiled by Goldman Sachs Global Investment Research and the providers of entertainment subscriptions.

Comparing Monthly Entertainment Subscription Costs

Here are the monthly subscription cost of various entertainment platforms as of April 2024:

SubscriptionMonthly Price (USD)Category
Spotify$10.99Music
YouTube Music$10.99Music
Apple Music$10.99Music
Audible$15.00Books
Scribd$11.99Books
Kindle Unlimited$11.99Books
Netflix$15.49Video
Sling TV$40.00Video
Disney+$13.99Video
Hulu+$17.99Video
Paramount+$11.99Video
Apple TV+$9.99Video
HBO Max$15.99Video
Amazon Prime Video$11.98Video
YouTube Premium$13.99Video
Apple Arcade$6.99Gaming
Google Play Pass$4.99Gaming
Xbox Game Pass Ultimate$16.99Gaming
Playstation Plus Premium$17.99Gaming
Nintendo Switch Online$3.99Gaming
NY Times (Digital)$4.00/month first six months,
$25 thereafter
News
Apple News+$12.99News
Wall Street Journal Digital$19.49/month first six months,
$38.99 thereafter
News

Prices represent standard individual plans with no ads excluding promotional periods/prices less than two months. YouTube Premium Video subscription includes YouTube Music which can be purchased seperately.

As we can see, the price of major music subscription services remains lower than many other forms of entertainment—with standard subscriptions costing 35% lower than Netflix in America.

Since late 2022, several music streaming platforms including Spotify, Apple, and YouTube, have increased their subscription price, marking the first increase in more than 10 years. In June, Spotify raised its price again, charging $11.99 per month for an individual plan.

Across video platforms, Amazon Prime Video makes up the largest share of the U.S. video-on-demand market, at 22% as of Q1 2024. Netflix falls closely behind, with a 21% share. Over the last two years, Netflix’s revenue has jumped following a password-sharing crackdown, integrating ads, and slowing content expenditures.

Often, streaming services add content to replace lost customers. This is because viewers will switch to providers that offer the shows they want to watch. Due to this churn, streaming providers lose on average 35% of their customers each year. To combat this, some providers are bundling content offerings to retain their customer base, such as Disney+, Hulu, and Max or Paramount+ and Showtime.

As an outlier from the pack, Sling TV offers live TV and sports broadcasting along with on-demand movies and shows, charging $40 per month.

When it comes to news subscriptions, major outlets charge among the highest in the dataset. With a monthly subscription price of $25 after the first six months, The New York Times has 9.7 million digital-only subscribers, roughly three times as many as The Wall Street Journal. These subscriptions are the biggest source of revenue for the publication, rising by more than eightfold over the last decade.

Continue Reading
Range ETFs Display Ad

Subscribe

Popular