Connect with us

Technology

Visualizing the Training Costs of AI Models Over Time

Published

on

See this visualization first on the Voronoi app.

This bubble graphic shows the training costs of AI models over time.

Visualizing the Training Costs of AI Models Over Time

This was originally posted on our Voronoi app. Download the app for free on iOS or Android and discover incredible data-driven charts from a variety of trusted sources.

Training advanced AI models like OpenAI’s ChatGPT and Google’s Gemini Ultra requires millions of dollars, with costs escalating rapidly.

As computational demands increase, the expenses for the computing power necessary to train them are soaring. In response, AI companies are rethinking how they train generative AI systems. In many cases, these include strategies to reduce computational costs given current growth trajectories.

This graphic shows the surge in training costs for advanced AI models, based on analysis from Stanford University’s 2024 Artificial Intelligence Index Report.

How Training Cost is Determined

The AI Index collaborated with research firm Epoch AI to estimate AI model training costs, which were based on cloud compute rental prices. Key factors that were analyzed include the model’s training duration, the hardware’s utilization rate, and the value of the training hardware.

While many have speculated that training AI models has become increasingly costly, there is a lack of comprehensive data supporting these claims. The AI Index is one of the rare sources for these estimates.

Ballooning Training Costs

Below, we show the training cost of major AI models, adjusted for inflation, since 2017:

YearModel NameModel Creators/ContributorsTraining Cost (USD)
Inflation-adjusted
2017TransformerGoogle$930
2018BERT-LargeGoogle$3,288
2019RoBERTa LargeMeta$160,018
2020GPT-3 175B (davinci)OpenAI$4,324,883
2021Megatron-Turing NLG 530BMicrosoft/NVIDIA$6,405,653
2022LaMDAGoogle$1,319,586
2022PaLM (540B)Google$12,389,056
2023GPT-4OpenAI$78,352,034
2023Llama 2 70BMeta$3,931,897
2023Gemini UltraGoogle$191,400,000

Last year, OpenAI’s GPT-4 cost an estimated $78.4 million to train, a steep rise from Google’s PaLM (540B) model, which cost $12.4 million just a year earlier.

For perspective, the training cost for Transformer, an early AI model developed in 2017, was $930. This model plays a foundational role in shaping the architecture of many large language models used today.

Google’s AI model, Gemini Ultra, costs even more, at a staggering $191 million. As of early 2024, the model outperforms GPT-4 on several metrics, most notably across the Massive Multitask Language Understanding (MMLU) benchmark. This benchmark serves as a crucial yardstick for gauging the capabilities of large language models. For instance, its known for evaluating knowledge and problem solving proficiency across 57 subject areas.

Training Future AI Models

Given these challenges, AI companies are finding new solutions for training language models to combat rising costs.

These include a number of approaches, such as creating smaller models that are designed to perform specific tasks. Other companies are experimenting with creating their own, synthetic data to feed into AI systems. However, a clear breakthrough is yet to be seen.

Today, AI models using synthetic data have shown to produce nonsense when asked with certain prompts, triggering what is referred to as “model collapse”.

Click for Comments

Technology

Ranked: The 20 Biggest Tech Companies by Market Cap

In total, the 20 biggest tech companies are worth over $20 trillion—nearly 18% of the stock market value globally.

Published

on

A portion of the top 20 biggest tech companies visualized as bubbles sized by market cap with Apple as the biggest.

Ranked: The 20 Biggest Tech Companies by Market Cap

This was originally posted on our Voronoi app. Download the app for free on iOS or Android and discover incredible data-driven charts from a variety of trusted sources.

The world’s 20 biggest tech companies are worth over $20 trillion in total. To put this in perspective, this is nearly 18% of the stock market value globally.

This graphic shows which companies top the ranks, using data from Companiesmarketcap.com.

A Closer Look at The Top 20

Market capitalization (market cap) measures what a company is worth by taking the current share price and multiplying it by the number of shares outstanding. Here are the biggest tech companies according to their market cap on June 13, 2024.

RankCompanyCountry/RegionMarket Cap
1AppleU.S.$3.3T
2MicrosoftU.S.$3.3T
3NvidiaU.S.$3.2T
4AlphabetU.S.$2.2T
5AmazonU.S.$1.9T
6MetaU.S.$1.3T
7TSMCTaiwan$897B
8BroadcomU.S.$778B
9TeslaU.S.$582B
10TencentChina$453B
11ASMLNetherlands$415B
12OracleU.S.$384B
13SamsungSouth Korea$379B
14NetflixU.S.$281B
15AMDU.S.$258B
16QualcommU.S.$243B
17SAPGermany$225B
18SalesforceU.S.$222B
19PDD Holdings (owns Pinduoduo)China$212B
20AdobeU.S.$206B

Note: PDD Holdings says its headquarters remain in Shanghai, China, and Ireland is used for legal registration for its overseas business.

 

Apple is the largest tech company at the moment, having competed with Microsoft for the top of the leaderboard for many years. The company saw its market cap soar after announcing its generative AI, Apple Intelligence. Analysts believe people will upgrade their devices over the next few years, since the new features are only available on the iPhone 15 Pro or newer.

Microsoft is in second place in the rankings, partly thanks to enthusiasm for its AI software which is already generating revenue. Rising profits also contributed to the company’s value. For the quarter ended March 31, 2024, Microsoft increased its net income by 20% compared to the same quarter last year.

Nvidia follows closely behind with the third-highest market cap, rising more than eight times higher compared to its value at the start of 2023. The company has recently announced higher profits, introduced a higher dividend, and reported that its next-generation GPU chip will start generating revenue later this year.

AI a Driver of the Biggest Tech Companies

It’s clear from the biggest tech companies that involvement in AI can contribute to investor confidence.

Among S&P 500 companies, AI has certainly become a focus topic. In fact, 199 companies cited the term “AI” during their first quarter earnings calls, the highest on record. The companies who mentioned AI the most were Meta (95 times), Nvidia (86 times), and Microsoft (74 times).

Continue Reading
Voronoi, the app by Visual Capitalist. Where data tells the story. Download on App Store or Google Play

Subscribe

Popular