2024/4/17 10:39 AI Index: State of AI in 13 Charts

Economy and Markets, Language Processing, Law, Regulation, and Policy

AI Index: State of AI in 13 Charts

In the new report, foundation models dominate,
benchmarks fall, prices skyrocket, and on the global
stage, the U.S. overshadows.
Apr 15, 2024 | Shana Lynch

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 1/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 2/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

This year’s AI Index — a 300-plus-page report

tracking 2023’s worldwide trends in AI — is out.

The index is an independent initiative at

the Stanford Institute for Human-Centered
Artificial Intelligence (HAI), led by the AI Index
Steering Committee, an interdisciplinary group of
experts from across academia and industry. This
year’s report covers the rise of multimodal
foundation models, major cash investments into

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 3/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

generative AI, new performance benchmarks,

shifting global opinions, and new major regulations.

Don’t have an afternoon to pour through the

findings? Check out the high level here.

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 4/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

A Move Toward Open-Sourced

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 5/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

This past year, organizations released 149

foundation models, more than double the number
released in 2022. Of these newly released models,
65.7% were open-source (meaning they can be
freely used and modified by anyone), compared
with only 44.4% in 2022 and 33.3% in 2021.

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 6/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

But At a Cost of Performance?

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 7/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

Closed-source models still outperform their open-

sourced counterparts. On 10 selected benchmarks,
closed models achieved a median performance
advantage of 24.2%, with differences ranging from
as little as 4.0% on mathematical tasks like GSM8K
to as much as 317.7% on agentic tasks like
AgentBench.

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 8/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

Biggest Players

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 9/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

Industry dominates AI, especially in building and

releasing foundation models. This past year Google
edged out other industry players in releasing the
most models, including Gemini and RT-2. In fact,
since 2019, Google has led in releasing the most
foundation models, with a total of 40, followed by
OpenAI with 20. Academia trails industry: This past
year, UC Berkeley released three models and
Stanford two.

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 10/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

Industry Dwarfs All

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 11/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

If you needed more striking evidence that corporate

AI is the only player in the room right now, this
should do it. In 2023, industry accounted for 72% of
all new foundation models.

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 12/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

Prices Skyrocket

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 13/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

One of the reasons academia and government have

been edged out of the AI race: the exponential
increase in cost of training these giant models.
Google’s Gemini Ultra cost an estimated $191
million worth of compute to train, while OpenAI’s
GPT-4 cost an estimated $78 million. In comparison,
in 2017, the original Transformer model, which
introduced the architecture that underpins virtually
every modern LLM, cost around $900.

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 14/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

What AI Race?

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 15/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

At least in terms of notable machine learning

models, the United States vastly outpaced other
countries in 2023, developing a total of 61 models
in 2023. Since 2019, the U.S. has consistently led in
originating the majority of notable models, followed
by China and the UK.

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 16/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

Move Over, Human

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 17/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

As of 2023, AI has hit human-level performance on

many significant AI benchmarks, from those testing
reading comprehension to visual reasoning. Still, it
falls just short on some benchmarks like
competition-level math. Because AI has been
blasting past so many standard benchmarks, AI
scholars have had to create new and more difficult
challenges. This year’s index also tracked several
of these new benchmarks, including those for tasks
in coding, advanced reasoning, and agentic
behavior.

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 18/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

Private Investment Drops (But We See You, GenAI)

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 19/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

While AI private investment has steadily dropped

since 2021, generative AI is gaining steam. In 2023,
the sector attracted $25.2 billion, nearly ninefold the
investment of 2022 and about 30 times the amount
from 2019 (call it the ChatGPT effect). Generative AI
accounted for over a quarter of all AI-related private
investments in 2023.

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 20/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

U.S. Wins $$ Race

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 21/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

And again, in 2023 the United States dominates in

AI private investment. In 2023, the $67.2 billion
invested in the U.S. was roughly 8.7 times greater
than the amount invested in the next highest
country, China, and 17.8 times the amount invested
in the United Kingdom. That lineup looks the same
when zooming out: Cumulatively since 2013, the
United States leads investments at $335.2 billion,
followed by China with $103.7 billion, and the
United Kingdom at $22.3 billion.

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 22/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

Where is Corporate Adoption?

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 23/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

More companies are implementing AI in some part

of their business: In surveys, 55% of organizations
said they were using AI in 2023, up from 50% in
2022 and 20% in 2017. Businesses report using AI to
automate contact centers, personalize content, and
acquire new customers.

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 24/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

Younger and Wealthier People Worry About Jobs

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 25/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

Globally, most people expect AI to change their

jobs, and more than a third expect AI to replace
them. Younger generations — Gen Z and millennials
— anticipate more substantial effects from AI
compared with older generations like Gen X and
baby boomers. Specifically, 66% of Gen Z compared
with 46% of boomer respondents believe AI will
significantly affect their current jobs. Meanwhile,
individuals with higher incomes, more education,
and decision-making roles foresee AI having a great
impact on their employment.

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 26/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

While the Commonwealth Worries About AI

Products

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 27/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

When asked in a survey about whether AI products

and services make you nervous, 69% of Aussies and
65% of Brits said yes. Japan is the least worried
about their AI products at 23%.

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 28/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

Regulation Rallies

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 29/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

More American regulatory agencies are passing

regulations to protect citizens and govern the use of
AI tools and data. For example, the Copyright Office
and the Library of Congress passed copyright
registration guidance concerning works that
contained material generated by AI, while the
Securities and Exchange Commission developed a
cybersecurity risk management strategy,
governance, and incident disclosure plan. The
agencies to pass the most regulation were the
Executive Office of the President and the Commerce
Department.

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 30/31
2024/4/17 10:39 AI Index: State of AI in 13 Charts

The AI Index was first created to track AI

development. The index collaborates with such
organizations as LinkedIn, Quid, McKinsey,
Studyportals, the Schwartz Reisman Institute, and
the International Federation of Robotics to gather
the most current research and feature important
insights on the AI ecosystem.

More News Topics

Economy and Markets, Language Processing, Law, Regulation, and Policy, Machine Learning

https://hai.stanford.edu/news/ai-index-state-ai-13-charts 31/31