Education
How Smart is ChatGPT?
Visualizing ChatGPT’s Performance in Human Exams
ChatGPT, a language model developed by OpenAI, has become incredibly popular over the past year due to its ability to generate human-like responses in a wide range of circumstances.
In fact, ChatGPT has become so competent, that students are now using it to help them with their homework. This has prompted several U.S. school districts to block devices from accessing the model while on their networks.
So, how smart is ChatGPT?
In a technical report released on March 27, 2023, OpenAI provided a comprehensive brief on its most recent model, known as GPT-4. Included in this report were a set of exam results, which we’ve visualized in the graphic above.
GPT-4 vs. GPT-3.5
To benchmark the capabilities of ChatGPT, OpenAI simulated test runs of various professional and academic exams. This includes SATs, the bar examination, and various advanced placement (AP) finals.
Performance was measured in percentiles, which were based on the most recently available score distributions for test takers of each exam type.
Percentile scoring is a way of ranking one’s performance relative to the performance of others. For instance, if you placed in the 60th percentile on a test, this means that you scored higher than 60% of test-takers.
The following table lists the results that we visualized in the graphic.
Category | Exam | GPT-4 Percentile | GPT-3.5 Percentile |
---|---|---|---|
Law | Uniform Bar Exam | 90 | 10 |
Law | LSAT | 88 | 40 |
SAT | Evidence-based Reading & Writing | 93 | 87 |
SAT | Math | 89 | 70 |
Graduate Record Examination (GRE) | Quantitative | 80 | 25 |
Graduate Record Examination (GRE) | Verbal | 99 | 63 |
Graduate Record Examination (GRE) | Writing | 54 | 54 |
Advanced Placement (AP) | Biology | 85 | 62 |
Advanced Placement (AP) | Calculus | 43 | 0 |
Advanced Placement (AP) | Chemistry | 71 | 22 |
Advanced Placement (AP) | Physics 2 | 66 | 30 |
Advanced Placement (AP) | Psychology | 83 | 83 |
Advanced Placement (AP) | Statistics | 85 | 40 |
Advanced Placement (AP) | English Language | 14 | 14 |
Advanced Placement (AP) | English Literature | 8 | 8 |
Competitive Programming | Codeforces Rating | <5 | <5 |
The scores reported above are for GPT-4 with visual inputs enabled. Please see OpenAI’s technical report for more comprehensive results.
As we can see, GPT-4 (released in March 2023) is much more capable than GPT-3.5 (released March 2022) in the majority of these exams. It was, however, unable to improve in AP English and in competitive programming.
Regarding AP English (and other exams where written responses were required), ChatGPT’s submissions were graded by “1-2 qualified third-party contractors with relevant work experience grading those essays”. While ChatGPT is certainly capable of producing adequate essays, it may have struggled to comprehend the exam’s prompts.
For competitive programming, GPT attempted 10 Codeforces contests 100 times each. Codeforces hosts competitive programming contests where participants must solve complex problems. GPT-4’s average Codeforces rating is 392 (below the 5th percentile), while its highest on a single contest was around 1,300. Referencing the Codeforces ratings page, the top-scoring user is jiangly from China with a rating of 3,841.
What’s Changed With GPT-4?
Here are some areas where GPT-4 has improved the user experience over GPT-3.5.
Internet Access and Plugins
A limiting factor with GPT-3.5 was that it didn’t have access to the internet and was only trained on data up to June 2021.
With GPT-4, users will have access to various plugins that empower ChatGPT to access the internet, provide more up to date responses, and complete a wider range of tasks. This includes third-party plugins from services such as Expedia which will enable ChatGPT to book an entire vacation for you.
Visual Inputs
While GPT-3.5 could only accept text inputs, GPT-4 has the ability to also analyze images. Users will be able to ask ChatGPT to describe a photo, analyze a chart, or even explain a meme.
Greater Context Length
Lastly, GPT-4 is able to handle much larger amounts of text and keep conversations going for longer. For reference, GPT-3.5 had a max request value of 4,096 tokens, which is equivalent to roughly 3,000 words. GPT-4 has two variants, one with 8,192 tokens (6,000 words) and another with 32,768 tokens (24,000 words).
![]() |
Interested in learning more about the impact artificial intelligence is having on the world of work? VC+ members have access to this special dispatch as well as our entire archive of VC+ content. Find out more. |
United States
Ranked: Most Popular U.S. Undergraduate Degrees (2011–2021)
Which degrees have increased in popularity over the last decade? And which disciplines have seen fewer and fewer students?

Ranked: Most Popular U.S. Undergraduate Degrees (2011–2021)
In an era of soaring tuition fees and mounting student debt, choosing which undergraduate degree to pursue has become a crucial decision for any aspiring college student. And it always helps to see which way the winds are blowing.
This visualization by Kashish Rastogi, based on data from the National Center for Education Statistics (NCES), examines the changing landscape of undergraduate degrees awarded between the 2010–2011 and 2020–2021 academic years.
Undergraduate Degrees Growing in Popularity
The NCES classifies all four-year bachelor degrees into 38 fields of study. Of these fields, 21 saw an increase in graduates in 2020–2021 compared to 2010–2011.
While only those with more than 30,000 graduates have been shown in the graphic (to prevent overrepresentation of large changes in small pools of graduates), the full list is available below.
Rank | Field of Study | 2010–2011 | 2020–2021 | % Change |
---|---|---|---|---|
1 | Business | 363,919 | 390,781 | +7% |
2 | Health Professions | 143,463 | 268,018 | +87% |
3 | Biomedical Sciences | 89,984 | 131,499 | +46% |
4 | Psychology | 100,906 | 126,944 | +26% |
5 | Engineering | 76,356 | 126,037 | +65% |
6 | Computer Sciences | 43,066 | 104,874 | +144% |
7 | Communication | 83,231 | 90,775 | +9% |
8 | Security & Law Enforcement | 47,600 | 58,009 | +22% |
9 | Interdisciplinary Studies | 42,473 | 54,584 | +29% |
10 | Leisure & Fitness Studies | 35,934 | 54,294 | +51% |
11 | Public Administration | 26,799 | 34,817 | +30% |
12 | Physical Sciences | 24,338 | 28,706 | +18% |
13 | Mathematics | 17,182 | 27,092 | +58% |
14 | Agriculture Sciences | 15,851 | 21,418 | +35% |
15 | Natural Resources & Conservation | 12,779 | 20,507 | +61% |
16 | Engineering Technologies | 16,187 | 18,562 | +15% |
17 | Transportation | 4,941 | 5,993 | +21% |
18 | Legal | 4,429 | 4,589 | +4% |
19 | Military Technologies | 64 | 1,524 | +2,281% |
20 | Science Technologies | 367 | 532 | +45% |
21 | Library Science | 96 | 119 | +24% |
Let’s take a look at the areas of study that were most popular, as well as some of the fastest growing fields:
Computer and Information Sciences
Bachelor’s degrees in this discipline have grown by 144% since 2010–2011, with over 100,000 graduates in 2020–2021. The allure of the tech sector’s explosive growth likely contributed to its popularity among students.
Health Professions
Undergraduate degrees in health professions saw an 87% increase, attracting nearly 260,000 graduates in 2020–2021. This field accounted for 13% of the total graduating class, reflecting the growing appeal of the healthcare sector.
Engineering
There were 50,000 more engineering graduates in the U.S. in 2021, up 65% from 2011. With a median income over $100,000 per year, engineering graduates can usually rely on good wages as well as versatility in future careers, capable of finding jobs in tech, design, and communication fields, and of course, becoming future entrepreneurs.
Biomedical Sciences
University graduates in this field, which focuses on the integration of the study of biology with health and medicine, grew by 46%. A subset of this category—epidemiology—has been in the limelight recently thanks to the COVID-19 pandemic.
Business
While this category recorded a modest 7% growth in graduates, its popularity has been indisputable in the last decade, representing the largest proportion of the graduating class in both 2011 and 2021.
Fields with Declining University Graduates (2011‒2021)
Meanwhile, 17 areas of study experienced declines in the number of completed university degrees. We explore some of the notable ones below:
Rank | Field of Study | 2010–2011 | 2020–2021 | % Change |
---|---|---|---|---|
1 | Social Sciences | 142,161 | 137,908 | -3% |
2 | Visual & Performing Arts | 93,939 | 90,022 | -4% |
3 | Education | 104,008 | 89,398 | -14% |
4 | Liberal Arts | 46,717 | 41,909 | -10% |
5 | English | 52,754 | 35,762 | -32% |
6 | History | 35,008 | 22,919 | -35% |
7 | Human Sciences | 22,438 | 22,319 | -1% |
8 | Foreign Languages | 21,705 | 15,518 | -29% |
9 | Philosophy & Religion | 12,830 | 11,988 | -7% |
10 | Architecture | 9,831 | 9,296 | -5% |
11 | Ethnic, Cultural & Gender Studies | 8,955 | 7,374 | -18% |
12 | Theology | 9,073 | 6,737 | -26% |
13 | Communications Tech | 4,858 | 4,557 | -6% |
14 | Personal & Culinary Services | 1,214 | 594 | -51% |
15 | Construction Trades | 328 | 221 | -33% |
16 | Mechanic & Repair | 226 | 221 | -2% |
17 | Precision Production | 43 | 28 | -35% |
English
Popular in the 1970s, the English undergraduate degree has gone through peaks (80s and 90s) and troughs (2000s and 10s) of popularity in the last 50 years. Between 2010–2011 and 2020–2021, the number of students with an English degree has fallen by a third.
The state of English’s woes are even making its way to pop culture, like in Netflix’s The Chair, which follows the head of a struggling English department at a major university.
Education
The existing teacher shortage in the United States does not seem to be getting fixed by a burgeoning supply of new grads. In fact, the number of university graduates in Education fell 14% between 2011 and 2021. With concerns around stagnant wages, burnout, and little to no support for supplies, many teachers are seeing an already demanding job becoming harder.
Liberal Arts
In the classic era, the liberal arts covered seven fields of study: rhetoric, grammar, logic, astronomy, mathematics, geometry, and music. Now, liberal art degrees include several other subjects: history, political science, and even philosophy—but students are meant to primarily walk away with critical thinking skills.
The modern world rewards specialization however, and a wider-scope liberal arts degree is seeing fewer takers, with a 10% drop in graduating students.
Where Does This Data Come From?
Source: The National Center for Education’s statistics from their Integrated Postsecondary Education Data System surveys. Numbers for both 2010–2011 and 2020–2021 academic years can be found from their summary tables by changing the award level code (bachelor degrees) and the year on the left-hand toolbar.
-
Wealth6 days ago
Mapped: The Migration of the World’s Millionaires in 2023
-
Maps2 weeks ago
Mapped: The Deadliest Earthquakes of the 21st Century
-
Oil and Gas5 days ago
Charted: The World’s Biggest Oil Producers
-
Inequality4 weeks ago
Visualizing the World’s Growing Millionaire Population (2012-2022)
-
Energy2 weeks ago
What Electricity Sources Power the World?
-
Space3 days ago
Which Companies Own the Most Satellites?
-
United States4 weeks ago
Mapped: The Richest Billionaires in U.S. States
-
Markets2 weeks ago
The 25 Worst Stocks by Shareholder Wealth Losses (1926-2022)