Education
How Smart is ChatGPT?
Visualizing ChatGPT’s Performance in Human Exams
ChatGPT, a language model developed by OpenAI, has become incredibly popular over the past year due to its ability to generate human-like responses in a wide range of circumstances.
In fact, ChatGPT has become so competent, that students are now using it to help them with their homework. This has prompted several U.S. school districts to block devices from accessing the model while on their networks.
So, how smart is ChatGPT?
In a technical report released on March 27, 2023, OpenAI provided a comprehensive brief on its most recent model, known as GPT-4. Included in this report were a set of exam results, which we’ve visualized in the graphic above.
GPT-4 vs. GPT-3.5
To benchmark the capabilities of ChatGPT, OpenAI simulated test runs of various professional and academic exams. This includes SATs, the bar examination, and various advanced placement (AP) finals.
Performance was measured in percentiles, which were based on the most recently available score distributions for test takers of each exam type.
Percentile scoring is a way of ranking one’s performance relative to the performance of others. For instance, if you placed in the 60th percentile on a test, this means that you scored higher than 60% of test-takers.
The following table lists the results that we visualized in the graphic.
| Category | Exam | GPT-4 Percentile | GPT-3.5 Percentile |
|---|---|---|---|
| Law | Uniform Bar Exam | 90 | 10 |
| Law | LSAT | 88 | 40 |
| SAT | Evidence-based Reading & Writing | 93 | 87 |
| SAT | Math | 89 | 70 |
| Graduate Record Examination (GRE) | Quantitative | 80 | 25 |
| Graduate Record Examination (GRE) | Verbal | 99 | 63 |
| Graduate Record Examination (GRE) | Writing | 54 | 54 |
| Advanced Placement (AP) | Biology | 85 | 62 |
| Advanced Placement (AP) | Calculus | 43 | 0 |
| Advanced Placement (AP) | Chemistry | 71 | 22 |
| Advanced Placement (AP) | Physics 2 | 66 | 30 |
| Advanced Placement (AP) | Psychology | 83 | 83 |
| Advanced Placement (AP) | Statistics | 85 | 40 |
| Advanced Placement (AP) | English Language | 14 | 14 |
| Advanced Placement (AP) | English Literature | 8 | 8 |
| Competitive Programming | Codeforces Rating | <5 | <5 |
The scores reported above are for GPT-4 with visual inputs enabled. Please see OpenAI’s technical report for more comprehensive results.
As we can see, GPT-4 (released in March 2023) is much more capable than GPT-3.5 (released March 2022) in the majority of these exams. It was, however, unable to improve in AP English and in competitive programming.
Regarding AP English (and other exams where written responses were required), ChatGPT’s submissions were graded by “1-2 qualified third-party contractors with relevant work experience grading those essays”. While ChatGPT is certainly capable of producing adequate essays, it may have struggled to comprehend the exam’s prompts.
For competitive programming, GPT attempted 10 Codeforces contests 100 times each. Codeforces hosts competitive programming contests where participants must solve complex problems. GPT-4’s average Codeforces rating is 392 (below the 5th percentile), while its highest on a single contest was around 1,300. Referencing the Codeforces ratings page, the top-scoring user is jiangly from China with a rating of 3,841.
What’s Changed With GPT-4?
Here are some areas where GPT-4 has improved the user experience over GPT-3.5.
Internet Access and Plugins
A limiting factor with GPT-3.5 was that it didn’t have access to the internet and was only trained on data up to June 2021.
With GPT-4, users will have access to various plugins that empower ChatGPT to access the internet, provide more up to date responses, and complete a wider range of tasks. This includes third-party plugins from services such as Expedia which will enable ChatGPT to book an entire vacation for you.
Visual Inputs
While GPT-3.5 could only accept text inputs, GPT-4 has the ability to also analyze images. Users will be able to ask ChatGPT to describe a photo, analyze a chart, or even explain a meme.
Greater Context Length
Lastly, GPT-4 is able to handle much larger amounts of text and keep conversations going for longer. For reference, GPT-3.5 had a max request value of 4,096 tokens, which is equivalent to roughly 3,000 words. GPT-4 has two variants, one with 8,192 tokens (6,000 words) and another with 32,768 tokens (24,000 words).
 |
Interested in learning more about the impact artificial intelligence is having on the world of work? VC+ members have access to this special dispatch as well as our entire archive of VC+ content. Find out more. |
Education
Ranked: The World’s Top 50 Endowment Funds
Endowment funds represent the investment arms of nonprofits. See the worlds top 50, which collectively have over $1 trillion in assets.
Ranked: The World’s Top 50 Endowment Funds
What do Harvard, the Church Commissioners for England, the NYC Metropolitan Museum of Art, and an entity on behalf of Saudi Arabia’s King Abdullah all have in common? They all have endowment funds.
An endowment fund is the investment arm of nonprofit institutions like universities, charities, and churches. The purpose of the fund is to invest the organization’s assets to fuel future operations and other important projects.
The world’s largest endowment funds have billions in investable assets, making them sizable players in the finance sector. Here, using data from Sovereign Wealth Fund Institute, we take a closer look at the world’s largest endowment funds by total assets.
Types of Endowment Funds
Overall, there are four main types of endowment structures.
- Unrestricted Endowment: A fund structure where assets are used at the full discretion of the institution
- Term Endowment: A fund structure with a fixed term time period before the principal can be spent
- Quasi Endowment: A donation to an endowment with a specific purpose to deploy that capital
- Restricted Endowment: A fund structure where the principal value from donations is held forever and only returns generated on the principal can be used
In addition, each endowment fund has different structures in regards to withdrawals, use of funds, and their general investment philosophy.
The Largest Endowment Funds
The largest endowment funds can be compared on a grand economic scale, in terms of assets.
To put it all into perspective, the largest 50 endowment funds represent over a trillion dollars in assets. Or for a more singular example, look at Harvard’s fund, which has an endowment greater than the entire GDP of countries like Serbia, Bolivia, or Slovenia.
Here’s how the top 50 rank.
| Rank | Endowment Fund | Total Assets | Region |
|---|---|---|---|
| 1 | Ensign Peak Advisors, Inc | $124,000,000,000 | North America |
| 2 | Japan Science and Technology Agency | $80,700,000,000 | Asia |
| 3 | Stanford University | $75,143,751,000 | North America |
| 4 | Harvard Management Company | $72,781,329,000 | North America |
| 5 | Yale University | $56,223,259,000 | North America |
| 6 | Princeton University | $44,460,038,000 | North America |
| 7 | MIT Investment Management Company | $42,526,492,000 | North America |
| 8 | Duke University | $30,385,835,000 | North America |
| 9 | New York University | $27,840,535,000 | North America |
| 10 | Columbia University in the City of New York | $24,698,782,000 | North America |
| 11 | University of Notre Dame | $24,599,541,000 | North America |
| 12 | KAUST Investment Management Company | $23,500,000,000 | Middle East |
| 13 | Emory University | $20,458,905,000 | North America |
| 14 | Johns Hopkins University | $18,037,751,000 | North America |
| 15 | Church Pension Fund | $17,773,649,171 | North America |
| 16 | University of Chicago | $17,276,136,000 | North America |
| 17 | Ohio State University | $16,006,851,000 | North America |
| 18 | Northwestern University | $15,855,683,000 | North America |
| 19 | Washington University in St Louis | $15,103,569,000 | North America |
| 20 | Penn State University, Office of Investment Management | $15,017,272,000 | North America |
| 21 | Notre Dame of Maryland University | $14,938,580,253 | North America |
| 22 | Cornell University | $14,850,618,000 | North America |
| 23 | University of Southern California | $14,495,427,000 | North America |
| 24 | Vanderbilt University | $13,883,495,000 | North America |
| 25 | University of Virginia Investment Management Compnay | $13,811,076,000 | North America |
| 26 | University of Tokyo | $13,285,270,000 | Asia |
| 27 | National University of Singapore | $12,626,100,000 | Asia |
| 28 | UNC Management Company | $11,986,857,000 | North America |
| 29 | University of Michigan Office of Investments | $11,900,000,000 | North America |
| 30 | General Authority of Awqaf | $11,238,371,192 | Middle East |
| 31 | Church Commissioners for England | $11,197,700,000 | Europe |
| 32 | J.Paul Getty Trust | $10,778,927,000 | North America |
| 33 | Trinity Wall Street Episcopal Church | $9,932,419,000 | North America |
| 34 | Unitersity of Utah | $9,827,602,000 | North America |
| 35 | Brown University | $9,793,108,000 | North America |
| 36 | Kamehameha Schools | $9,326,013,000 | North America |
| 37 | Dartmouth College | $9,078,340,000 | North America |
| 38 | Hong Kong Jockey Club | $8,603,580,000 | Asia |
| 39 | Rice University | $8,424,555,000 | North America |
| 40 | The Leona M. and Harry B. Helmsley Charitable Trust | $8,313,588,000 | North America |
| 41 | University of Pittsburgh | $8,011,856,000 | North America |
| 42 | Nature Conservancy | $7,870,380,000 | North America |
| 43 | University of Toronto Asset Management Corporation | $7,329,730,000 | North America |
| 44 | University of Rochester | $7,149,025,000 | North America |
| 45 | Virginia Commonwealth University | $6,985,495,306 | North America |
| 46 | Purdue University | $6,755,500,000 | North America |
| 47 | University of Miami | $6,582,600,000 | North America |
| 48 | University of Minnesota | $6,304,508,000 | North America |
| 49 | Caltech Investment Office | $6,252,584,000 | North America |
| 50 | Metropolitan Museum of Art of New York City | $5,588,554,000 | North America |
The largest endowment fund, Ensign Peak Advisors, is based in Salt Lake City, Utah, and manages the assets for the Mormon Church (officially known as the Church of Jesus Christ of Latter-day Saints). The church itself has over 16 million members worldwide and is the fourth largest church in America.
The Japan Science and Technology Agency (JST) is a national research and development agency that plays a core role in promoting technology, innovation, and science within society. In 1995, Japan’s government passed the Science and Technology Basic Plan and the JST came to life and now has over $80 billion in assets as well as offices in Paris, Washington, Singapore, and Beijing.
Just two funds come from the Middle East. The King Abdullah University of Science and Technology (KAUST) with $23.5 billion and the General Authority of Awqaf. KAUST is ranked 95th amongst universities in the world and made history in the country by being Saudi Arabia’s first mixed-gender university.
The General Authority of Awqaf has $11 billion in assets and was established as a public authority to manage endowments and enhance Saudi Arabia’s various goals for societal development. “Awqaf” in Arabic loosely translates to assets that are donated or purchased for general or specific charitable causes that are socially beneficial.
On the environmental side is the Nature Conservancy, which has $7.8 billion in assets. The charity is estimated to have protected more than 100 million acres of land.
American Universities Dominate
Universities are one leading category from the world of endowment funds, particularly those from the United States. In fact, universities make up 39 of the top 50 endowment funds, with 35 of them based in America.
Historically, Harvard has been the largest, but Stanford has edged ahead in recent years. Stanford has $75 billion in assets compared to Harvard’s $73 billion. These vast amounts of money have not gone unnoticed, and elite universities are facing mounting criticism in some circles.
“When Harvard’s total admitted freshman class is 1,400 people—and they have an endowment that is the GDP of El Salvador—they’re not a nonprofit, they’re a hedge fund educating the children of their investors.” – Professor Scott Galloway
With student debt rising to $1.6 trillion, it’s likely these universities may face greater criticism around how they use the wealth available to them in endowment funds.
Sizable Influence
The top endowment funds carry considerable influence within the world of finance. While they all have billions to invest, each has very different objectives and intentions on how to deploy their capital.
And despite being non-profit organizations, endowment funds are seeing their overall assets exceed those held by many other investment funds, such as sovereign wealth funds, hedge funds, and private equity firms.
-
VC+3 weeks agoComing Soon: Here’s What’s Coming to VC+ Next
-
Healthcare4 weeks agoRanked: The Best U.S. States for Retirement
-
Demographics2 weeks agoVisualizing the American Workforce as 100 People
-
Commodities4 weeks agoCharted: Commodities vs Equity Valuations (1970–2023)
-
Batteries2 weeks agoHow EV Adoption Will Impact Oil Consumption (2015-2025P)
-
Money4 weeks agoVisualizing the Assets and Liabilities of U.S. Banks
-
Education2 weeks agoRanked: The World’s Top 50 Endowment Funds
-
Markets4 weeks agoVisualized: Real Interest Rates by Country