Order From Chaos: How Big Data Will Change the World
Harnessing the exponential surge in data creates big opportunities.
Thanks to Purefunds Big Data ETF (BDAT) for helping us put this together.
IBM estimates that each day, 2.5 quintillion bytes of data are created or replicated. That’s the equivalent of a million hard drives filling up with data every hour.
The current volume of data created is substantial: it is so much that 90% of the world’s data has been created in the last two years. However, the amount of information today pales in comparison to what our future holds, as the rate at which data is created is accelerating exponentially.
It’s for this reason that The Economist estimates that there will be roughly 7x more data in 2020 than there was in 2014.
Where Does Big Data Come From?
Big data comes from both internal and external sources. Internally, millions of old documents and records are scanned and archived by businesses. Most of the time, no detailed analytics have ever been run on this information. Externally, the public web offers millions of data sets published for public consumption by government, economic, census, and other sources.
There’s also a broad spectrum of data that exists that can be a part of both of these categories: social media posts, documents, emails, business applications, machine log data, media, and sensor data can all be collected, processed, and analyzed. To get a sense of the extent of this information, here’s what is created every hour just from social media and email: 72 hours of video uploaded to Youtube, 4 million search inquiries on Google, 200 million emails sent, 2.5 million shares on Facebook, and 300,000 tweets made.
Big Data = Big Opportunities for Business
With proper analysis, Big Data can lead to new understandings of consumer behaviour, better management decisions, new innovations, and improved risk management. However, there are big challenges in making use of so much information.
- Too much data creates an information overload.
- Organizing and storing all of this data can be problematic.
- Companies don’t know how to use all of this data to create insight.
To organize and make sense of it all, data scientists use the three V’s to describe Big Data.
Volume is the scale at which data is created, and includes the massive amounts of information derived from phones, internet users, machine logs, and internet of things.
Velocity is the analysis of streaming data: for example, modern cars have 100 sensors that monitor different systems in real-time.
Variety is the different forms of data, and it reflects the fact that data comes in all shapes and forms. Finding a way to harmonize multiple types of data can be quite a challenge. Research finds that organizations spend up to 80% of their time modelling and preparing data, rather than actually gaining insight.
Let’s see how companies have been able to use Big Data to create opportunity.
Case Studies of Big Data
Macy’s adjusts pricing in near-real time for 73 million items based on demand and inventory.
American Express developed predictive models that analyze historical transactions and 115 variables to forecast the loyalty of customers. Using this data, they can see if customers may be potentially closing their accounts in the near future. Launching a pilot program in Australia, the company can now identify 24% of accounts in the country that will close in the next four months.
Walmart built a new search engine for their website that includes semantic data relying on text analysis, machine learning, and even synonym mining to create better search results. Online shoppers have been more likely to complete purchases as a result by 10% to 15%, increasing revenue by billions.
Los Angeles and Santa Cruz police departments have used an algorithm that is typically used to predict earthquakes, now using it to look at crime data. The software can predict where crimes are likely to occur down to 500 square feet. In areas the software is being used, there has been a 33% reduction in burglaries and a 21% reduction in violent crimes.
Today’s data centers occupy the land to equivalent to almost 6,000 football fields. By 2020, the amount of digital information is expected to increase exponentially to more than 7x of what it is today.
In healthcare alone, Big Data is expected to eventually save $300 billion per year in healthcare analytics. Retailers may increase margins up to 60% through Big Data analytics.
“Information is the oil of the 21st century, and analytics is the combusion engine.” – Peter Sondergaard, Gartner Research.
Visualizing the World’s Top Social Media and Messaging Apps
From Twitter to TikTok, this infographic compares the universe of social media and messaging platforms by number of monthly active users.
The Social Media Universe in 2022
For a time, life in the social media universe was mostly uneventful. Consider these spicy (at the time) headlines:
- Even President Obama Thinks That Facebook Isn’t Cool Anymore (Techcrunch, 2014)
- Jack Dorsey Returns to Twitter as Chief, to Shrugs and Quips (New York Times, 2015)
- Instagram’s new stories are a near-perfect copy of Snapchat (Mashable, 2016)
In hindsight, the years leading up to 2016 were downright sleepy in comparison with what would follow. Donald Trump’s meteoric, tweet-powered rise to the presidency. The Cambridge Analytica scandal. Congressional hearings on privacy and bias. TikTok at the center of souring U.S.–China relations. Each new day brought a fresh wave of controversy the shores of once infallible social media platforms.
Today, the honeymoon phase is long over and the messiness of running a global social platform is now on full display. Nowhere is this more evident than Twitter during the current Elon Musk transitional period—but more details on that later.
For now, let’s explore the social media universe in 2022.
Mapping the Social Media and Messaging Universe
In 2022, the social universe is looking more crowded than in previous years.
The scale of Meta’s platforms still dominate thanks to their global reach, but there are a number of smaller networks fighting for market share. Here’s a look at popular platforms, organized from largest to smallest active userbase:
Meanwhile, here are the top 10 social media and messaging platforms by publicly-available monthly active users:
|Rank||Platform Name||Parent Company||Primary Function||Monthly Active Users|
|#1||Meta Platforms||Social network||2.9 billion|
|#2||YouTube||Alphabet||Video content||2.3 billion|
|#3||Meta Platforms||Messaging||2.0 billion|
|#4||Messenger||Meta Platforms||Messaging||1.3 billion|
|#5||Meta Platforms||Video content||1.2 billion|
|#7||TikTok||ByteDance||Video content||732 million|
|#9||Douyin||ByteDance||Video content||600 million|
YouTube is the only true competition for Meta’s scale and reach. Alphabet’s video content hub with social features boasts more than two billion monthly active users. YouTube’s embrace of the creator economy is nudging the platform further into pure social media territory with the introduction of “handles”.
As seen in the visualization above, China has its own ecosystem of large social and messaging platforms—the largest of these being WeChat.
The only platform in the top 20 that is not based in either the U.S. or China is the privacy-focused messaging app, Telegram. The Dubai-based company has a unique backstory. It was created after the founders of Russian social network VK left the country after resisting government pressure to release data on the social network’s users in Ukraine.
Today, there are also a number of smaller, special interest platforms. OnlyFans, for example, is focused on adult content creators. Parler and Truth Social appeal to users who want fewer constraints on the content they post and consume. BeReal aims to create more authentic moments by prompting users to post a photo at a random time each day.
Below, we dig into a few of these platforms into more depth.
Big Trouble in Little Metaverse
Having a figurehead CEO is a double-edged sword. When things are going well, the market rallies around the successful leader. Case in point, Mark Zuckerberg was named Time’s Person of the Year in 2010. Even as recently as 2016, Glassdoor named the Facebook founder the “most admired tech CEO”.
On the flip side, when the tide turns, it turns fast. After a series of controversies, Zuckerberg took a multi-billion-dollar gamble by renaming his entire company Meta and pivoting its focus to the burgeoning idea of a metaverse. Meta’s New Horizons platform is rumored to have plateaued at about 200,000 active users, which is underwhelming for a company that still reaches a sizable slice of humanity with its other services.
Part of Meta’s near-term success hinges on VR headsets being a hot gift this holiday season. Meta’s cheapest headset is $400, which could be a tough sell in today’s economic environment.
Of course, it’s too early to know whether Zuckerberg’s gamble will pay off. As always, all is forgiven once a business unit takes off and becomes profitable.
Microblogging with Macro Expectations
Twitter has a complicated history.
The company was launched in the shadow of Facebook’s massive growth, and was saddled with expectations that were tough to meet. Although Twitter has an engaged and influential audience, it hasn’t managed to monetize them at the level of Meta’s platforms (for better or worse). The introduction of Twitter Blue in 2021 did not resonate with users at the scale the company hoped, and “fleets” were essentially written off as a failed experiment.
In addition, Twitter is a magnet for criticism and debate around free speech, in part because of its central place in political discourse.
These issues are directly related to the company’s recent sale to Elon Musk. At the time of this article, Twitter finds itself in the midst of a painful, and very public, internal restructuring.
Social media has always been dominated by Facebook and its related apps. When a new challenger came along, Facebook either acquired it (Instagram, WhatsApp), or “acquired” their features (Snapchat). TikTok is the first challenger to keep its momentum and growth, even as Instagram rolled out very similar features.
TikTok is also a rare case of a Chinese tech product crossing over into Western markets. The ascendancy of TikTok was not without controversy though. Suspicion over Chinese access to user data continues to be an issue both in the U.S., and in other large markets around the world. TikTok has been banned in India since 2020.
Despite these headwinds, TikTok remains wildly popular. The short-form video platform was the number one downloaded app on the planet, and it remains a favorite of the all-important Gen Z demographic.
We Shall Surveil
In recent years, neighborhood-based social networks have sprung up and gained traction. NextDoor used physical letters sent to adjacent addresses to supercharge its growth, while Neighbors piggybacked off the popularity of Ring’s doorbell cameras. Although members post about more benign topics such as lost cats and where to find a good plumber, crime is an increasingly common theme as well.
Apps like Neighbors and Citizen have a more overt focus on crime and safety. While the growth of these apps reflects an obvious interest preventing crime, critics point out that the ubiquity of personal surveillance equipment and forums built purely around public safety promote a culture of suspicion in communities.
This type of social network is still quite new, so it remains to be seen if they remain niche communities, or grow into something bigger.
Chaos and Opportunity
It was Sun Tzu who famously said, “In the midst of chaos, there is also opportunity”.
This is the risk and opportunity in the social media universe today. With their massive networks and high switching costs (e.g. personalization, library of existing posts), the largest platforms have created moats that make life hard for upstart brands looking to replace established platforms. On the other hand, controversy on platforms like Twitter and Facebook may cause some users to consider new options.
The multi-billion-dollar question—is dissatisfaction with major platforms temporary, or will emerging networks like Mastodon or BeReal hit critical mass and become new staples for people connecting online. Time will tell.
Visualized: FTX’s Leaked Balance Sheet
As Sam Bankman-Fried’s crypto exchange FTX files for bankruptcy, this graphic visualizes FTX’s balance sheet leaked by the Financial Times.
Visualizing FTX’s Balance Sheet Before Bankruptcy
In a difficult year for the crypto space that has been full of hacks, failing funds, and decentralized stablecoins going to zero, nothing has compared to FTX and Sam Bankman-Fried’s (SBF) rapid implosion.
After an astronomical rise in the crypto space over the past three years, crypto exchange FTX and its founder and CEO SBF have come crashing back down to earth, largely unraveled by their misuse of customer funds and illicit relationship with trading firm Alameda Research.
This graphic visualizes FTX’s leaked balance sheet dated to November 10th, and published by the Financial Times on November 12th. The spreadsheet shows nearly $9 billion in liabilities and not nearly enough illiquid cryptocurrency assets to cover the hole.
How did FTX wind up in this position?
How FTX’s Bankruptcy Unfolded
FTX’s eventual bankruptcy was sparked by a report on November 2nd by CoinDesk citing Alameda Research’s balance sheet. The article reported Alameda’s assets to be $14.6 billion, including $3.66 billion worth of unlocked FTT and $2.16 billion of FTT collateral.
With more than one-third of Alameda’s assets tied up in FTX’s exchange token FTT (including loans backed by the token), eyebrows were raised among the crypto community.
Four days later on November 6th, Alameda Research’s CEO, Caroline Ellison, and Sam Bankman-Fried addressed the CoinDesk story as unfounded rumors. However, on the same day, Binance CEO Changpeng Zhao (CZ) announced that Binance had decided to liquidate all remaining FTT on their books, kicking off a -7.6% decline in the FTT token on the day.
Back and Forth with Binance’s CZ
While Ellison publicly offered to buy CZ’s FTT directly “over the counter” to avoid further price declines and SBF claimed in a now-deleted tweet that “FTX is fine. Assets are fine.”, FTX users were withdrawing their funds from the exchange.
The next day, the acquisition fell apart as Binance cited corporate due diligence, leaving SBF to face a multi-directional liquidity crunch of users withdrawing funds and rapidly declining token prices that made up large amounts of FTX and Alameda’s assets and collateral for loans.
FTX’s Liabilities and Largely Illiquid Assets
In the final days before declaring bankruptcy, FTX CEO Sam Bankman-Fried attempted a final fundraising in order restore stability while billions in user funds were being withdrawn from his exchange.
The balance sheet he sent around to prospective investors was leaked by the Financial Times, and reveals the exchange had nearly $9 billion in liabilities while only having just over $1 billion in liquid assets. Alongside the liquid assets were $5.4 billion in assets labeled as “less liquid” and $3.2 billion labeled as “illiquid”.
When examining the assets listed, FTX’s accounting appears to be poorly done at best, and fraudulently deceptive at worst.
Of those “less liquid” assets, many of the largest sums were in assets like FTX’s own exchange token and cryptocurrencies of the Solana ecosystem, which were heavily supported by FTX and Sam Bankman-Fried. On top of this, for many of these coins the liquidity simply wouldn’t have been there if FTX had attempted to redeem these cryptocurrencies for U.S. dollars or stablecoin equivalents.
While the liquid and less liquid assets on the balance sheet amounted to $6.3 billion (still not enough to equal the $8.9 billion in liabilities), many of these “less liquid” assets may as well have been completely illiquid.
Relationship with Alameda Research
When looking at FTX’s financials in isolation, it’s impossible to understand how one of crypto’s largest exchanges ended up with such a lopsided and illiquid balance sheet. Many of the still unfolding details lie in the exchange’s relationship with SBF’s previous venture that he founded, trading firm Alameda Research.
Founded by SBF in 2017, Alameda Research primarily operated as a delta-neutral (a term that describes trading strategies like market making and arbitrage that attempt to avoid taking directional risk) trading firm. In the summer of 2021, SBF stepped down from Alameda Research to focus on FTX, however his influence and connection with the firm was still deeply ingrained.
A report from the Wall Street Journal cites how Alameda was able to amass crypto tokens ahead of their announced public FTX listings, which were often catalysts in price surges. Alongside this, a Reuters story has revealed how SBF secretly moved $10 billion in funds to Alameda, using a bookkeeping “back door” to avoid internal scrutiny at FTX.
While SBF responded to the Reuters story by saying they “had confusing internal labeling and misread it,” there are few doubts that this murky relationship between Alameda Research and FTX was a fatal one for the former billionaire’s empire.
Datastream4 days ago
The Rising Demand for Nature-based Climate Solutions
Markets2 weeks ago
Ranked: The World’s 100 Biggest Pension Funds
Money4 days ago
Mapped: Which Countries Have the Highest Inflation?
Money4 weeks ago
The U.S. and China Account for Half the World’s Household Wealth
Misc2 weeks ago
Every Song With Over 1 Billion Spotify Streams
Datastream2 days ago
Disruptive Materials: Visualizing America’s Import Dependency
Green4 weeks ago
Visualizing Mismanaged Plastic Waste by Country
Markets2 weeks ago
All of the World’s Money and Markets in One Visualization (2022)