Why Big Data Keeps Getting Bigger
The sun never sets on the creation of new data.
Yes, the rate of generation may slow down at night as people send fewer emails and watch fewer videos. But for every person hitting the hay, there is another person on the opposite side of the world that is turning their smartphone on for the day.
As a result, the scale of data being generated—even when we look at it through a limited lens of one minute at a time—is quite mind-boggling to behold.
The Data Explosion, by Source
Today’s infographic comes to us from Domo, and it shows the amount of new data generated each minute through several different platforms and technologies.
Let’s start by looking at what happens every minute from a broad perspective:
- Americans use 4,416,720 GB of internet data
- There are 188,000,000 emails sent
- There are 18,100,000 texts sent
- There are 390,030 apps downloaded
Now lets look at platform-specific data on a per minute basis:
- Giphy serves up 4,800,000 gifs
- Netflix users stream 694,444 hours of video
- Instagram users post 277,777 stories
- Youtube users watch 4,500,000 videos
- Twitter users send 511,200 tweets
- Skype users make 231,840 calls
- Airbnb books 1,389 reservations
- Uber users take 9,772 rides
- Tinder users swipe 1,400,000 times
- Google conducts 4,497,420 searches
- Twitch users view 1,000,000 videos
Imagine being given the task to build a server infrastructure capable of handling any of the above items. It’s a level of scale that’s hard to comprehend.
Also, imagine how difficult it is to make sense of this swath of data. How does one even process insights from the many billions of Youtube videos watched per day?
Why Big Data is Going to Get Even Bigger
The above statistics are already mind-bending, but consider that the global total of internet users is still growing at roughly a 9% clip. This means the current rate of data creation is still just scratching the surface of its ultimate potential.
In fact, as We Are Social’s recent report on internet usage reveals, a staggering 367 million new internet users were added in between January 2018 and January 2019:
Global internet penetration sits at 57% in 2019, meaning that billions of more people are going to be using the above same services—including many others that don’t even exist yet.
Combine this with more time spent on the internet per user and technologies like 5G, and we are only at the beginning of the big data era.
Visualizing Moore’s Law in Action (1971-2019)
Can the predictions from Moore’s Law keep up with technological innovation spanning almost 50 years? Watch this stunning animation to find out.
Animation: Visualizing Moore’s Law in Action (1971-2019)
The pace of technological progress keeps accelerating.
There are many ways to show this, but perhaps the simplest way is to create a visual representation of Moore’s Law in action.
Today’s animation comes to us from DataGrapha, and it compares the predictions of Moore’s Law with data from actual computer chip innovations occurring between 1971 to 2019.
Defining Moore’s Law
Moore’s Law was originally derived from an observation by Gordon Moore, the co-founder of Fairchild Semiconductor and later the co-founder and CEO of Intel.
In 1965, Moore wrote that the number of components in a dense integrated circuit (i.e., transistors, resistors, diodes, or capacitors) had been doubling with every year of research, and he predicted that this would continue for another decade.
Later on in 1975, he revised his prediction to the doubling occurring every two years.
Like the animation, the following chart from Our World in Data helps plot out the predictions of Moore’s Law versus real world data — note that the Y Axis is logarithmic:
The prophetic prediction of Moore’s Law has led to exponential progress in computing — as well as for everything else touched by computers.
It’s no surprise then, especially given that the modern information age is largely driven by increasingly efficient computing, that this law has had a trickle down effect on nearly every significant aspect of global innovation.
An Accelerated Pace of Change
Moore’s Law has translated into a faster rate of change for society as a whole.
A new idea, like the smartphone, can get immediate traction because of instantaneous communication, increased global connectivity, and the ubiquity of information. New tech advancements can now change business or culture in a heartbeat:
Further, since software is a “layer” built upon the foundation of computing, it means that digital products can be replicated at almost no marginal cost. This is why a phenomenon like Pokémon Go was able to captivate 50 million users in just 19 days.
Imagine this kind of scalability, when applied to things like artificial intelligence or virtual reality.
Is Moore’s Law Dead or Alive?
As with any enduring prediction, there are always naysayers out there that will boldly forecast an imminent end to the trend.
Since the 2000s, there has been an ongoing debate within the semiconductor community on whether Moore’s Law will continue its reign, or if progress will ultimately sputter out as certain physical limitations catch up with the process of miniaturization.
Earlier in 2019, Nvidia CEO Jensen Huang declared that Moore’s Law is no longer possible. For what it’s worth, Intel still says technology in chipmaking always finds a way to advance — while TSMC has recently said the law is actually alive and well.
Regardless of who is right, Moore’s Law has held true for close to 50 years, and its repercussions will continue to be felt in almost every aspect of life and society going forward.
Wired World: 35 Years of Submarine Cables in One Map
Watch the explosive growth of the global submarine cable network, and learn who’s funding the next generation of cables.
You could be reading this article from nearly anywhere in the world and there’s a good chance it loaded in mere seconds.
Long gone are the days when images would load pixel row by pixel row. Now, even high-quality video is instantly accessible from almost everywhere. How did the internet get so fast? Because it’s moving at the speed of light.
The Information Superhighway
The miracle of modern fiber optics can be traced to a single man, Narinder Singh Kapany. The young physicist was skeptical when his professors asserted that light ‘always travels in a straight line’. His explorations into the behavior of light eventually led to the creation of fiber optics—essentially, beaming light through a thin glass tube.
The next step to using fiber optics as a means of communication was lowering the cable’s attenuation rate. Throughout the 1960-70s, companies made gains in manufacturing, reducing the number of impurities and allowing light to cross great distances without a dramatic decrease in signal intensity.
By the mid-1980s, long distance fiber optic cables had finally reached the feasibility stage.
Crossing the Pond
The first intercontinental fiber optic cable was strung across the floor of the Atlantic Ocean in 1988. The cable—known as TAT-8*—was spearheaded by three companies; AT&T, France Télécom, and British Telecom. The cable was able to carry the equivalent of 40,000 telephone channels, a ten-fold increase over its galvanic predecessor, TAT-7.
Once the kinks of the new cable were worked out, the floodgates were open. During the course of the 1990s, many more cables hit the ocean floor. By the dawn of the new millennium, every populated continent on Earth was connected by fiber optic cables. The physical network of the internet was beginning to take shape.
As today’s video from ESRI shows, the early 2000s saw a boom in undersea cable development, reflecting the uptick in internet usage around globe. In 2001 alone, eight new cables connected North America and Europe.
From 2016-2020, over 100 new cables were laid with an estimated value of $14 billion. Now, even the most remote Polynesian islands have access to high-speed internet thanks to undersea cables.
*TAT-8 does not appear in the video above as it was retired in 2002.
The Shifting Nature of Cable Construction
Even though nearly every corner of the globe is now physically connected, the rate of cable construction is not slowing down.
This is due to the increasing capacity of new cables and our appetite for high-quality video content. New cables are so efficient that the majority of potential capacity along major cable routes will come from cables that are less than five years old.
Traditionally, a consortium of telecom companies or governments would fund cable construction, but tech companies are increasingly funding their own submarine cable networks.
Amazon, Microsoft and Google own close to 65% market share in cloud data storage, so it’s understandable that they’d want to control the physical means of transporting that data as well.
These three companies now own 63,605 miles of submarine cable. While laying cable is a costly endeavor, it’s necessary to meet surging demand—content providers’ share of data transmission skyrocketed from around 8% to nearly 40% over the past decade.
A Bright Future for Dark Fiber
At the same time, more aging cables will be taken offline. Even though signals are no longer traveling through this network of “dark fiber”, it’s still being put to productive use. It turns out that undersea telecom cables make a very effective seismic network, helping researchers study offshore earthquakes and the geologic structures on the ocean floor.
Markets11 months ago
The Jeff Bezos Empire in One Giant Chart
Maps1 year ago
Mercator Misconceptions: Clever Map Shows the True Size of Countries
Advertising10 months ago
Meet Generation Z: The Newest Member to the Workforce
Misc1 year ago
24 Cognitive Biases That Are Warping Your Perception of Reality
Advertising9 months ago
How the Tech Giants Make Their Billions
Technology11 months ago
The 20 Internet Giants That Rule the Web
Chart of the Week11 months ago
Chart: The World’s Largest 10 Economies in 2030
Environment10 months ago
The World’s 25 Largest Lakes, Side by Side