Connect with us

Technology

Order From Chaos: How Big Data Will Change the World

Published

on

[imagemap id=”2964″]

Order From Chaos: How Big Data Will Change the World

Harnessing the exponential surge in data creates big opportunities.

Thanks to Purefunds Big Data ETF (BDAT) for helping us put this together.

IBM estimates that each day, 2.5 quintillion bytes of data are created or replicated. That’s the equivalent of a million hard drives filling up with data every hour.

The current volume of data created is substantial: it is so much that 90% of the world’s data has been created in the last two years. However, the amount of information today pales in comparison to what our future holds, as the rate at which data is created is accelerating exponentially.

It’s for this reason that The Economist estimates that there will be roughly 7x more data in 2020 than there was in 2014.

Where Does Big Data Come From?

Big data comes from both internal and external sources. Internally, millions of old documents and records are scanned and archived by businesses. Most of the time, no detailed analytics have ever been run on this information. Externally, the public web offers millions of data sets published for public consumption by government, economic, census, and other sources.

There’s also a broad spectrum of data that exists that can be a part of both of these categories: social media posts, documents, emails, business applications, machine log data, media, and sensor data can all be collected, processed, and analyzed. To get a sense of the extent of this information, here’s what is created every hour just from social media and email: 72 hours of video uploaded to Youtube, 4 million search inquiries on Google, 200 million emails sent, 2.5 million shares on Facebook, and 300,000 tweets made.

Big Data = Big Opportunities for Business

With proper analysis, Big Data can lead to new understandings of consumer behaviour, better management decisions, new innovations, and improved risk management. However, there are big challenges in making use of so much information.

  • Too much data creates an information overload.
  • Organizing and storing all of this data can be problematic.
  • Companies don’t know how to use all of this data to create insight.

To organize and make sense of it all, data scientists use the three V’s to describe Big Data.

Volume is the scale at which data is created, and includes the massive amounts of information derived from phones, internet users, machine logs, and internet of things.

Velocity is the analysis of streaming data: for example, modern cars have 100 sensors that monitor different systems in real-time.

Variety is the different forms of data, and it reflects the fact that data comes in all shapes and forms. Finding a way to harmonize multiple types of data can be quite a challenge. Research finds that organizations spend up to 80% of their time modelling and preparing data, rather than actually gaining insight.

Let’s see how companies have been able to use Big Data to create opportunity.

Case Studies of Big Data

Macy’s adjusts pricing in near-real time for 73 million items based on demand and inventory.

American Express developed predictive models that analyze historical transactions and 115 variables to forecast the loyalty of customers. Using this data, they can see if customers may be potentially closing their accounts in the near future. Launching a pilot program in Australia, the company can now identify 24% of accounts in the country that will close in the next four months.

Walmart built a new search engine for their website that includes semantic data relying on text analysis, machine learning, and even synonym mining to create better search results. Online shoppers have been more likely to complete purchases as a result by 10% to 15%, increasing revenue by billions.

Los Angeles and Santa Cruz police departments have used an algorithm that is typically used to predict earthquakes, now using it to look at crime data. The software can predict where crimes are likely to occur down to 500 square feet. In areas the software is being used, there has been a 33% reduction in burglaries and a 21% reduction in violent crimes.

Big Market

Today’s data centers occupy the land to equivalent to almost 6,000 football fields. By 2020, the amount of digital information is expected to increase exponentially to more than 7x of what it is today.

In healthcare alone, Big Data is expected to eventually save $300 billion per year in healthcare analytics. Retailers may increase margins up to 60% through Big Data analytics.

“Information is the oil of the 21st century, and analytics is the combusion engine.” – Peter Sondergaard, Gartner Research.

Continue Reading
Comments

Bitcoin

Mapping the Major Bitcoin Forks

Bitcoin forks play a key role in Bitcoin’s evolution as a blockchain. While some have sparked controversy, most Bitcoin forks have been a sign of growth.

Published

on

Mapping the Major Bitcoin Forks

The emergence of Bitcoin took the world by storm through its simplicity and innovation. Yet, plenty of confusion remains around the term itself.

The Bitcoin blockchain—not to be confused with the bitcoin cryptocurrency—involves a vast global network of computers operating on the same distributed database to process massive volumes of data every second.

These transactions tell the network how to alter this distributed database in real-time, which makes it crucial for everyone to agree on how these changes should be applied. When the community can’t come to a mutual agreement on what changes, or when such rule changes should take effect, it results in a blockchain fork.

Today’s unique subway-style map by Bitcoin Magazine shows the dramatic and major forks that have occurred for Bitcoin. But what exactly is a Blockchain fork?

Types of Blockchain Forks

Forks are common practice in the software industry and happen for one of two reasons:

  • Split consensus within the community
    These forks are generally disregarded by the community because they are temporary, except in extreme cases. The longer of the two chains is used to continue building the blockchain.
  • Changes to the underlying rules of the blockchain
    A permanent fork which requires an upgrade to the current software in order to continue participating in the network.

There are four major types of forks that can occur:

1. Soft Forks

Soft forks are like gradual software upgrades—bug fixes, security checks, and new features—for those that upgrade right away.

These forks are “backwards compatible” with the older software; users who haven’t upgraded still have access to the network but may not be able to use all functionality in the current version.

2. Hard Forks

Hard forks are like a new OS release—upgrading is mandatory to continue using the software. Because of this, hard forks aren’t compatible with older versions of the network.

Hard forks are a permanent division of the blockchain. As long as enough people support both chains, however, they will both continue to exist.

The three types of hard forks are:

  • Planned
    Scheduled upgrades to the network, giving users a chance to prepare. These forks typically involve abandoning the old chain.
  • Contentious
    Caused by disagreements in the community, forming a new chain. This usually involves major changes to the code.
  • Spin-off Coins
    Changes to Bitcoin’s code that create new coins. Litecoin is an example of this—key changes included reducing mining time from 10 minutes to 2.5 minutes, and increasing the coin supply from 21 million to 84 million.

3. Codebase Forks

Codebase forks copy the Bitcoin code, allowing developers to make minor tweaks without having to develop the entire blockchain code from scratch. Codebase forks can create a new cryptocurrency or cause unintentional blockchain forks.

4. Blockchain Forks

Blockchain forks involve branching or splitting a blockchain’s whole transaction history. Outcomes range from “orphan” blocks to new cryptocurrencies.

Splitting off the Bitcoin network to form a new currency is much like a religious schism—while most of the characteristics and history are preserved, a fork causes the new network to develop a distinct identity.

Summarizing Major Bitcoin Forks

Descriptions of major forks that have occurred in the Bitcoin blockchain:

  • Bitcoin / Bitcoin Core
    The first iteration of Bitcoin was launched by Satoshi Nakamoto in 2009. Future generations of Bitcoin (aka Bitcoin 0.1.0) were renamed Bitcoin Core, or Bitcore, as other blockchains and codebases formed.
  • BTC1
    A codebase fork of Bitcoin. Developers released a hard fork protocol called Segwit2x, with the intention of having all Bitcoin users eventually migrate to the Segwit2x protocol. However, it failed to gain traction and is now considered defunct.
  • Bitcoin ABC
    Also a codebase fork of Bitcoin, Bitcoin ABC was intentionally designed to be incompatible with all Bitcoin iterations at some point. ABC branched off to form Bitcoin Cash in 2017.
  • Bitcoin Gold, Bitcoin Diamond, Other Fork Coins
    After the successful yet contentious launch of Bitcoin Cash, other fork coins began to emerge. Unlike the disagreement surrounding Bitcoin Cash, most were simply regarded as a way to create new coins.

Some of the above forks were largely driven by ideology (BTC1), some because of mixed consensus on which direction to take a hard fork (Bitcoin ABC), while others were mainly profit-driven (Bitcoin Clashic)—or a mix of all three.

Where’s the Next Fork in the Road?

Forks are considered an inevitability in the blockchain community. Many believe that forks help ensure that everyone involved—developers, miners, and investors—all have a say when disagreements occur.

Bitcoin has seen its fair share of ups and downs. Crypto investors should be aware that Bitcoin, as both a protocol and a currency, is complex and always evolving. Even among experts, there is disagreement on what constitutes a soft or hard fork, and how certain geopolitical events have played a role in Bitcoin’s evolution.

Subscribe to Visual Capitalist

Thank you!
Given email address is already subscribed, thank you!
Please provide a valid email address.
Please complete the CAPTCHA.
Oops. Something went wrong. Please try again later.

Continue Reading

Technology

Why Big Data Keeps Getting Bigger

Visualizing the vast amount of data produced every single minute, and why it’s still early days in the big data era of technology.

Published

on

Why Big Data Keeps Getting Bigger

The sun never sets on the creation of new data.

Yes, the rate of generation may slow down at night as people send fewer emails and watch fewer videos. But for every person hitting the hay, there is another person on the opposite side of the world that is turning their smartphone on for the day.

As a result, the scale of data being generated—even when we look at it through a limited lens of one minute at a time—is quite mind-boggling to behold.

The Data Explosion, by Source

Today’s infographic comes to us from Domo, and it shows the amount of new data generated each minute through several different platforms and technologies.

Let’s start by looking at what happens every minute from a broad perspective:

  • Americans use 4,416,720 GB of internet data
  • There are 188,000,000 emails sent
  • There are 18,100,000 texts sent
  • There are 390,030 apps downloaded

Now lets look at platform-specific data on a per minute basis:

  • Giphy serves up 4,800,000 gifs
  • Netflix users stream 694,444 hours of video
  • Instagram users post 277,777 stories
  • Youtube users watch 4,500,000 videos
  • Twitter users send 511,200 tweets
  • Skype users make 231,840 calls
  • Airbnb books 1,389 reservations
  • Uber users take 9,772 rides
  • Tinder users swipe 1,400,000 times
  • Google conducts 4,497,420 searches
  • Twitch users view 1,000,000 videos

Imagine being given the task to build a server infrastructure capable of handling any of the above items. It’s a level of scale that’s hard to comprehend.

Also, imagine how difficult it is to make sense of this swath of data. How does one even process insights from the many billions of Youtube videos watched per day?

Why Big Data is Going to Get Even Bigger

The above statistics are already mind-bending, but consider that the global total of internet users is still growing at roughly a 9% clip. This means the current rate of data creation is still just scratching the surface of its ultimate potential.

In fact, as We Are Social’s recent report on internet usage reveals, a staggering 367 million new internet users were added in between January 2018 and January 2019:

Internet user growth

Global internet penetration sits at 57% in 2019, meaning that billions of more people are going to be using the above same services—including many others that don’t even exist yet.

Combine this with more time spent on the internet per user and technologies like 5G, and we are only at the beginning of the big data era.

Subscribe to Visual Capitalist

Thank you!
Given email address is already subscribed, thank you!
Please provide a valid email address.
Please complete the CAPTCHA.
Oops. Something went wrong. Please try again later.

Continue Reading
Foran Mining Company Spotlight

Subscribe

Join the 100,000+ subscribers who receive our daily email

Thank you!
Given email address is already subscribed, thank you!
Please provide a valid email address.
Please complete the CAPTCHA.
Oops. Something went wrong. Please try again later.

Popular