Why you need to merge big data, IoT and blockchain

These technologies can provide a platform for the expansion of each of their individual technologies, playing off the strengths of one another


The trifecta of information technology advancements in the early twenty-first century, big data, Internet of Things (IoT) and blockchain have been on everyone's lips since they broke into the mainstream. While we understand each one of these things separately, combining these into a single system would make for an extremely secure predictive practice that is able to learn from itself and to adapt to suit a number of situations. While this is simply speaking in broad terms, these three bits of technology can provide a platform for the expansion of each of their individual technologies, playing off the strengths of one another. But how exactly do we merge these three otherwise disparate technologies into one solid unit?

IoT and big data

As Science Direct notes, the increase in data created by the IoT is on such an exponential scale that the overlaps with big data are immediately apparent. The IoT is probably the largest single generator of data that a company may have. Since IoT devices will be generating fresh data regularly and those devices can number from the dozens to the hundreds of thousands, this data stream will be an uninterrupted flow of raw readings that can then be complied along certain lines in order to draw conclusions. Determining which metrics we want to follow and which metrics are safely ignored is where data scientists come in. Big data is only as good as the insights it can provide to the business. In this sense the IoT is the natural partner to big data. One collects the data and the other turns it into actionable information.

The blockchain

Trust is a very important tool, and up until very recently, someone's word was all one had to go on when determining the level of trust that person was allowed. The blockchain seeks to make it less of a guess and more of a mathematically based calculation. Blockchain, also called Distributed Ledger Technology (DLT), is best known for its popularity in cryptocurrency - the nascent attempt to move away from fiat currency into something more trustworthy. The idea of trust shouldn't depend on an individual or an organization, but rather on a loosely associated network of people. In the case of a blockchain, each individual person (or unit) knows everything about the whole blockchain, and in order to tamper with the blockchain, one would have to simultaneously edit at least half of the active nodes on the network simultaneously. The impossibility of this situation increases as more and more people adopt the blockchain.

In the simplest terms, according to Forbes, a blockchain can be compared to a distributed database, where all the people using the database know the entire contents of the database, and as new entries are added, they are updates across the chain. While the nitty-gritty of blockchain processing is not what we intend to dive into, the fact of the matter is that blockchain provides a unique method of keeping data secure as well and automatically verifying that data so it cannot be tampered with. What better use than to apply it to all that big data we're already collecting from our IoT devices?

The chinks in the armor

Big data and IoT have perennially faced issues with their innate security. Because IoT devices are so small and simple, it's not far-fetched to imagine a spoofing methodology that copies an IoT device's key to login to the Big Data collection database, gaining a backdoor entry into data which they can change to their heart’s content without anyone being aware of it. While security in IoT devices have been steadily improving, the idea of a centralized data store that Big Data enjoys using so much has the same flaws. A single server can be spoofed easily enough to force IoT devices to send their collected data to a third party for free. And the scariest part is that in this case, the company using the IoT devices legitimately may be completely unaware this is happening.

Visit DATAx New York on December 12–13, 2018

Patching the holes

Consultancy UK grants that the things that make blockchain technology attractive are the speed with which data is processed, the ability of the data to be tamper-proof and redundant, and that the blockchain is consistent across all nodes in operation. This by itself solves two of the problems that currently plague the combination of big data and IoT. By introducing a few distributed databases that store the data collected from the IoT's in multiple locations, in the event of one of the databases being compromised, the rest would be able to interact without any worry of the system being taken down. Additionally, the IoT devices can submit their data to the databases using blockchain technology to write to the database, creating blocks of data which can then be processed by specialized big data algorithms in a safe and secure manner.

A New Paradigm for Data

As time goes along we will understand more and more how these technologies interact with each other. Once the first few steps are taken to combine these advancements, we may actually see more positive moves to create specialized systems that have integration built-in for the cross-interaction of these systems. The evolution of a system usually needs a drastic change and while all of these systems are new, that doesn't mean they are set in stone. They are supposed to evolve to match the needs of their users, and by combining with each other that evolution can only be sped up.

The simons foundation open new center for computational mathematicssmall

Read next:

The Simons Foundation open new Center for Computational Mathematics