Edge Computing And The Future Of Machine Learning

Consumer uses of AI will increasingly rely on the data processed near the source

23Jan

The last sixty years have seen commercial computing oscillate between centralization and decentralization, determined largely by the needs of the technology of the day. In the 1960s, everything was focused on developing computing power in a single device. In the 1980s, it was all about multiple dumb terminals connecting to a mainframe. Since the late 2000s, the trend has been firmly towards centralization, with computing increasingly pushed out to the cloud. However, the pendulum has already started to swing back. The rise of Internet of Things (IoT) technology is seeing computing move back to the ‘edge’ of local networks and intelligent devices, in a shift that has profound implications for machine learning.

In a traditional cloud computing architecture, the actual processing of data occurs far away from the source. Edge computing pushes the generation, collection, and analysis of data out to the point of origin, rather than to a data center or cloud. In the case of the IoT, this means it takes place at the devices and sensors. Writing on Telcomtv, Ian Scales compared edge computing to the human nervous system, arguing that: ‘There’s an important component in human physiology called the Autonomic System which more or less does for us humans what edge computing is designed to do for the cloud. The relevant part of the Autonomic ‘system’ is the ‘sympathetic nervous system’. This is the thing a doctor is tapping into when he or she hits you on the knee with that little hammer - it’s designed to trigger your ‘quick response mobilizing system’. This allows you to jump into action without actually engaging the brain to think about it first.’

Edge computing is advantageous to machine learning for a number of reasons. Firstly, because edge computing relies on proximity to the source of the data, it minimizes incurred latency. If we are to look at AI as a tool for completing human tasks, such as driving, it is the instinctive reactions mentioned by Scales that are the most important to replicate. For example, self-driving cars generate as much as 25 gigabytes of data an hour. This has to be uploaded, analyzed, and instructions sent back. In a cloud infrastructure, the excessive latency could well mean that vehicles end up failing to react to any of the many sudden events that unfold on the road on a daily basis. This could potentially lead to a death. In edge computing, the data doesn't have to make any round trip to the cloud, significantly reducing latency and leading to real-time, automated decision-making.

There are, of course, limitations to what you can do at the edge. Today’s machine learning algorithms are designed to run on powerful servers. Therefore, in the case of driverless cars, much of the heavy lifting still takes place in the cloud, with algorithms trained using millions of miles of recorded driving data before being deployed at the edge for inference. Increasingly, however, in other applications, we are starting to see algorithms trained locally too. This is far more cost-effective, requiring less ongoing bandwidth and storage cost. Swim, for example, is a streaming data analytics startup that uses a distributed network architecture to operate self-training machine learning at the edge in real-time. According to Simon Crosby, Swim's CTO, it is actually the only way for many AI applications to make commercial sense, which is particularly important for the many startups looking to operate in the space who lack the funds and cloud infrastructure of their larger rivals.

And it is not just the likes of Swim that are getting involved. Towards the end of 2017, Microsoft also announced that they were integrating their Azure Machine Learning with Azure IoT Edge. Ted Way, Senior Program Manager at Microsoft, wrote in a blog post of the integration at the time that:

'There many use cases for the intelligent edge, where a model is trained in the cloud and then deployed to an edge device. For example, a hospital wants to use AI to identify lung cancer on CT scans. Due to patient privacy and bandwidth limitations, a large CT scan containing hundreds of images can be analyzed on a server in the hospital, rather than sending it to the cloud [...] In the example of finding lung cancer on CT scans, Azure ML can be used to train models in the cloud using big data and CPU or GPU clusters. The model is operationalized to a Docker container with a REST API, and the container image can be stored in a registry such as Azure Container Registry.

In the hospital, an Azure IoT Edge gateway such as a Linux server can be registered with Azure IoT Hub in the cloud. Using IoT Hub, the pipeline can be configured as a JSON file. For example, a data ingest container knows how to talk to devices, and the output of that container goes to the ML model. The edge configuration JSON file is deployed to the edge device, and the edge device knows to pull the right container images from container registries.'

Apple too are bringing AI to their smartphones with the new iPhone X, with the phone’s new A11 Bionic chip. This addition has strangely flown under the radar, with the press around the technology instead focusing on the Face ID feature, which the A11 Bionic chip actually powers. AI is now commonplace in smartphones, particularly for the speech recognition that enables digital assistants. However, this has been mostly powered by the cloud. Apples decision to include a neural engine dedicated to handling specific machine learning algorithms in the phone suggests that this is where the industry is heading in future.

Claims that we are witnessing the death of cloud are premature, however, we are becoming reliant on the edge layer for AI that has a real impact on everyday life. Processing is increasingly going to occur wherever it is best placed for any given application because of the significant cost savings and reduced latency, and for AI to flourish, edge computing will be vital. 

Would you like more information on Machine Learning? Join the Machine Learning Innovation Summit in San Francisco in May
Amazonechos

Read next:

How Amazon Alexa Works

i