• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
Codemotion Magazine

Codemotion Magazine

We code the future. Together

  • Discover
    • Live
    • Tech Communities
    • Hackathons
    • Coding Challenges
    • For Kids
  • Watch
    • Talks
    • Playlists
    • Edu Paths
  • Magazine
    • Backend
    • Frontend
    • AI/ML
    • DevOps
    • Dev Life
    • Soft Skills
    • Infographics
  • Talent
    • Discover Talent
    • Jobs
  • Partners
  • For Companies
Home » AI/ML » Machine Learning » The Future of Machine Learning at the Edge
Machine Learning

The Future of Machine Learning at the Edge

Machine Learning at the Edge is already proving its worth despite some limitations. Here, we explore these and look at future developments that will transform the technology.

November 30, 2020 by Mark Patrick, Mouser Electronics

Table Of Contents
  1. The limitations of current edge devices
  2. How this impacts machine learning at the edge
  3. Advances that will boost Machine Learning at the edge
  4. What the future will hold for machine learning at the edge
  5. Conclusions

Over the last few months, I have shown you how Machine Learning at the edge is improving our lives. I also shared some practical examples to give you a chance to try it for yourselves. In this article, I look at the current limitations of edge ML and explore where the technology will go in the future.

The limitations of current edge devices

In the first article of the series, I showed why moving machine learning to the network edge makes sense. I showed how edge ML solves many of the practical limitations for machine learning. 

For instance, it allows ML models to run without network connectivity, or where security issues limit network access. It also allows Machine Learning to be embedded into end-devices, such as headphones or car stereos. However, all these applications rely on the use of embedded MCUs or other low-power processors. This, in turn, imposes a real limit on the capabilities of edge ML.

Processing power of MCUs

Machine Learning at the Edge requires the use of devices that only draw small amounts of power. Moreover, the devices mustn’t overheat and can only be passively cooled. Just imagine wearing headphones that get uncomfortably hot, or need the use of a fan! 

The problem is, there is a direct correlation between processing power,  power drawn, and heat production.

Over the decades, chip manufacturers have tried various ways to improve this. These include miniaturisation of chip dies, splitting processing between different cores, and the use of co-processors. But the current generation of MCUs is still lagging behind.

32-bit limitations

One of the key ways to increase processing power is moving to 64-bit (or higher) architectures. These instantly bring increases in performance because the chip can do more processing in every clock cycle. 

They also allow larger data structures to be stored in the cache, which speeds up many processing tasks. However, 64-bit processors are much more power-hungry than 16- or 32-bit ones. As a result, the majority of high-performance MCUs are 32-bit. 

So, what are the practical implications of this? Firstly, it means data operations are slower, with data needing to be read to and from memory more often. Secondly, it limits the complexity of data structures you can create. As we will see, this causes problems for deep learning and other advanced forms of ML. Finally, it imposes an absolute limit on the addressable memory space. That, in turn, impacts its efficiency and power.

How this impacts machine learning at the edge

Machine Learning is a rapidly-evolving field. But creating ML models relies on high-power processors and specialised servers. This means the most advanced forms of ML are simply not possible at the network edge. Let’s look at some of the things that are not possible with the current generation of MCUs.

Unsupervised learning at the edge

The majority of standard Machine Learning models rely on supervised learning. That is, you take a large set of labelled data and use this to teach a model to identify useful patterns. The problem is, often there is little or no existing labelled data. 

In such cases, data scientists turn to unsupervised learning. This seeks to identify interesting patterns in the data, rather than match patterns with labels. 

This approach can be used in two ways. Firstly, you may simply want to identify important patterns in the data. These can help with tasks like classification or identifying anomalies. 

Secondly, it allows you to identify important features in the data. You can then create a small labelled data set and combine this with supervised learning to generate a more complex model. 

Unsupervised learning is hard to do on current edge devices because they lack the raw processing power. As a result, Machine Learning at the edge is limited to tasks where there is an existing supervised learning model. 

Continuous learning

A supervised ML model is based on a fixed period in time. Imagine a model that predicts future electricity requirements based on historic demand and the current weather conditions. The model is probably quite accurate when it is first trained. But now fast-forward five years. New appliances are more efficient than old ones, leading to a reduction in power consumed. 

Houses are better-insulated, meaning there is less need for heating in winter and cooling in summer. Also, people are ever more aware of energy conservation  and seek to reduce their electricity use. As a result, your demand model is no longer accurate.

machine learning at the edge
Between 2005 and 2018, UK electricity demand dropped significantly

There are two ways to cope with this problem. The first is to simply replace the model periodically with one trained using the latest data. This works quite well, but only where the changes are gradual and where you can actually update the model. 

The second is to use continuous learning. This is a relatively new field in Machine Learning that allows models to continually adjust themselves in response to the latest data. As a result, the model never becomes stale and out of date. 

Deep learning at the edge

Deep Learning is one of the most powerful Machine Learning techniques. It uses deep neural networks to create extremely complex ML models. Some deep learning systems exhibit self-learning. 

DeepMind, for instance, was able to learn to play the Chinese game of Go. Over a number of months, it iterated a huge number of models that played against each other. It used this to learn new moves while discarding less successful models. In the end, it was able to beat a human world champion Go player. 

Deep Learning is only possible with extremely powerful hardware that is capable of a large number of parallel operations. Typically, it requires dedicated clusters of high performance servers, such as Microsoft Azure NC-series. It is certainly beyond the capabilities of low-cost, low-power edge devices.

Advances that will boost Machine Learning at the edge

Fortunately, MCU architectures are advancing as fast as CPUs. That means they broadly follow Moore’s Law. This law is named after Gordon Moore, one of the cofounders of Intel. Moore posited that the number of transistors on a CPU would double every two years. This is roughly equivalent to a doubling in computational power every 18 months.

Moore's Law

The image above shows how this law has held true over 50 years of CPU developments. If MCUs advance at the same rate, we will see rapid increases in capabilities over the coming years to the advantage of Machine Learning on the Edge. 

64-bit architectures

64-bit ARM processors have existed for some time now, and are available with 10nm fabrication, which helps reduce their power consumption. As an example, the latest iteration of the ARM Cortex A-35 draws 32% less power than its predecessor and is 25% more efficient. 

However, this is still a premium processor, costing hundreds of dollars. This means it is not yet suitable for mainstream edge ML applications. But just think what will be possible once these high-performance devices drop dramatically in price.

Increased performance for ARM

The majority of high performance edge devices rely on ARM architecture processors. This is because they combine high performance with lower power draw. The very latest ARM architecture chips, such as Apple’s M1, offer impressive processing capabilities, and can even use passive cooling.

However, they are also hugely expensive and relatively power-hungry by edge-device standards. But they show that chip manufacturers are still making real advances. Over time, we will see similar advances in the lower-power ARM cores that are used in high-end MCUs. 

Embedded neural networks

The most advanced edge devices are already offering significant machine learning capabilities. Apple’s new M1 embeds their Neural Engine, which claims to offer 11 trillion operations per second. The Neural Engine is a specialised bit of silicon specifically designed to run neural network models.

Currently, such capabilities are reserved for high-end processors but in future they will become mainstream. The approach could also be ported to MCUs and other low-end devices. At that point, we will start to see true deep learning at the edge. 

Hardware acceleration

DeepMind is one of the most complex deep learning systems to date. But smaller deep learning systems are routinely used nowadays for tasks like advanced image recognition. 

However, even a relatively small deep neural network consists of large numbers of layers and many thousands of artificial neurons. Such models need significant parallel processing. This means they are usually run on highly specialised servers that use GPUs to boost their processing performance. This is known as hardware acceleration.

For edge devices, pairing low-power GPUs with MCUs could provide similar hardware acceleration capabilities. This would help edge devices harness deep learning and unsupervised learning. 

Mobile phone chipsets already incorporate relatively powerful GPUs. After all, they often need to drive retina HD screens. Devices like Raspberry Pi show how such mobile phone chipsets can be repurposed as edge processing platforms. Raspberry Pi even offers a specific compute module designed to be embedded into devices.

What the future will hold for machine learning at the edge

So, what advances might we see once more powerful edge devices are available? Let’s look back at two of the applications from earlier articles and see how these will develop in future.

Computer vision

In the fourth article, I introduced computer vision at the edge. The system was relatively simple—it was able to recognise the presence or absence of a human face. I also mentioned that you can train your own model to be able to recognise other images. However, there are two fundamental limitations because of the lack of computing power in MCUs:

  1. Your model will only be able to recognise obviously different objects. For instance, it can distinguish a human face from the background. Or a dog from a cat. But it will struggle to identify a human from another human. 
  2. The model must be trained on a separate system and then embedded into the MCU. This means the model cannot be easily modified or updated in the field.

Security companies already sell edge devices that are capable of facial recognition. But these are powered by CPUs or high-performance mobile phone chipsets. That makes them expensive. 

In the near future, even cheap MCUs will be capable of this. And in time, they will be powerful enough to offer continuous learning, allowing them to constantly update their models.

Voice recognition

I also showed how Machine Learning on the edge is powering a revolution in voice control and voice recognition. However, current systems can only recognise a small number of terms. 

In effect, they suffer from the same limitations as I described above. But what if the edge devices became more powerful? What might become possible in future?

  • Voiceprint recognition. One definite advance will be the ability to recognise individual voices. Amazon’s Alexa already offers this feature, but Alexa devices rely on being connected to the cloud for this. In future, you will be able to train an edge device to recognise your voice, offering additional security for voice control systems.
  • NLP at the edge. A key problem with voice recognition is actually understanding what you mean. The example we looked at before was only able to give a response to specific commands. But increasingly powerful MCUs will allow the natural language processing to happen within the edge device itself.
  • Reinforcement learning. Voice recognition systems often mishear or misunderstand what we say. Reinforcement learning is a form of continuous learning where you can provide feedback to the model. This works much like teaching a child to ride a bike by trial and error. When the child gets it wrong, they fall off. Over time, they learn what made them fall off and eventually, they learn how to ride the bike. In the same way, you can teach a voice recognition system to learn when it makes a mistake, so it will improve over time. More powerful edge devices will soon offer this sort of capability directly.

Conclusions

Machine Learning on the edge  is still in its early stages. Over the next few years it will become increasingly mainstream. Technologies that are currently only found in high power ARM CPUs will soon make their way into lower-cost MCUs. This will allow edge ML to offer capabilities that are equivalent to the best cloud-based systems.

facebooktwitterlinkedinreddit
Share on:facebooktwitterlinkedinreddit

Tagged as:Edge Computing Machine Learning

5 Rules of Engagement When it Comes to Data Visualization
Previous Post
10 Tips for Your Next eCommerce Web Design Project
Next Post

Related articles

  • Ludwig Toolbox Makes Deep Learning Accessible to All
  • AI Ladder: the IBM Approach to Artificial Intelligence
  • The Challenge of Ethics in AI
  • How AI Can Help Solve the Challenges of Economic and Financial Inequality
  • Voice Control: Building Your Voice Assistant
  • Seeing Is Believing: Image Recognition on a €10 MCU
  • ML at the Edge: a Practical Example
  • Getting started with edge machine learning
  • The cutting edge of real-time AI
  • Understanding AI: Training

Primary Sidebar

Learn new skills for 2023 with our Edu Paths!

Codemotion Edu Paths for 2023

Codemotion Talent · Remote Jobs

Java Developer & Technical Leader

S2E | Solutions2Enterprises
Full remote · Java · Spring · Docker · Kubernetes · Hibernate · SQL

AWS Cloud Architect

Kirey Group
Full remote · Amazon-Web-Services · Ansible · Hibernate · Kubernetes · Linux

Front-end Developer

Wolters Kluwer Italia
Full remote · Angular-2+ · AngularJS · TypeScript

Flutter Developer

3Bee
Full remote · Android · Flutter · Dart

Latest Articles

web accessibility standards, guidelines, WCAG

Implementing Web Accessibility in the Right Way

Web Developer

devops, devsecops, cibersecurity, testing

3 Data Breaches in Web Applications and Lessons Learned

Cybersecurity

The influence of Artificial Intelligence in HR

Devs Meet Ethics: the Influence of Artificial Intelligence In HR

AI/ML

google earth engine

What is Google Earth Engine and Why It’s Key For Sustainability Data Analysis

Data Science

Footer

  • Magazine
  • Events
  • Community
  • Learning
  • Kids
  • How to use our platform
  • Contact us
  • Become a Contributor
  • About Codemotion Magazine
  • How to run a meetup
  • Tools for virtual conferences

Follow us

  • Facebook
  • Twitter
  • LinkedIn
  • Instagram
  • YouTube
  • RSS

© Copyright Codemotion srl Via Marsala, 29/H, 00185 Roma P.IVA 12392791005 | Privacy policy | Terms and conditions

Follow us

  • Facebook
  • Twitter
  • LinkedIn
  • Instagram
  • RSS