{"id":12359,"date":"2020-11-30T09:51:55","date_gmt":"2020-11-30T08:51:55","guid":{"rendered":"https:\/\/www.codemotion.com\/magazine\/?p=12359"},"modified":"2022-01-05T20:06:45","modified_gmt":"2022-01-05T19:06:45","slug":"future-machine-learning-edge","status":"publish","type":"post","link":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/machine-learning\/future-machine-learning-edge\/","title":{"rendered":"The Future of Machine Learning at the Edge"},"content":{"rendered":"\n\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t\tTable Of Contents\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t
  1. The limitations of current edge devices<\/a>
  2. How this impacts machine learning at the edge<\/a>
  3. Advances that will boost Machine Learning at the edge<\/a>
  4. What the future will hold for machine learning at the edge<\/a>
  5. Conclusions<\/a><\/ol>\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\n\n\n

    Over the last few months, I have shown you how Machine Learning<\/span> at the edge <\/strong>is improving our lives. I also shared some practical examples to give you a chance to try it for yourselves. In this article, I look at the current limitations of edge ML<\/em> and explore where the technology will go in the future.<\/p>\n\n\n\n

    The limitations of current edge devices<\/h2>\n\n\n\n

    In the first article<\/a> of the series, I showed why moving machine learning to the network edge makes sense. I showed how edge ML<\/em> solves many of the practical limitations for machine learning. <\/p>\n\n\n\n

    For instance<\/span>, it allows ML <\/strong>models to run without network connectivity, or where security issues limit network access<\/span>. It also allows Machine Learning to be embedded<\/span> into end-devices, such as headphones or car stereos. However, all these applications<\/span> rely on the use of embedded<\/span> MCUs<\/em> or other low-power processors. This, in turn, imposes a real limit on the capabilities<\/span> of edge ML.<\/p>\n\n\n\n

    Processing power of MCUs<\/h3>\n\n\n\n

    Machine Learning at the Edge<\/span><\/strong> requires the use of devices<\/span> that only draw small amounts of power. Moreover, the devices<\/span> mustn\u2019t overheat and can only be passively cooled. Just imagine wearing headphones that get uncomfortably hot, or need the use of a fan! <\/p>\n\n\n\n

    The problem is, there is a direct correlation between processing power<\/span>,  power drawn, and heat production.<\/p>\n\n\n\n

    Over the decades, chip manufacturers have tried various ways to improve this. These include miniaturisation of chip dies<\/em>, splitting processing<\/span> between different cores, and the use of co-processors<\/span>. But the current generation of MCUs is still lagging behind.<\/p>\n\n\n\n

    32-bit limitations<\/h3>\n\n\n\n

    One of the key ways to increase processing power<\/span> is moving to 64-bit<\/span><\/strong> (or higher) architectures<\/span>. These instantly bring increases in performance<\/span> because the chip can do more processing<\/span> in every clock cycle. <\/p>\n\n\n\n

    They also allow larger data<\/span> structures to be stored in the cache, which speeds up many processing tasks. However, 64-bit<\/span> processors are much more power-hungry than 16- or 32-bit ones. As a result, the majority of high-performance<\/span> MCUs are 32-bit. <\/p>\n\n\n\n

    So, what are the practical implications of this? Firstly, it means data<\/span> operations are slower, with data<\/span> needing to be read to and from memory<\/span> more often. Secondly, it limits the complexity of data<\/span> structures you can create. As we will see, this causes problems for deep learning and other advanced forms of ML<\/span>. Finally, it imposes an absolute limit on the addressable memory space. That, in turn, impacts its efficiency and power<\/strong>.<\/p>\n\n\n\n

    How this impacts machine learning at the edge<\/h2>\n\n\n\n

    Machine Learning<\/span> is a rapidly-evolving field. But creating ML models relies on high-power processors and specialised servers<\/span>. This means the most advanced forms of ML<\/span> are simply not possible at the network edge<\/strong>. Let\u2019s look at some of the things that are not possible with the current generation of MCU<\/span>s.<\/p>\n\n\n\n

    Unsupervised learning at the edge<\/h3>\n\n\n\n

    The majority of standard Machine Learning<\/span> models rely on supervised learning<\/span><\/em>. That is, you take a large set of labelled data<\/span> and use this to teach a model to identify useful patterns<\/span>. The problem is, often there is little or no existing labelled data<\/span>. <\/p>\n\n\n\n

    In such cases, data<\/span> scientists turn to unsupervised learning<\/span><\/a>. This seeks to identify interesting patterns<\/span> in the data<\/span>, rather than match patterns<\/span> with labels. <\/p>\n\n\n\n

    This approach can be used in two ways. Firstly, you may simply want to identify important patterns<\/span> in the data<\/span>. These can help with tasks like classification<\/span> or identifying anomalies. <\/p>\n\n\n\n

    Secondly, it allows you to identify important features<\/span> in the data<\/span>. You can then create a small labelled data set<\/span><\/strong> and combine this with supervised learning to generate a more complex model. <\/p>\n\n\n\n

    Unsupervised learning<\/span> is hard to do on current edge devices<\/span> because they lack the raw processing power<\/span>. As a result, Machine Learning<\/span> at the edge is limited to tasks where there is an existing supervised learning model. <\/p>\n\n\n\n

    Continuous learning<\/h3>\n\n\n\n

    A supervised ML model<\/strong> is based on a fixed period in time. Imagine a model that predicts future electricity requirements based on historic demand and the current weather conditions. The model is probably quite accurate<\/span> when it is first trained. But now fast-forward five years. New appliances are more efficient than old ones, leading to a reduction in power consumed. <\/p>\n\n\n\n

    Houses are better-insulated, meaning there is less need for heating in winter and cooling in summer. Also, people are ever more aware of energy conservation  and seek to reduce their electricity use. As a result, your demand model is no longer accurate<\/span>.<\/p>\n\n\n\n

    \"machine
    Between 2005 and 2018, UK electricity demand dropped significantly<\/figcaption><\/figure><\/div>\n\n\n\n

    There are two ways to cope with this problem. The first is to simply replace the model periodically with one trained using the latest data<\/span>. This works quite well, but only where the changes are gradual and where you can actually update the model. <\/p>\n\n\n\n

    The second is to use continuous learning<\/strong>. This is a relatively new field in Machine Learning<\/span> that allows models to continually adjust themselves in response to the latest data<\/span>. As a result, the model never becomes stale and out of date. <\/p>\n\n\n\n

    Deep learning at the edge<\/h3>\n\n\n\n

    Deep Learning<\/span><\/strong> is one of the most powerful Machine Learning<\/span> techniques. It uses deep neural networks<\/em> to create extremely complex ML models. Some deep learning systems exhibit self-learning. <\/p>\n\n\n\n

    DeepMind<\/em>, for instance<\/span>, was able to learn to play the Chinese game of Go<\/span>. Over a number of months, it iterated a huge number of models that played against each other. It used this to learn new moves while discarding less successful models. In the end, it was able to beat a human world champion<\/a> Go player. <\/p>\n\n\n\n

    Deep Learning<\/span> is only possible with extremely powerful hardware<\/span> that is capable of a large number of parallel operations. Typically, it requires dedicated clusters of high performance<\/span> servers<\/span>, such as Microsoft Azure NC-series<\/a>. It is certainly beyond the capabilities<\/span> of low-cost, low-power edge devices<\/span>.<\/p>\n\n\n\n

    Advances that will boost Machine Learning at the edge<\/h2>\n\n\n\n

    Fortunately, MCU architectures<\/span> are advancing as fast as CPUs<\/span>. That means they broadly follow Moore\u2019s Law<\/a>. This law is named after Gordon Moore<\/span><\/em>, one of the cofounders of Intel<\/span>. Moore<\/span> posited that the number of transistors on a CPU<\/span> would double every two years. This is roughly equivalent to a doubling in computational power every 18 months.<\/p>\n\n\n\n

    \"Moore's<\/figure><\/div>\n\n\n\n

    The image above shows how this law has held true over 50 years of CPU<\/span> developments. If MCUs advance at the same rate, we will see rapid increases in capabilities<\/span> over the coming years to the advantage of Machine Learning<\/span> on the Edge. <\/p>\n\n\n\n

    64-bit architectures<\/h3>\n\n\n\n

    64-bit<\/span> ARM<\/span> processors have existed for some time now, and are available with 10nm<\/span> fabrication, which helps reduce their power consumption. As an example, the latest iteration of the ARM Cortex A-35 draws 32% less power than its predecessor and is 25% more efficient. <\/p>\n\n\n\n

    However, this is still a premium processor, costing hundreds of dollars. This means it is not yet suitable for mainstream edge ML applications<\/strong>. But just think what will be possible once these high-performance devices drop dramatically in price.<\/p>\n\n\n\n

    Increased performance for ARM<\/h3>\n\n\n\n

    The majority of high performance edge devices rely on ARM architecture processors. This is because they combine high performance with lower power draw. The very latest ARM architecture chips, such as Apple\u2019s M1, offer impressive processing capabilities, and can even use passive cooling.<\/p>\n\n\n\n

    However, they are also hugely expensive and relatively power-hungry by edge-device standards. But they show that chip manufacturers are still making real advances. Over time, we will see similar advances in the lower-power ARM cores that are used in high-end MCUs. <\/p>\n\n\n\n

    Embedded neural networks<\/h3>\n\n\n\n

    The most advanced edge devices are already offering significant machine learning capabilities. Apple\u2019s new M1 embeds their Neural Engine<\/strong>, which claims to offer 11 trillion operations per second. The Neural Engine is a specialised bit of silicon specifically designed to run neural network models.<\/p>\n\n\n\n

    Currently, such capabilities are reserved for high-end processors but in future they will become mainstream. The approach could also be ported to MCUs and other low-end devices. At that point, we will start to see true deep learning at the edge<\/strong>. <\/p>\n\n\n\n

    Hardware acceleration<\/h3>\n\n\n\n

    DeepMind is one of the most complex deep learning systems to date. But smaller deep learning systems are routinely used nowadays for tasks like advanced image recognition. <\/p>\n\n\n\n

    However, even a relatively small deep neural network consists of large numbers of layers and many thousands of artificial neurons. Such models need significant parallel processing<\/em>. This means they are usually run on highly specialised servers that use GPU<\/span>s to boost their processing performance. This is known as hardware acceleration<\/span><\/strong>.<\/p>\n\n\n\n

    For edge devices, pairing low-power GPUs with MCUs could provide similar hardware acceleration capabilities. This would help edge devices harness deep learning and unsupervised learning. <\/p>\n\n\n\n

    Mobile phone chipsets already incorporate relatively powerful GPUs. After all, they often need to drive retina HD screens. Devices like Raspberry Pi<\/span><\/a> show how such mobile phone chipsets can be repurposed as edge processing platforms. Raspberry Pi even offers a specific compute module designed to be embedded into devices.<\/p>\n\n\n\n

    What the future will hold for machine learning at the edge<\/h2>\n\n\n\n

    So, what advances might we see once more powerful edge devices are available? Let\u2019s look back at two of the applications from earlier articles and see how these will develop in future.<\/p>\n\n\n\n

    Computer vision<\/h3>\n\n\n\n

    In the fourth article<\/a>, I introduced computer vision<\/span> at the edge<\/em>. The system was relatively simple\u2014it was able to recognise the presence or absence of a human face. I also mentioned that you can train your own model to be able to recognise other images. However, there are two fundamental limitations because of the lack of computing power in MCUs:<\/p>\n\n\n\n

    1. Your model will only be able to recognise obviously different objects. For instance, it can distinguish a human face from the background. Or a dog from a cat. But it will struggle to identify a human from another human. <\/li>
    2. The model must be trained on a separate system and then embedded into the MCU. This means the model cannot be easily modified or updated in the field.<\/li><\/ol>\n\n\n\n

      Security companies already sell edge devices that are capable of facial recognition. But these are powered by CPUs or high-performance mobile phone chipsets. That makes them expensive. <\/p>\n\n\n\n

      In the near future, even cheap MCUs will be capable of this. And in time, they will be powerful enough to offer continuous learning<\/a>, allowing them to constantly update their models.<\/p>\n\n\n\n

      Voice recognition<\/h3>\n\n\n\n

      I also showed how Machine Learning on the edge is powering a revolution in voice control and voice recognition<\/span><\/a>. However, current systems can only recognise a small number of terms. <\/p>\n\n\n\n

      In effect, they suffer from the same limitations as I described above. But what if the edge devices became more powerful? What might become possible in future?<\/p>\n\n\n\n