{"id":7038,"date":"2020-08-03T10:10:58","date_gmt":"2020-08-03T08:10:58","guid":{"rendered":"https:\/\/www.codemotion.com\/magazine\/?p=7038"},"modified":"2022-01-05T20:05:27","modified_gmt":"2022-01-05T19:05:27","slug":"edge-machine-learning","status":"publish","type":"post","link":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/machine-learning\/edge-machine-learning\/","title":{"rendered":"Getting started with edge machine learning"},"content":{"rendered":"\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t\tTable Of Contents\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t
  1. Introduction<\/a>
  2. A typical ML workflow<\/a>
  3. The hardware requirements for edge ML\u00a0<\/a>
  4. TensorFlow Lite: enabling ML at the edge<\/a>
  5. How to deploy a TensorFlow Lite model<\/a>
  6. Conclusions<\/a><\/ol>\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\n\n\n

    In this six-article series from Mouser Electronics<\/a>, we explore why AI<\/span> is moving to the network edge and the technology<\/span> that\u2019s making it possible. The first article<\/a> examined why AI<\/span> needs to move to the edge. Here, we dive in to the hardware and tools that are making edge machine learning a reality. <\/p>\n\n\n\n

    RELATED CONTENT<\/h4><\/div>

    The cutting edge of real-time AI<\/a><\/p><\/div><\/div>

    <\/div><\/div><\/div><\/div>\n\n\n\n

    Introduction<\/h2>\n\n\n\n

    Artificial intelligence<\/span> is primarily driven by machine learning<\/span> models. Typically, these models<\/span> are extremely processor-intensive, so they tend to be run in data<\/span> centres. However, as we saw already<\/a>, this causes issues for some AI applications<\/span>. Running your ML model<\/span> in the core of the network creates four issues:<\/p>\n\n\n\n

    1. Latency<\/span><\/strong>. More and more AI applications need to work in real-time. These range from self-driving vehicles to real-time facial recognition. Real-time implies a maximum latency of the order of 1ms. Clearly, that\u2019s only possible if your ML models are running at the network edge.<\/li>
    2. Not-spots<\/strong>. Many industrial AI applications have to run in environments with limited or no network access. Other times, local security policies may block access to outside networks. In both these cases, your only option is edge ML.<\/li>
    3. Security<\/span> and privacy<\/span><\/strong>. AI applications often handle sensitive data. This ranges from biometric data (e.g. face pictures) to sensitive medical data. Laws like GDPR<\/span> mean it is often easier (and more secure) to handle this sort of data locally, rather than send it across the network. <\/li>
    4. Costs<\/strong>. All major cloud providers offer specialist virtual machines designed to run complex ML models at scale. However, these instances are generally quite expensive. Nowadays, it is often preferable to try and run your models at the edge, thus saving you money.<\/li><\/ol>\n\n\n\n

      So, edge machine learning<\/span> is definitely desirable. But how do you actually go about doing it? First, we need to understand what it takes to create a machine learning<\/span> model<\/span>.<\/p>\n\n\n\n

      A typical ML workflow<\/h2>\n\n\n\n

      There are three main forms of machine learning<\/span>. These are supervised learning<\/a>, unsupervised learning<\/a> and reinforcement learning<\/a>. The most common is probably supervised learning<\/span>, so we will focus on this. In all cases, you are trying to teach a computer<\/span> to identify interesting patterns<\/span> in data<\/span>. In supervised learning<\/span>, you provide a set of labelled data<\/a>. The computer<\/span> learns to identify the data<\/span> features<\/span> that match with each label<\/span>. You can then get the computer to correctly label new data it sees<\/a>. As a concrete example, you might provide your ML model<\/span> with a million photos containing cats. The model<\/span> learns<\/span> to identify cats and can now use inference<\/span> to tell you whether a new photo also contains a cat. <\/p>\n\n\n\n

      There are seven steps involved in training your model<\/span>:<\/p>\n\n\n\n

      1. Data ingestion<\/strong>. This consists of gathering all your raw labelled data in one location (typically a data lake in the cloud).<\/li>
      2. Feature engineering<\/strong>. Here, you clean up your data in order to simplify the number of features. This includes removing features that only exist in some data, merging other features, and deleting irrelevant features.<\/li>
      3. Model selection<\/strong>. Nowadays, there are many thousands of different ML models available. To get some idea of the number, take a look at the Model Zoo<\/a>. Each model is optimised for a particular use. Data scientists use their knowledge and experience to select a suitable model for each dataset.<\/li>
      4. Training<\/strong>. This is the key step where your model is trained. Training is an incremental process that can take some time. You typically use 70-75% of your data for training. The rest is then used for evaluation and optimisation. <\/li>
      5. Evaluate<\/strong>. At this stage, you test whether your model performs well. There are a number of different metrics to assess the model. These include using a confusion matrix<\/a> and the F1 score<\/a>. What you must avoid is over-fitting of your model. This happens when your model becomes too good at identifying the training data and loses the ability to recognise interesting features in other data.<\/li>
      6. Hyperparameter tuning<\/strong>. This involves refining the model by tweaking the hyperparameters to improve accuracy. Common approaches include gradient-descent and Bayesian optimisation.<\/li>
      7. Deploy<\/span><\/strong>. The final step is to deploy your trained model. This is where things get interesting for edge ML. Traditionally, your model is deployed in a data centre or on a server. However, in edge ML, you are typically deploying it on a microcontroller.<\/li><\/ol>\n\n\n\n

        What forms of model are suitable for edge ML?<\/h3>\n\n\n\n

        There are several categories of ML models that are ideal for running at the edge. These include:<\/p>\n\n\n\n