{"id":1016,"date":"2019-11-05T08:00:52","date_gmt":"2019-11-05T07:00:52","guid":{"rendered":"http:\/\/cmagazine.test\/understanding-ai-inference\/"},"modified":"2021-12-23T13:02:21","modified_gmt":"2021-12-23T12:02:21","slug":"understanding-ai-inference","status":"publish","type":"post","link":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/","title":{"rendered":"Understanding AI: Inference"},"content":{"rendered":"

Inferencing<\/strong> is the second phase of machine learning<\/span><\/strong>, following on from the initial training<\/strong> phase. During the training phase, the algorithm generates a new model or repurposes a pre-trained model for a specific application and helps the model learn its parameters. During the inferencing phase, predictions and decisions on new data<\/span> are made \u2013 based on the learned parameters.<\/p>\n

Learning requires a significant amount of time, computation power and electricity. In contrast, the inferencing phase requires less processing and draws less power too. However, the traditional way of computing in the central cloud may be just too resource-intensive for IoT devices. Each IoT node residing at the edge collects large datasets, making edge-to-cloud (and conversely cloud-to-edge) data transfer expensive and slow. Instead of relying on the cloud-based servers to do all the processing, \u201ccomputing<\/span> at the edge<\/strong>\u201d performs most calculations directly and only transfers relevant information back to the cloud (and vice versa) when completely necessary. While computing at the edge reduces data transfer costs and time, this model also has certain drawbacks. For example, the need for IoT devices to be power-efficient runs contrary to the hefty amount of processing power that learning and inferencing demand. This is a problem that accelerators for AI edge computing<\/span> can potentially address.<\/p>\n

AI Accelerators<\/h2>\n

Both hardware- and software-based AI accelerators expedite machine learning<\/span>. Hardware acceleration can target training, inferencing or possibly both. In some instances, the hardware can reduce the power requirement. In other cases, the hardware can improve the processing capacity. Several main types of chips, or processing units, exist for hardware acceleration. These include central processing units (CPUs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs)<\/a>, system-on-chips (SoCs), application-specific integrated circuits (ASICs), vision processing units (VPCs) and neuromorphic ICs. In addition to hardware acceleration, solutions on the market also comprise software-based approaches \u2013 like machine learning<\/span> frameworks for improving AI software development and optimizing system performance.<\/p>\n

CPUs and GPUs<\/h2>\n

CPUs are what AI traditionally uses. While CPUs are designed to be all-purpose, they are often inadequate in supporting the massive calculations used in model generation and inferencing. In response, companies<\/span> including ARM (with its DynamIQ product offering) and Samsung (with its Exynos 9 series), have started making AI-specific chips. While ARM and Samsung have chosen to stick with AI-specific CPUs, others are shifting toward GPUs.<\/p>\n

Originating in the video gaming industry and built for processing massive datasets, GPUs are a good match for machine learning<\/span>. Because GPUs have more processing units per chip and higher throughput, plus more parallel processing capability than CPUs, they cut down computation time significantly. In addition, a GPU\u2019s single processing unit weighs less than the multiple units CPUs use, making GPUs a better fit for constrained IoT devices, which require small and nimble components. The companies<\/span> that are making AI-specific GPUs include AMD (Radeon Instinct), NEC (SX-Aurora), NVIDIA (DGX) and Qualcomm (Adreno).<\/p>\n

FPGAs<\/h2>\n

While CPUs and GPUs have considerable processing power at their disposal and are effective for accelerating learning and inferencing, they spend a lot of time and energy moving data<\/span> between memory and processing. Since CPUs and GPUs are densely packed with circuits, they can often overheat and cause system failures. For remotely located IoT devices, the combination of high energy consumption and potential system failures is far from ideal. It makes sense to find a way to offload some tasks to more energy-efficient hardware.<\/p>\n

Based on programmable logic, FPGAs are a type of IC that can be reconfigured by customers or designers in the field after production. While generally not as powerful as CPUs or GPUs, FPGAs offer fast processing for some calculations (such as multiplication, addition, integration, differentiation and exponentials) by computing inside the chip instead of transferring data<\/span>. Although an FPGA offers more flexibility, it tends to be quite bulky, so miniaturization for IoT devices is a challenge for this type of chip. The major companies<\/span> that offer AI-targeted inference chips include NVIDIA (TensorRT) and Xilinx. Also, Microsoft is using FPGA chips to accelerate inference, and Intel is currently expanding its FPGA portfolio<\/a>.<\/p>\n

SoCs<\/h2>\n

SoCs contains can contain a combination of electronic components (microprocessors, microcontrollers, digital signal processors, on-chip memory, hardware accelerators, etc.). Due to the integration of the components onto a single semiconductor substrate, a SoC is more powerful than a microcontroller chip. In a smartphone<\/span>, the SoC might integrate video, audio and image processing capabilities. ARM has developed its Machine Learning<\/span> Processor and its Object Detection Processor \u2013 and these will be incorporated into SoCs in the future. HiSilicon, a Huawei-backed company, has licensed the IP from ARM to to make SoCs that are seeing preliminary utilization in phone handsets and tablets. Also, HiSilicon is making the Ascend chips for Huawei. Another big player in the SoC space is Arteris, which is developing a network-on-chip interconnect fabric technology (FlexNoC) that many mobile and wireless companies<\/span> are using. Because Arteris holds a dominant position in IP, it has a bird\u2019s-eye view of the space. Other companies<\/span> likely to soon be making a play in the AI SoC market include Intel (via its Movidius subsidiary), NXP, Renesas, Toshiba, Texas Instruments and STMicroelectronics.<\/p>\n

ASICs, VPUs and Neuromorphic Chips<\/h2>\n

ASICs are specifically built for accelerating the training of deep learning algorithms, with examples including Google<\/span>\u2019s Edge TPU and Intel\u2019s Nervana. A vision processing unit (VPU) is designed to accelerate machine vision tasks and run machine vision algorithms, such as convolutional neural networks (CNNs) \u2013 so VPU video processing capabilities differ from those of a GPU, which does not offer the same type of task-specific processing. Examples of VPUs include Intel\u2019s Movidius Myriad chips, Google<\/span>\u2019s Pixel Visual Core, Microsoft\u2019s HoloLens, Inuitive\u2019s NU series and Mobileye\u2019s EyeQ.<\/p>\n

Digital chips and analog chips have their respective deficiencies: digital circuitry is precise but gobbles energy, while analog circuitry keeps both latency and energy consumption low but lacks precision. Therefore, researchers are looking for ways to combine the technical advantages of digital and analog chips while sidestepping the weaknesses. Inspired by the human brain, neuromorphic chips are designed to adhere to what is essentially a digital architecture, but use analog circuitry for mixed-signal processing<\/span>. IBM<\/span>\u2019s TrueNorth is a neuromorphic processor targeting sensor data pattern recognition and intelligence tasks. Also, Columbia University, Stanford University\u2019s \u2018Brains in Silicon\u2019 project, and the DARPA-backed University of Michigan IC Lab are all working on various aspects of neuromorphic system implementation.<\/p>\n

Machine Learning Frameworks<\/h2>\n

AI accelerators also include software<\/span>. For example, machine learning<\/span> frameworks, which can be interfaces, libraries or tools, help reduce the complexity<\/span> associated with machine learning<\/span> so that developers can build models and optimize performance<\/span> more quickly and easily. Such frameworks are built to specific languages, like Python or Java<\/span>. Some of the most popular open-source machine learning frameworks come from Amazon (AWS), Apache, Caffe2, Keras, Theano, Microsoft (Azure<\/span>) and Google<\/span> (TensorFlow). Also, some companies<\/span> offer in-house platforms. For example, Intel\u2019s OpenVINO toolkit is a software and hardware accelerator that optimizes inference with CNN models. In addition, Qualcomm\u2019s Snapdragon is a mobile platform and a software accelerator, IBM<\/span> has its Watson machine learning accelerator platform, and Huawei has recently launched its MindSpore AI framework.<\/p>\n

First Steps with AI<\/h2>\n

Mouser now offers various items of hardware that can form the initial building blocks for AI implementation. Intel\u2019s plug-and-play Neural Compute Stick 2<\/a> can aid engineers with early prototyping of deep neural networks. It relies on the company<\/span>\u2019s Movidius X VPU to deliver a compelling mix of power efficiency and performance<\/span> \u2013 attaining 4TOPS. Targeted at industrial computing<\/span>, the highly compact AAEON UP AI Core processing module<\/a> is based on the mini-PCI Express format. It also features an Intel Movidius VPU (this time the Myriad 2 2450 \u2013 with 512MBytes of DDR memory, plus 12 VUW programmable SHAVE cores and dedicated vision accelerators all built in). The Gumstix Aerocore 2 board<\/a> employs an array of NVIDIA Jetson TX1 and TX2 CUDA cores to give it strong parallel processing capabilities, along with an ARM Cortex-M4 microcontroller and numerous peripherals. It is particularly well suited to object recognition, production line inspection and various other kinds of machine vision.<\/p>\n

Looking to the Future<\/h2>\n

With NVIDIA remaining dominant in industrial AI applications, most newcomers are focusing on the IoT AI space. GreenWave and Reduced Energy Microsystems are in the low-power chip arena, while Mythic and Syntiant are developing battery-powered processors. Similarly, Wiliot is making a Bluetooth chip that can be powered by ambient radio frequencies<\/span>. In the massive parallel data processing space, there are Vathys, Graphcore, Cerebras and Wave Computing. Meanwhile, Hailo Technologies and Horizon Robotics are working on specialized chips for autonomous vehicles. In the deep learning space, BrainChip has made the first spiking neural processor, Thinci has rolled out a streaming graph processor, and Gyrfalcon is developing a deep learning processor with proprietary AI processing in memory (APiM) technology. Lastly, at Groq, the ex-Googlers who designed Google<\/span>\u2019s TPU are developing a chip with ultra-low latency. As the field of machine learning<\/span> witnesses astonishing progress, many technical challenges remain for IoT edge computing \u2013 with hardware and software developers<\/span> continuing to reach for a superior processing performance\/energy efficiency balance.<\/p>\n

If you enjoyed this article and would like to read more on AI, other technology topics, new products, and more SUBSCRIBE HERE.<\/a><\/p>\n

Mouser Electronics<\/strong> is a worldwide leading authorised distributor of semiconductors and electronic components for over 800 industry-leading manufacturers. They specialise in the rapid introduction of new products and technologies for design engineers and buyers. Mouser Electronics extensive product offering includes semiconductors, interconnects, passives, and electromechanical components.<\/p>\n

About the author<\/h3>\n

Mark Patrick<\/strong> joined Mouser Electronics in July 2014 having previously held senior marketing roles at RS Components. Prior to RS, Mark spent 8 years at Texas Instruments in Applications Support and Technical Sales roles and holds a first class Honours Degree in Electronic Engineering from Coventry University.<\/p>\n","protected":false},"excerpt":{"rendered":"

Inferencing is the second phase of machine learning, following on from the initial training phase. During the training phase, the algorithm generates a new model or repurposes a pre-trained model for a specific application and helps the model learn its parameters. During the inferencing phase, predictions and decisions on new data are made \u2013 based… Read more<\/a><\/p>\n","protected":false},"author":83,"featured_media":2216,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_editorskit_title_hidden":false,"_editorskit_reading_time":6,"_editorskit_is_block_options_detached":false,"_editorskit_block_options_position":"{}","_uag_custom_page_level_css":"","_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[46],"tags":[],"collections":[],"class_list":{"0":"post-1016","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-ai-ml","8":"entry"},"yoast_head":"\nUnderstanding AI: Inference - Codemotion Magazine<\/title>\n<meta name=\"description\" content=\"An overview about which IoT devices can support the process of inference in any AI\/Machine Learning application, and why some devices are better than others\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Understanding AI: Inference\" \/>\n<meta property=\"og:description\" content=\"An overview about which IoT devices can support the process of inference in any AI\/Machine Learning application, and why some devices are better than others\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/\" \/>\n<meta property=\"og:site_name\" content=\"Codemotion Magazine\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/Codemotion.Italy\/\" \/>\n<meta property=\"article:published_time\" content=\"2019-11-05T07:00:52+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-12-23T12:02:21+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1146\" \/>\n\t<meta property=\"og:image:height\" content=\"675\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Mark Patrick, Mouser Electronics\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@CodemotionIT\" \/>\n<meta name=\"twitter:site\" content=\"@CodemotionIT\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Mark Patrick, Mouser Electronics\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/\"},\"author\":{\"name\":\"Mark Patrick, Mouser Electronics\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/664e4da6990fc1344a2299435a542654\"},\"headline\":\"Understanding AI: Inference\",\"datePublished\":\"2019-11-05T07:00:52+00:00\",\"dateModified\":\"2021-12-23T12:02:21+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/\"},\"wordCount\":1639,\"publisher\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3.png\",\"articleSection\":[\"AI\/ML\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/\",\"name\":\"Understanding AI: Inference - Codemotion Magazine\",\"isPartOf\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3.png\",\"datePublished\":\"2019-11-05T07:00:52+00:00\",\"dateModified\":\"2021-12-23T12:02:21+00:00\",\"description\":\"An overview about which IoT devices can support the process of inference in any AI\/Machine Learning application, and why some devices are better than others\",\"breadcrumb\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/#primaryimage\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3.png\",\"contentUrl\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3.png\",\"width\":1146,\"height\":675,\"caption\":\"AI Inference\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.codemotion.com\/magazine\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"AI\/ML\",\"item\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Machine Learning\",\"item\":\"https:\/\/www.codemotion.com\/magazine\/ai-ml\/machine-learning\/\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Understanding AI: Inference\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#website\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/\",\"name\":\"Codemotion Magazine\",\"description\":\"We code the future. Together\",\"publisher\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.codemotion.com\/magazine\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#organization\",\"name\":\"Codemotion\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/codemotionlogo.png\",\"contentUrl\":\"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/codemotionlogo.png\",\"width\":225,\"height\":225,\"caption\":\"Codemotion\"},\"image\":{\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/Codemotion.Italy\/\",\"https:\/\/x.com\/CodemotionIT\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/664e4da6990fc1344a2299435a542654\",\"name\":\"Mark Patrick, Mouser Electronics\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/0d35fad9fee01e991637b67f54ae7cb8b001b5d2c1e4f7c1942b2105dad5a9bf?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/0d35fad9fee01e991637b67f54ae7cb8b001b5d2c1e4f7c1942b2105dad5a9bf?s=96&d=mm&r=g\",\"caption\":\"Mark Patrick, Mouser Electronics\"},\"url\":\"https:\/\/www.codemotion.com\/magazine\/author\/mark-patrick\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Understanding AI: Inference - Codemotion Magazine","description":"An overview about which IoT devices can support the process of inference in any AI\/Machine Learning application, and why some devices are better than others","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/","og_locale":"en_US","og_type":"article","og_title":"Understanding AI: Inference","og_description":"An overview about which IoT devices can support the process of inference in any AI\/Machine Learning application, and why some devices are better than others","og_url":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/","og_site_name":"Codemotion Magazine","article_publisher":"https:\/\/www.facebook.com\/Codemotion.Italy\/","article_published_time":"2019-11-05T07:00:52+00:00","article_modified_time":"2021-12-23T12:02:21+00:00","og_image":[{"width":1146,"height":675,"url":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3.png","type":"image\/png"}],"author":"Mark Patrick, Mouser Electronics","twitter_card":"summary_large_image","twitter_creator":"@CodemotionIT","twitter_site":"@CodemotionIT","twitter_misc":{"Written by":"Mark Patrick, Mouser Electronics","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/#article","isPartOf":{"@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/"},"author":{"name":"Mark Patrick, Mouser Electronics","@id":"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/664e4da6990fc1344a2299435a542654"},"headline":"Understanding AI: Inference","datePublished":"2019-11-05T07:00:52+00:00","dateModified":"2021-12-23T12:02:21+00:00","mainEntityOfPage":{"@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/"},"wordCount":1639,"publisher":{"@id":"https:\/\/www.codemotion.com\/magazine\/#organization"},"image":{"@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/#primaryimage"},"thumbnailUrl":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3.png","articleSection":["AI\/ML"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/","url":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/","name":"Understanding AI: Inference - Codemotion Magazine","isPartOf":{"@id":"https:\/\/www.codemotion.com\/magazine\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/#primaryimage"},"image":{"@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/#primaryimage"},"thumbnailUrl":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3.png","datePublished":"2019-11-05T07:00:52+00:00","dateModified":"2021-12-23T12:02:21+00:00","description":"An overview about which IoT devices can support the process of inference in any AI\/Machine Learning application, and why some devices are better than others","breadcrumb":{"@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/#primaryimage","url":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3.png","contentUrl":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3.png","width":1146,"height":675,"caption":"AI Inference"},{"@type":"BreadcrumbList","@id":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/understanding-ai-inference\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.codemotion.com\/magazine\/"},{"@type":"ListItem","position":2,"name":"AI\/ML","item":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/"},{"@type":"ListItem","position":3,"name":"Machine Learning","item":"https:\/\/www.codemotion.com\/magazine\/ai-ml\/machine-learning\/"},{"@type":"ListItem","position":4,"name":"Understanding AI: Inference"}]},{"@type":"WebSite","@id":"https:\/\/www.codemotion.com\/magazine\/#website","url":"https:\/\/www.codemotion.com\/magazine\/","name":"Codemotion Magazine","description":"We code the future. Together","publisher":{"@id":"https:\/\/www.codemotion.com\/magazine\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.codemotion.com\/magazine\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.codemotion.com\/magazine\/#organization","name":"Codemotion","url":"https:\/\/www.codemotion.com\/magazine\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.codemotion.com\/magazine\/#\/schema\/logo\/image\/","url":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/codemotionlogo.png","contentUrl":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/codemotionlogo.png","width":225,"height":225,"caption":"Codemotion"},"image":{"@id":"https:\/\/www.codemotion.com\/magazine\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/Codemotion.Italy\/","https:\/\/x.com\/CodemotionIT"]},{"@type":"Person","@id":"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/664e4da6990fc1344a2299435a542654","name":"Mark Patrick, Mouser Electronics","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.codemotion.com\/magazine\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/0d35fad9fee01e991637b67f54ae7cb8b001b5d2c1e4f7c1942b2105dad5a9bf?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/0d35fad9fee01e991637b67f54ae7cb8b001b5d2c1e4f7c1942b2105dad5a9bf?s=96&d=mm&r=g","caption":"Mark Patrick, Mouser Electronics"},"url":"https:\/\/www.codemotion.com\/magazine\/author\/mark-patrick\/"}]}},"featured_image_src":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3-600x400.png","featured_image_src_square":"https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3-600x600.png","author_info":{"display_name":"Mark Patrick, Mouser Electronics","author_link":"https:\/\/www.codemotion.com\/magazine\/author\/mark-patrick\/"},"uagb_featured_image_src":{"full":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3.png",1146,675,false],"thumbnail":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3-150x150.png",150,150,true],"medium":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3-300x177.png",300,177,true],"medium_large":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3-768x452.png",768,452,true],"large":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3-1024x603.png",1024,603,true],"1536x1536":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3.png",1146,675,false],"2048x2048":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3.png",1146,675,false],"small-home-featured":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3.png",100,59,false],"sidebar-featured":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3-180x128.png",180,128,true],"genesis-singular-images":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3-896x504.png",896,504,true],"archive-featured":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3-400x225.png",400,225,true],"gb-block-post-grid-landscape":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3-600x400.png",600,400,true],"gb-block-post-grid-square":["https:\/\/www.codemotion.com\/magazine\/wp-content\/uploads\/2019\/11\/UnderstandingAI-InferenceFigA3_3-600x600.png",600,600,true]},"uagb_author_info":{"display_name":"Mark Patrick, Mouser Electronics","author_link":"https:\/\/www.codemotion.com\/magazine\/author\/mark-patrick\/"},"uagb_comment_info":0,"uagb_excerpt":"Inferencing is the second phase of machine learning, following on from the initial training phase. During the training phase, the algorithm generates a new model or repurposes a pre-trained model for a specific application and helps the model learn its parameters. During the inferencing phase, predictions and decisions on new data are made \u2013 based……","lang":"en","_links":{"self":[{"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/posts\/1016","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/users\/83"}],"replies":[{"embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/comments?post=1016"}],"version-history":[{"count":6,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/posts\/1016\/revisions"}],"predecessor-version":[{"id":14334,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/posts\/1016\/revisions\/14334"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/media\/2216"}],"wp:attachment":[{"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/media?parent=1016"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/categories?post=1016"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/tags?post=1016"},{"taxonomy":"collections","embeddable":true,"href":"https:\/\/www.codemotion.com\/magazine\/wp-json\/wp\/v2\/collections?post=1016"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}