The term Industry 4.0 (aka the Fourth Industrial Revolution) has come into vogue in recent years to describe the shift of industrial technology to networked and smart technologies. Innovations like the Internet of Things (IoT), cloud computing, AI and intelligent analytics are now being used to enhance manufacturing processes, optimise distribution networks and above all, improve industrial safety standards.
One of the key innovations used in Industry 4.0 is computer vision. Computer vision is an interdisciplinary field that ultimately aims to reproduce the perceptual and interpretative functions of human vision. It has come into its own in recent years and is now used to analyse static images and video material for medical analysis, industrial quality assurance, autonomous production processes and industrial hazard detection to promote safe working practices. It is this latter innovation that enables significant improvements and better compliance with relevant safety regulations.
In this article, we’ll cover some of the applications of computer vision and AI/ML in industry, before taking a look at how such technologies can be integrated with a microservices architecture to power innovative safety guards in industrial contexts. In particular, we’ll outline the success story of Konica Minolta’s Smart Mirror and its use by a semiconductor manufacturer and another company in the food production sector for Health&Safety purposes and to avoid product contamination.
Computer Vision and AI/ML for safer workspaces
Work on computer vision first began in earnest in the late 1960s. It was a key component in the emerging field of artificial intelligence (AI), which aimed to reproduce human cognitive and perceptual functions through digital processing. Though research has continued in the field ever since it is only in more recent years that AI has really started to make an impact on society. The combination of faster computers, improved network infrastructures and, above all, more intelligent processing has led to genuinely useful systems for both consumer-level and industrial applications.
Computer vision began with basic image analysis. Detecting lines, colours and forms can provide statistical information about an image or video clip, but where computer vision really gets interesting is when intelligent object detection and classification comes in. This is where ‘seeing’ really takes place. The development of neural networks and machine learning (ML) techniques have allowed a level of computer vision of significant subtlety and utility.
There are now many outlets for computer vision in Industry 4.0, especially when combined with advanced robotics. For example, complex production processes can be automated that previously were only possible with human operators. Process analysis and decision-making can be directed by intelligent algorithms and large datasets. And quality can be better assured by intelligent visual fault discovery. These applications lead to faster and more reliable production, with efficiency gains all round.
Computer vision is also invaluable for the maintenance of safety standards. For the consumer, it allows contamination of food or materials to be detected early, preventing possible ingestion of harmful substances. But for workers in industrial environments, computer vision also has a role: in monitoring equipment for process risks or faults, as well as operators’ personal safety provisions, like adequate protective equipment and compliance with regulations. A key component of these systems is the Smart Mirror.
Smart Mirror: how it works
At a basic level, smart mirrors use deep learning, computer vision and digital imaging technologies to reproduce the functions of a standard mirror. Where the latter simply reflects light back to form an image of what is before it, smart mirrors do something similar but with the intervention of digital image processing and analysis. If a subject stands in front of a smart mirror, they can see themselves reproduced on a large video screen in real-time. If this was all a smart mirror could do, there wouldn’t be much point in it, but there is, of course, much more.
In industrial settings, smart mirrors can form an important component in Industry 4.0 technologies. For example, with computer vision and AI-based analysis, it is possible to conduct safety inspections or compliance checks in a range of settings.
Konica Minolta’s Smart Mirror is an excellent example. While it is fast and easy to use, beneath the surface lie some of the most innovative AI technologies today. They draw on deep learning models to form intelligent interpretations in real-time. The Smart Mirror helps employees avoid hazards in various industries, such as chemicals, manufacturing and construction.
To take one successful scenario, Smart Mirror can provide immediate assessments of personal protective equipment to ensure that staff are adequately prepared for the risks they might encounter. Not only does this keep employees safe and legally compliant, but also helps to avoid product damage or contamination. Konica Minolta’s Smart Mirror automatically guides users through the inspection process with on-screen instructions and requires no technical expertise to operate.
A smart mirror doesn’t just reproduce visual forms, it also has the capacity to detect, identify and categorise people and objects. The objects the mirror detects can then be analysed and combined with data from other sources. For example, it can use face recognition technology to pinpoint users. One rapidly expanding outlet is in clothes retail and fashion, where smart mirrors are being adopted to offer additional marketing, recommendations and even virtual cosmetics or hairstyle try-outs. In these contexts, they can be seen as an extension of the kinds of functionality we’re familiar with from smartphones and tablets.
Smart mirrors also have more serious uses in medicine. For example, smart mirrors are being used to assess wellness – in one case, cardio-metabolic risk levels being calculated from minute measurements of facial features.
The Architecture: Microservices with docker-compose
The Smart Mirror’s seamless functionality is supported by an innovative microservice architecture. As well as the data that forms the primary input, it uses multiple machine learning models to generate analytical inferences. This requires a number of technologies to function smoothly.
- Sensors – Smart Mirror makes use of an RGB sensor to retrieve camera streaming, a thermal sensor to run temperature checks of the human body, a proximity sensor for checking the distance of the subject in front of the mirror (and starting it) and an RFID reader to scan the employee badge and cross-check user IDs. The RGB and thermal streaming are analysed through image processing and computer vision algorithms, making it easier to account for the safety of each individual and helping to ensure proper authorisation policies are followed.
- Docker Compose is used to define the overall container architecture. It allows easy set-up and tear-down with a single config file. Docker images, containers and services are all managed from here. The docker service architecture allows model interfaces to be combined as required and scaled across multiple devices.
- Triton Inference Server is a standardised AI platform that enables developers to incorporate models from a number of GPU-based infrastructures. It is designed to integrate with container services for live model updates, providing a seamless interface to devices and easy deployment.
- Redis and Kafka are used for message brokering and stream processing. An open-source, no-SQL data store, Redis provides high-performance data access, while Kafka is used to persist messages, with its higher data-handling capacities.
- These coordinated platforms allow the business logic to be defined in relatively simple terms. The AI aspects of the system are then responsive to real-world logic and handle the underlying implementation through the innovatively-designed microservice architecture.
Real-life use of Smart Mirror
To understand the power of Konica Minolta’s Smart Mirror, it’s worth looking in more detail at some real-life cases in which this solution was applied with great results.
One successful use case of Smart Mirror took place when a semiconductor company needed a solution that was able to check employees’ compliance with PPE requirements at the entrance of the production facility, also required to check for skin or hair exposure to avoid contamination on the production line.
In another real-case scenario, Smart Mirror was used again successfully when a food manufacturing company also needed to ensure PPE compliance checks, and the hair exposure of personnel in order to minimize any risk of food contamination taking place.
On the other hand, the recent pandemic brought an acute awareness of the need for safety measures in public and institutional settings. Social distancing, personal hygiene and adequate personal protective equipment (PPE) all became essential, especially in healthcare settings.
With these concerns in mind, the University Hospital of Olomouc, in the Czech Republic, looked to AI and computer vision for solutions with the expertise of Konica Minolta. Here, they made use of a product that ensured mask detection and PPE compliance in hospital environments to bolster personnel safety.
Konica Minolta developed an intuitive and simple-to-use user interface for the system, to ensure that protocol violations were easily identifiable. This means that even those with no technical know-how can see immediately where the problems lie and take fast remedial action.