Having covered various aspects of why and how to bring Deep Learning (DL) inference into edge devices, we now look at applications of DL inference in edge System-on-Chip (SoC) devices. Though DL has a plethora of application possibilities, we focus on use cases of DL inference in edge devices. These use cases demand one or more attributes including:
- Very low latency
- Real-time response
- Data security
- Data privacy
- Low power consumption
- Adherence to the regulations of regional and country compliance
- Low cost with limited network bandwidth, unreliable or no network connectivity.
The business opportunities of these use cases are huge, as Gartner predicts that 50% of enterprise-generated data will be processed at the edge by 2022, and Mind Commerce predicts that the global AI chipsets market will approach $13.4B USD by 2023.(1)
The use cases of DL in edge devices can be broadly classified as handling various types of structured and unstructured data including images, video, audio, non-imaging data from various sensors. These use cases can be grouped into performing various DL functions including image classification, object detection, clustering, pattern detection and data classification or in any combination thereof. Also, it is necessary to derive correlation between several sensor inputs and external factors that produce huge volumes of data.
These use cases apply to various market segments including:
- Energy & utilities
- Oil & gas
- Industrial
- Manufacturing
- Medical
- Wearables
- Security & surveillance
- Autonomous vehicles
- Robotics
- Consumer products
- Network intrusion detection
In these industries, typically IoT devices with various sensors or drones are used for real-time monitoring of performance, wear and tear of various machinery and turbines; cracks, leaks or spills of far-flung extraction facilities, oil and gas tanks; emission levels, etc. As long as this real-time monitoring is limited to triggering preventive maintenance, deriving cloud-based intelligence for anomaly prediction from data collected may be acceptable, assuming there is sufficient network connectivity and bandwidth. However, if any of these predicted anomalies causing disastrous situations must be acted upon instantly, the system needs to trigger corresponding actuators. This implies bringing in deep learning inference to edge devices like IoT or drones, which is likely to be increasingly adopted as time goes on.
Industrial
In an industrial environment, productivity, safety, and regulatory compliance are extremely important. This means devices connected to the IoT need to be monitored in real-time and proactively maintained rather than requiring reactive repairs. In this context, DL inference in edge devices with associated sensors will enable the predictive maintenance of critical equipment through anomaly detection. This can send out critical alerts if there is any anomaly or deviant data is collected that is different from machine-learnt typical behavior, such as vibration, speed or heat.
Manufacturing
In a manufacturing setup, DL inference in edge devices such as robots monitor in real-time and detect anomalies in high-velocity assembly lines, which will improve the production yield. In addition, a manufacturing robot will need to derive intelligence out of its sensors as well as from nearby equipment to optimize performance, carry out coordinated tasks and avoid causing injury to nearby workers. Robots are also typically deployed in moving parts across the factory floor. The DL inference in such robots can enable the autonomous movement by finding their own best possible route. If DL inference is done in the cloud, it can cause severe performance issues and even worse safety issues.
Medical and wearables
DL can play a significant role in improving medical diagnosis and suggesting the best treatment option by using patient health information, X-ray, CT scan, blood reports, etc. Medical devices can be trained with huge global patient datasets that can diagnose diseases, predict treatment that worked best and point out further medical complications, with trained models. If not at the edge, such DL inference should run close to the edge, such as on the hospital private cloud.
The DL inference can also play a significant role in various procedures and surgeries using endoscopes. If the DL inference engine is fitted along with the camera in an endoscope, it can derive intelligence out of what it is seeing based on the trained models, and can assist the doctor on-the-fly.
Wearables collect large amounts of vital health information including heart rhythm, blood pressure, breathing patterns and blood glucose levels. The DL inference running in the wearables can predict the necessity of preventive health care and even alert to certain emergency indicators.
Security and Surveillance
If the DL inference engine is integrated into a surveillance camera, it can derive intelligence from what it is seeing and alert and record events in real-time. Another good thing about this edge DL is that every surveillance camera can have a different pre-trained model based on what it is supposed to detect. For example, if a drone identifies an accident, it can provide valuable information about the wreck to pedestrians nearby.
Even better, DL inference at the edge will be able to predict accidents and crimes before they occur by detecting preconditions based on the data collected from noise sensors, video cameras, and even smart trash bins in smart cities.
Autonomous vehicles
DL inference at the edge is absolutely necessary for self-driving cars running at high speeds, as action needs to be taken instantly. Typically, various types of sensors and a fusion of the intelligence derived from them is needed for self-driving cars including imaging, Radar and Lidar.
Network intrusion detection and QoS (Quality of Service) improvement
The DL inference can be used in Network Traffic classification (NTC), as port or payload-based approaches have challenges. Port-based approaches are less reliable and payload-based approaches have difficulty in maintaining changing patterns for Deep Packet Inspection (DPI) and concerns about maintaining adequate encryption. DL inference relies on flow-statistics, which relies on information from only the packet header and not the payload. The NTC is key for network intrusion detection by stopping attacks that do not have known signatures, as well as in improving QoS.
Though there are exciting opportunities of applications of DL inference in edge devices, its success depends on how well experts in respective market segments train the DL machines. The machine training dataset and model accuracy determines how well DL inference can do the job. Finally, care must be exercised as there can always be a “corner case” which was not known to the machine.
Next up, Part 3 will elaborate on architectural details of edge devices with Deep Learning.
(1) https://www.gartner.com/smarterwithgartner/what-edge-computing-means-for-infrastructure-and-operations-leaders/
