Accelerating Image Recognition with Edge Computing

An Edge Server to Accelerate Inference

This article focuses on the use case of a search and rescue robot in the context of the Industrial Internet of Things (IIoT). In this use case, a robot or unmanned aerial vehicle searches for subjects of interest (e.g., humans) in a designated area. The robot performs this task by periodically capturing high-resolution images of the landscape and running inference, i.e., applying a machine learning (ML) model to identify a list of predicted objects and their location, with a certain degree of confidence in the prediction.

During a search and rescue mission, missing subjects may be in a life-threatening situation. Therefore, the inference must be as fast and accurate as possible. Thanks to computing power improvements in IIoT, resource-constrained robots can run computationally light ML models in real-time, with low battery usage. However, these models lack the required accuracy to find subjects that may be only a few pixels wide.

Computationally heavy ML models are more accurate, but executing them in the robot results in a significant increase in inference time and battery usage. Similarly to recent works in the literature (Spatharakis et al., 2022), the robot may choose to offload such a process to a more powerful local edge computing server deployed by the search and rescue team. In this work, we use an edge server instead of a Cloud infrastructure to reduce the transmission overhead. The edge server has the necessary resources to run a heavier ML model in real-time. Before offloading, the robot encodes the image to minimize transmission time. Fig. 1 depicts the system’s flowchart.

Figure 1. Search and rescue system flowchart.

Innovation: Benchmarking for Real-time Image Processing

In our project, we follow current trends in edge robotics. Recent works (Chinchali et al., 2021, Cai et al., 2021) involving real-time image recognition, data processing and robot learning rely heavily on a task offloading scheme to increase the accuracy of specific applications, while reducing response time and preserving resources on IIoT devices.

Approximate computing is an innovative idea in selecting between different efficiency or quality configurations for a compute-intensive application that can be executed in different computing resources (i.e., GPU or CPU). However, deciding whether to offload such tasks is essential. Task offloading decision schemes are crucial in maximizing resource utilization while preserving quality of service (QoS) requirements. Moreover, using the edge computing paradigm minimizes communication overhead. We believe this work is a significant preliminary benchmark using state-of-the-art software to support such a framework.

Methodology

We conducted experiments to showcase the main aspects of the proposed scenario, namely: (i) YOLOv5 accuracy benchmark using different sets of image parameters and (ii) JPEG XL compression benchmark using different efforts and quality factors.

YOLOv5 Accuracy Benchmark

YOLOv5 (You Only Look Once version 5) is a machine learning algorithm that allows real-time object detection. In this experiment, we study the height and quality factor impact of an image on the accuracy of YOLOv5 predictions to identify the parameters that allow the fastest transmission time with minimal loss of accuracy.

We encoded images in the JPEG XL format with different quality factors. The quality factor is a real number of up to 100 that controls the visual quality of the encoded image. A quality factor of 100 produces a mathematically lossless image while a quality factor of 90 produces a visually lossless image. A lower quality factor reduces the bitrate of the encoded image at the cost of lower visual fidelity.

Fig. 2 shows the results. Images with a quality factor of 100 generally yield the most accurate results. However, as shown in the next experiment, these images have a much higher bitrate than the other quality factors, which increases their transmission time. In the offloading mechanism, a reasonable compromise between the transmission time and accuracy of the predictions may be to encode with a quality factor of 90 and use a higher resolution.

Figure 2. Accuracy of the YOLOv5 predictions as a function of height and quality factor of the images.

JPEG XL Benchmark

libjxl is the reference implementation of the JPEG XL image format specification. It generally offers a better compression ratio, image fidelity and encoding/decoding speed than other formats. In this experiment, we study the impact of effort and quality factor on the bitrate and encoding speed to identify the parameters that allow the fastest transmission time with minimal loss of accuracy. We encoded images using libjxl with different quality factors and effort. We ignored quality factors below 40 because, as shown in Fig. 2, the accuracy of the predictions is too low. Efforts are presets that enable or disable JPEG XL encoding tools. Slower animals (e.g., kitten) enable more tools to reduce the bitrate at the cost of a slower encoding speed.

Fig. 3 and Table 1 show the results. The behavior of JPEG XL is not deterministic. Images with different content and resolution yield different results. However, effort generally offers diminishing returns. For example, “kitten,” compared to “cheetah,” allows an average bitrate saving of 9.5% while being 97.5% slower to encode.

Figure 3. Average bitrate, in bits per pixel, as a function of the quality factor for each effort. The y-axis is in a logarithmic scale.

Table 1. Average encoding speed, in megapixels per second, as a function of the quality factor for each effort.

Conclusion

In this paper, we describe how we benchmarked technologies that support an edge offloading mechanism in an Edge IIoT search and rescue use case. Our results allow us to identify the efficiency or quality configurations that will enable the system to operate in real-time. In subsequent works, we will build the edge offloading mechanism.

Additional Information

2022. Brochu, D. Spatharakis, D. Dechouniotis, A. Leivadeas and S. Papavassiliou. 2022. “Benchmarking real-time image processing for offloading at the edge”. In IEEE International Mediterranean Conference on Communication and Networking (MeditCom), p. 90-93.