Socionext Inc. ("Socionext", or "Company"), a leader in SoC design and application technology, announced that it has jointly developed a new deep learning algorithm with the research group of Professor Nagahara, Institute of Data Capability Science, Osaka University, which does not require the production of huge data. It can accurately detect objects and recognize images under extremely low light conditions by simply fusing multiple models. Socionext Mr. Yukio Sasakawa and Prof. Nagahara from Osaka University reported the research results at the European International Conference on Computer Vision (ECCV 2020) held from 23 to 28 August (BST).

In recent years, despite the rapid development of computer vision technology, the image quality obtained by vehicle cameras and security systems in low-light environments is still not ideal, and the image recognition performance is poor. Continuously improving image recognition performance in low-light environments is still one of the main issues facing computer vision technology. A paper called "Learning to See in the Dark" [1] in CVPR2018 has introduced a deep learning algorithm using the RAW image data of the image sensor, but this algorithm needs to produce more than 200,000 images and more than 1.5 million images. Only one annotation [2] dataset can be used for end-to-end learning, which is time-consuming and expensive, and it is difficult to achieve commercialization (see Figure 1 below).

Figure 1: "Learning to See in the Dark" and RAW image recognition project

In order to solve the above problems, the joint research team of Socionext and Osaka University proposes a learning method using Domain Adaptation through machine learning methods such as Transfer Learning and Knowledge Distillation, that is, using existing datasets. To improve the performance of the target domain model, the details are as follows (Figure 2):

(1) Build an inference model using existing datasets; _(2) Extract knowledge from the above inference models through transfer learning; _(3) Merge models using Glue layer; _(4) Build and generate models through knowledge distillation.

Figure 2: Domain Adaptation Method developed this time

In addition, combining the domain adaptation method and the object detection YOLO model [3], and using the RAW images captured under extreme low-light conditions, a "YOLO in the Dark" detection model can also be constructed. The YOLO in the Dark model enables learning of an object detection model on RAW images using only existing datasets. For those who cannot detect the image after correcting the brightness of the image by using the existing YOLO model (as shown in Figure a), it can be confirmed that the object is detected normally by directly recognizing the RAW image (as shown in Figure b). At the same time, the test results found that the processing amount required for YOLO in the Dark model recognition processing is about half of the conventional model combination (as shown in Figure c below).

Figure 3: "YOLO in the Dark" renderings

The "direct recognition of RAW images" developed by the domain adaptation method this time can be applied not only to object detection in extreme dark conditions, but also to automotive cameras, security systems, and industries. In the future, Socionext also plans to integrate this technology into the company's self-developed image signal processor (ISP) to develop next-generation SoCs, and develop new camera systems based on such SoCs to further improve the company's product performance and help the industry to upgrade.

European International Conference on Computer Vision (ECCV 2020)

Date: August 23-28 (BST)

Location: Online meeting

Topic: YOLO in the Dark - Domain Adaptation Method for Merging Multiple Models -

Speaker: Socionext Inc. Mr. Yukihiro Sasakawa, Prof. Nagahara, Osaka University

Link: https://eccv2020.eu/

Notes:

[1] “Learning to See in the Dark”: CVPR2018, Chen et al.

[2] MS COCO dataset as an example (https://cocodataset.org/)

[3] YOLO (You Only Look Once): One of the deep learning object detection methods

About Socionext

Socionext Inc. is a global innovative company that designs, develops and sells System-on-chips. The company focuses on the world's advanced technologies centered on the consumer, automotive and industrial sectors, and continues to drive the development of today's diverse applications. Socionext brings together world-class expertise, experience and a rich IP portfolio to provide customers with cost-effective solutions and customer experiences. Founded in 2015, the company is headquartered in Yokohama, Japan, and has offices in Japan, Asia, the United States and Europe to lead its product development and sales.

For more details, please visit Socionext official website: http://www.socionext.com.

All company or product names mentioned in this press release are trademarks or registered trademarks of their respective owners. The information published above is the information as of the time of publication, and is subject to change in the future without prior notice. Thank you for your understanding.