NEC develops technology for disaster damage assessment using a Large Language Model (LLM) and image analysis

3 Mins read

NEC Corporation (TSE: 6701) has developed a technology for disaster damage assessment using a Large Language Model (LLM) and image analysis, enabling users to immediately and accurately assess the extent and location of damage from the multitude of images collected when a disaster strikes. In future, NEC will contribute to speeding up evacuation guidance, rescue efforts, and other initial response activities in the event of a disaster by providing this technology to government ministries and municipal authorities in charge of disaster response.

narrowing down field images for damage assessment

Figure 1: Result of narrowing down field images

In recent times, the global community has observed an increase in both the frequency and severity of natural calamities, including heightened instances of heavy rains and substantial earthquakes. In the face of such disasters, it becomes imperative to rapidly and precisely evaluate the extent of destruction in order to facilitate prompt actions like guiding evacuations and conducting rescue operations for those affected by the disaster. However, achieving a swift initial response presents a challenge due to the inadequacy of available information. Current resources such as precipitation and seismic intensity maps, along with textual reports from residents, lack the necessary level of detail concerning the scope and specific locations of the damages. In contrast, images sourced from disaster-stricken areas and shared with local authorities and relevant organizations (captured via smartphones, dash cams, CCTV, etc.) offer significant potential, as they contain comprehensive damage-related data along with geographical coordinates.

To accelerate initial response activities, NEC has developed a technology that can be leveraged for quickly and accurately narrowing down which images are necessary for disaster damage assessment, and then display these images on a map with street address-level accuracy.

Features of the new technology

1. Narrows down field images as per the user’s intent

Images can be narrowed down from an array of field images to include only those aligned with a user’s intent by making use of semantic interpretation of words of an LLM and image analysis to determine image similarity.

Image recognition technology has traditionally been used for narrowing down images to those that are relevant for users. However, these technologies can only recognize images that are pre-trained, thereby limiting which images can be narrowed down. This makes it difficult to conduct surveys in accordance with a user’s intent, which varies according to the type and scale of a disaster, the affected areas, and the extent of the situation.

This newly developed technology leverages Large Language Models to narrow down field images using keywords. Moreover, employing image analysis and specifying the scene a user would like to search for allows to narrow down scenes that are difficult to express in words. In other words, by combining LLM and image analysis, users will be able to accurately narrow down the images to only those that match their intentions, thereby enabling them to quickly respond to disasters as they unfold.

2. Shows the extent and location of damage on a map at street address level

In the case of field images for which the location of a disaster-stricken area is unknown, it is possible to estimate the location with street address-level accuracy. It can then be shown on a map by comparing the images with aerial imagery and map data covering extensive areas.

While NEC has developed technologies for estimating locations by utilizing satellite images and aerial photographs in the past(1), this latest technology has achieved the world’s highest matching accuracy(2)(3), enabling location estimation with greater precision, even for field images captured during a disaster.

To estimate locations, this technology automatically extracts areas such as roads, buildings, and traffic signals from field images and then matches them with map layout information (the shape and layout of roads, buildings, etc.). In this way, it is possible to get a highly accurate estimation of a photographed location, even for images of partially collapsed buildings or partially flooded roads, by proactively using information on roads with a lower risk of damage in the event of an earthquake and information on buildings with a lower risk of submergence in the event of flooding to perform matching.

NEC plans to put this newly developed technology into practical use by FY2025 to help accelerate evacuation guidance, rescue efforts, and other initial response activities in the event of a disaster. In addition, NEC remains committed to making society safer, more secure, and more convenient by expanding the use of LLM and image analysis technology to other applications.

(1) February 10, 2022: NEC develops technology capable of estimating the locations of landscape images from satellite imagery and aerial photographs (press release available in Japanese only)
(2) Top-1% retrieval rate based on retrieval tasks involving 8,884 aerial images from the publicly available CVUSA data set (in the case of ground view images for a 90 degrees field of view)
(3) SemGeo: Semantic Keywords for Cross-View Image Geo-Localization

Featured image source: Pixabay

Read next: Fujitsu, ISID, DIPRO start collaboration to realize DX in the manufacturing industry

Leave a Reply

Your email address will not be published. Required fields are marked *

+ 82 = 89