Reduce risks to firefighters, settlements and (critical) infrastructure

Imaging solutions combined with AI and 5G - A new approach and promising to assess forest health and detect and monitor wildfires

The project

The number of wildfires increased dramatically in recent years due to diminishing forest health caused by the drastically changing climate across the world. Longer dry weather periods and infections by a variety of diseases, e.g., bark beetles affect the large forest areas in Northern Europe. These changes result in lower carbon accumulation, vast ecological changes in addition to financial losses.

In 2022 in Germany 2397 wildfires were registered with a size of more than 3058 Hectares of totally destroyed forests. Most of them in Brandenburg and Saxonia

Monitoring forest areas with aircraft or UAVs can help to detect potential issues in time. In addition, observation towers equipped with the latest camera and sensor technology – and combined with Artificial Intelligence (AI) to detect smoke columns as an indication of a wildfire – can be a perfect solution to assist in the fight against the destruction of vast amounts of wildlife habitats.

A research project – initiated by Landkreis Goerlitz in Saxonia and the BMDV (Ministry for Digital Matters and Mobility), Germany in cooperation with GGS – focused on monitoring an overall forest area of 170 sqkm in the Northeast of Germany.

The project delivery was based on a multitude of approaches: Due to the large area covered, a surveying aircraft and UAVs were used for monitoring by air, while on the ground observation towers with the latest sensor technology and ground sensor were used.

Camera setup for aerial survey

The camera setup for the aerial survey was based on GGS’s OIS-Technology with an addition of NIR and thermal cameras.

GGS chose a Phase One iXM-RS 150 with a 90 mm lens for Nadir imaging. Two additional Phase One Achromatic iXM-100 cameras with NIR filter (band 700-850 nm and 750-850 nm) and two 70 mm lenses were added.

For generating a 3D surface Model, four Phase One iXM-100 oblique cameras each equipped with an 80 mm lens were also integrated into the camera pod. Alongside the Nadir cameras, two further thermal cameras capture the entire footprint with lower resolution to indicate the influence of micro-climate changes on the forest health.

Figure 1: OIS camera pod

Data processing

The gathered data – RGB, CIR and NDVI – was processed with Phase One’s iX Capture software first, before further processing with specialized photogrammetric software to generate true orthophotos.

The red Edge data was generated after producing the orthophotos of both NIR bands and refined by Raster conversation between the 2 NIR orthophotos. The RGB data of the oblique system (Nadir and oblique) was processed with Skyline’s software package Photomesh to generate a 3D model.

Figure 2: CIR

Figure 3: NDVI

Examples RGB/CIR/NDVI/NIR1/NIR2 of same area

Next steps - The generate data models which are then used for a variety of applications:

Forest health assessment

The RGB, NIR, CIR, NDVI and Red Edge orthophotos are used by forest experts to detect high-risk areas for wildfires, areas suffering from drought or diseases – such as the bark beetle – or to assess soil health status.

These areas are then labelled as high risk and hence have higher priority regarding fire observation. The red edge detection in the band between 700 and 750 nm allows for faster and easier detection of health problems than NIR/CIR images alone. Thus, it can be labelled as a rapid indicator of stress within trees.

Emergency maps

The RGB orthophotos are used as part of the emergency navigation system for firefighters in combination with other information e.g., the accessibility of paths, lakes for accessing water, info on infrastructure as well as areas that are inaccessible for a variety of reasons.

This data e.g., a true orthophoto mosaic, are part of a cloud-based GIS application. This data is updated frequently with ancillary data e.g., data gathered by UAV monitoring of wildfires – detailing the fire’s dimensions, its dynamic as well as the associated risk for nearby settlements and infrastructure. This navigation application is also used to coordinate and direct the emergency responders.

3D-Simulation

The 3D data from the oblique cameras is used to generate a perfect 3D model of the area being processed with Skyline’s Photomesh. Using ten Skylines Terra Explorer Software helps to perform analytics in the Surface model e.g. visual line of sight of the observation towers and optimized UAV flights to detect fire nests.

Using different AI-modules, the 3D model also allows to simulate wildfires to check whether the fires are visible from more remote points like the towers or UAV flights at a certain height using different flight strategies.

Observation towers

There are already three observation towers available with the 170 sqkm, however only one of these is yet equipped with an older camera system. As a next step, an innovative camera array capturing RGB, NIR, AC and Thermal data – which are mounted on a gimbal – will be installed on two towers. The gimbal is scanning the area at 270 degrees while the cameras are taking images with a 60% sidelap.

This data will be streamed via 5G to a server (central crisis management server) and analyzed using AI algorithms to detect wildfires. In combination with the observations from the other towers, the location can be easily calculated and verified. If a wildfire is detected, an alarm will be generated, and a UAV deployed for further assessment.

Figure 4: 3D model of the forest area during data processing using the Skyline software

UAV Inspection

Smaller areas can be monitored with UAVs equipped with RGB, NIR and Red Edge cameras. To ensure a long flight time, these drones only carry small format compact cameras. After an alarm was raised, initially a single UAV will be deployed, whereas in certain situations, a UAV swarm may be sent out to observe the larger for additional fire nests. The data gathered – in combination with visual inspection of wind speed and direction – is crucial to determine the speed and direction of fires. Streamed via the 5G network, the combination of all data gathered informs the strategy and deployment of the emergency services, while mitigating the risk to firefighters, settlements and infrastructure.

Figure 5: Fire Simulations to train AI-modules