Hyper 3D-AI

Artificial Intelligence for 3D multimodal point cloud classification

The aim of this project is to develop software tools that can efficiently analyse images in a three-dimensional context – for example medical images or pictures from cameras mounted on self-driving cars. Artificial intelligence (AI) is already capable of detecting anomalies in MRI images, for example, by classifying the image data. However, many of the existing AI algorithms only work with two-dimensional images. While they can analyse neighbouring pixels in the image, they cannot recognise whether they reside on the same plane as each other in reality.

“We work with point clouds, where we have three-dimensional coordinates for each point,” says Dr. Sandra Lorenz of the Helmholtz Institute Freiberg for Resource Technology. “That is a completely different architecture from what is used for analysing pixels in photos. However, the current methods can’t really cope properly with these point clouds yet, even though point clouds offer a much better depiction of the real world.” The researchers now want to close this gap. By characterising pixels in 3D space, this will open up new possibilities in fields like exploration and mining, medicine and autonomous systems.

AI should then be able to achieve multimodal classification, or in other words distinguish objects or domains out of data coming from multiple sensors. For mining, as one possible application, this could mean the software would automatically recognise deposits of mineral raw materials, for example, based on spectral properties or colours.

Other projects


Hyper 3D-AI

Artificial Intelligence for 3D multimodal point cloud classification

The aim is to develop an artificial intelligence that can achieve the fusion of two-dimensional data with three-dimensional information. Based on this, the software would simultaneously be able to recognise image characteristics as well as the spatial relationships between different objects.
Image: DLR


On the trail of the mystery of the laws of calving

Researchers still face a mystery when it comes to the laws by which glaciers calve. This project aims to use satellite imagery, artificial intelligence, mathematical optimisation and a new data processing pipeline to track the movements of glacier fronts in Antarctica to get closer to solving the mystery.


Solar Image-based Modelling

The aim of the project is to develop an algorithm by which computers can automatically predict the space weather. This will make use of datasets of solar images that have been captured from space. The method could replace computationally demanding physics-based models and deliver space weather forecasts long before the effects of solar events are […]