Hyper 3D-AI

The aim of this project is to develop software tools that can efficiently analyse images in a three-dimensional context – for example medical images or pictures from cameras mounted on self-driving cars. Artificial intelligence (AI) is already capable of detecting anomalies in MRI images, for example, by classifying the image data. However, many of the existing AI algorithms only work with two-dimensional images. While they can analyse neighbouring pixels in the image, they cannot recognise whether they reside on the same plane as each other in reality.

“We work with point clouds, where we have three-dimensional coordinates for each point,” says Dr. Sandra Lorenz of the Helmholtz Institute Freiberg for Resource Technology. “That is a completely different architecture from what is used for analysing pixels in photos. However, the current methods can’t really cope properly with these point clouds yet, even though point clouds offer a much better depiction of the real world.” The researchers now want to close this gap. By characterising pixels in 3D space, this will open up new possibilities in fields like exploration and mining, medicine and autonomous systems.

AI should then be able to achieve multimodal classification, or in other words distinguish objects or domains out of data coming from multiple sensors. For mining, as one possible application, this could mean the software would automatically recognise deposits of mineral raw materials, for example, based on spectral properties or colours.