Published on 13.11.2023
Deep Dive into Imaging at the 2nd Incubator Summer Academy
Once again, together with the platforms Helmholtz.AI, HIFIS, HIDA and HMC we welcomed doctoral and postdoctoral researchers of the Helmholtz Association to the Incubator Summer Academy, held virtually on 18-29 September 2023 in the interactive space of GatherTown. Participants had the opportunity to engage in introductory lectures, hackathons, deep dive workshops, and hands-on tutorials, offering a comprehensive exploration of state-of-the-art data science methods and skills.
Helmholtz Imaging curated a course package that spanned the entire imaging pipeline. The courses covered a spectrum of topics, from image reconstruction and image segmentation to visualization and image analysis. In this recap, we present a brief overview of the discussed topics, along with materials for your personal training and use.
Deep Dive into Regularization in Image Reconstruction: From Model to Data Driven Methods
Workshop organizers: Martin Burger, Lorenz Kuger, Samira Kabri, Tim Roith
The goal of image reconstruction is to retrieve an image that can only be accessed implicitly through certain measurements. In X-ray tomography, for example, the measurement created by scanning a human body is called a sinogram, from which we aim to reconstruct an internal image of the body. Image reconstruction problems are examples of inverse problems, which we explored from various perspectives in this course: The dry-run consisted of getting to know the main pitfalls in inverse problems and how they are traditionally circumvented by so-called regularization methods. Longing for the deep dive, we proceeded to contrast these model-based methods with a fully data-driven end-to-end image reconstruction approach, using the popular U-Net architecture. To compromise between both extremes, we settled on a Plug and Play approach combining information from both the model and the data. Finally, we ventured into uncharted waters with the Bayesian perspective. Here, our interest lies not only in a single reconstruction but rather in a probability distribution of reconstructions, which makes it possible to quantify uncertainty.
Deep Dive into Machine Learning for Instance Segmentation
Workshop organizers: Dagmar Kainmüller, Peter Johannes Hirsch
Instance Segmentation refers to the task of delineating each distinct object of interest in an image, like e.g. cell nuclei in microscopy images from the biomedical domain, or cars and pedestrians in street scenes. Building machine learning models to solve instance segmentation tasks is challenging, in particular in case many similar-looking objects are densely clustered. In our course, we first introduced basic methods for encoding object instances into machine learning model outputs. We then dived into the challenge of training such models to focus on topologically relevant situations, i.e., where a decision needs to be made on whether there are two closely adjacent instances, or rather a single larger one. After this lecture-style part, we went hands-on into an instance segmentation challenge in colab, where participants were able to play with different instance representations and training objectives.
Access to the hands-on tutorial and instructions
Deep Dive into Rendering 3D datasets
Workshop organizer: Deborah Schmidt
In our workshop dedicated to rendering volumetric images, we primarily focused on voxel-based datasets, which have a wide array of applications across various fields including medical imaging (such as MRI and CT Scans), geospatial analysis, and fluid dynamics simulations. These can be visualized using techniques such as volume rendering, isosurface extraction, and slice-based views. We demonstrated how to transform segmentations into meshes that define the shapes of 3D objects and illustrated how to render these shapes effectively in Blender.
Deep Dive into Machine Learning- based Image Analysis
Workshop organizers: Paul Jäger, Lukas Klein, Kim-Celine Kahl
In recent years, there have been many advances in medical imaging, moving from manual radiomics-based decisions to fully automated, deep learning-based analysis. In our lecture, we introduced the fundamentals and current trends in Machine Learning-based image analysis. We began with a straightforward introduction to Deep Learning, covering basic concepts like perceptrons, gradient descent, overfitting, underfitting, and the importance of hyperparameters. We also explored neural network architectures such as Convolutional Neural Networks (CNNs) and Transformers and finally had an outlook on the currently hot-discussed topic of foundation models. Moving forward, we discussed the common challenges encountered in Machine Learning-based image analysis, such as hyperparameter selection, the need to be aware of shifts in data distribution, and the choice of appropriate evaluation metrics. The lecture then shifted to practical applications of AI in medical imaging, showcasing tasks ranging from classification and object detection to techniques like reconstruction and anomaly detection. We also introduced recent advances like Large Language Models (LLMs) and Multimodal LLMs. Additionally, we explored the concept of explainable AI (XAI), which empowers domain experts to generate insights from explanations generated by algorithms. In the final part of the lecture, we conducted an interactive workshop demonstrating a practical application of deep learning-based image analysis. We walked through the nnU-Net framework which enables training state-of-the-art segmentation models for biomedical problems, even for domain experts without extensive machine learning expertise. The workshop included a brief overview of nnU-Net, followed by a step-by-step demonstration using Python. We worked with datasets from the Medical Segmentation Decathlon, training a model on one dataset and applying a pretrained model to predict segmentations on another. Finally, we showcased the predictions made by the model using MITK, a medical image viewer. In the image below, you can see the segmentation of a prostate in various views, including a 3D perspective.
Access to the tutorial incl. code and instructions
Stay tuned for more upcoming events! Sign up for our newsletter: https://helmholtz-imaging.de/newsletter/