Dr. Annika Reinke

DKFZ research representative

Dr. Annika Reinke earned her PhD degree in 2023, focusing on eliminating flaws in biomedical image analysis validation. She continues her work as a postdoctoral researcher and deputy head by addressing underrepresented societally relevant topics, particularly scientific benchmarking and validation. Leading the “Validation of Intelligent Systems” group, Annika aims to enhance the quality of validation pipelines for biomedical AI algorithms. She holds influential positions in various international groups, including the secretary of the MICCAI Special Interest Group on biomedical challenges and the chair of the MONAI Working Group on Evaluation and Benchmarking.

Publications

Christodoulou, E., Reinke, A., Houhou, R., Kalinowski, P., Erkan, S., Sudre, C. H., Burgos, N., Boutaj, S., Loizillon, S., Solal, M., Rieke, N., Cheplygina, V., Antonelli, M., Mayer, L. D., Tizabi, M. D., Cardoso, M. J., Simpson, A., Jäger, P. F., Kopp-Schneider, A., … Maier-Hein, L. (2024). Confidence intervals uncovered: Are we ready for real-world medical imaging AI? (arXiv:2409.17763). arXiv. https://doi.org/10.48550/arXiv.2409.17763

Mais, L., Hirsch, P., Managan, C., Kandarpa, R., Rumberger, J. L., Reinke, A., Maier-Hein, L., Ihrke, G., & Kainmueller, D. (2024). FISBe: A Real-World Benchmark Dataset for Instance Segmentation of Long-Range thin Filamentous Structures. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 22249–22259. https://doi.org/10.1109/CVPR52733.2024.02100

Reinke, A., Tizabi, M. D., Baumgartner, M., Eisenmann, M., Heckmann-Nötzel, D., Kavur, A. E., Rädsch, T., Sudre, C. H., Acion, L., Antonelli, M., Arbel, T., Bakas, S., Benis, A., Buettner, F., Cardoso, M. J., Cheplygina, V., Chen, J., Christodoulou, E., Cimini, B. A., … Maier-Hein, L. (2024). Understanding metric-related pitfalls in image analysis validation. Nature Methods, 1–13. https://doi.org/10.1038/s41592-023-02150-0

Kofler, F., Möller, H., Buchner, J. A., de la Rosa, E., Ezhov, I., Rosier, M., Mekki, I., Shit, S., Negwer, M., Al-Maskari, R., Ertürk, A., Vinayahalingam, S., Isensee, F., Pati, S., Rueckert, D., Kirschke, J. S., Ehrlich, S. K., Reinke, A., Menze, B., … Piraud, M. (2023). Panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps (arXiv:2312.02608). arXiv. https://doi.org/10.48550/arXiv.2312.02608

Brandenburg, J. M., Jenke, A. C., Stern, A., Daum, M. T. J., Schulze, A., Younis, R., Petrynowski, P., Davitashvili, T., Vanat, V., Bhasker, N., Schneider, S., Mündermann, L., Reinke, A., Kolbinger, F. R., Jörns, V., Fritz-Kebede, F., Dugas, M., Maier-Hein, L., Klotz, R., … Wagner, M. (2023). Active learning for extracting surgomic features in robot-assisted minimally invasive esophagectomy: a prospective annotation study. Surgical Endoscopy, 37(11), 8577–8593. https://doi.org/10.1007/s00464-023-10447-6

Adler, T. J., Nölke, J.-H., Reinke, A., Tizabi, M. D., Gruber, S., Trofimova, D., Ardizzone, L., Jaeger, P. F., Buettner, F., Köthe, U., & Maier-Hein, L. (2023). Application-driven Validation of Posteriors in Inverse Problems (arXiv:2309.09764). arXiv. https://doi.org/10.48550/arXiv.2309.09764

Tran, T. N., Adler, T. J., Yamlahi, A., Christodoulou, E., Godau, P., Reinke, A., Tizabi, M. D., Sauer, P., Persicke, T., Albert, J. G., & Maier-Hein, L. (2023). Sources of performance variability in deep learning-based polyp detection. International Journal of Computer Assisted Radiology and Surgery, 18(7), 1311–1322. https://doi.org/10.1007/s11548-023-02936-9

Wagner, M., Müller-Stich, B.-P., Kisilenko, A., Tran, D., Heger, P., Mündermann, L., Lubotsky, D. M., Müller, B., Davitashvili, T., Capek, M., Reinke, A., Reid, C., Yu, T., Vardazaryan, A., Nwoye, C. I., Padoy, N., Liu, X., Lee, E.-J., Disch, C., … Bodenstedt, S. (2023). Comparative validation of machine learning algorithms for surgical workflow and skill analysis with the HeiChole benchmark. Medical Image Analysis, 86, 102770. https://doi.org/10.1016/j.media.2023.102770

Eisenmann, M., Reinke, A., Weru, V., Tizabi, M. D., Isensee, F., Adler, T. J., Ali, S., Andrearczyk, V., Aubreville, M., Baid, U., Bakas, S., Balu, N., Bano, S., Bernal, J., Bodenstedt, S., Casella, A., Cheplygina, V., Daum, M., de Bruijne, M., … Maier-Hein, L. (2023). Why is the winner the best? (arXiv:2303.17719). arXiv. https://doi.org/10.48550/arXiv.2303.17719

Rädsch, T., Reinke, A., Weru, V., Tizabi, M. D., Schreck, N., Kavur, A. E., Pekdemir, B., Roß, T., Kopp-Schneider, A., & Maier-Hein, L. (2023). Labelling instructions matter in biomedical image analysis. Nature Machine Intelligence, 1–11. https://doi.org/10.1038/s42256-023-00625-5

Roß, T., Bruno, P., Reinke, A., Wiesenfarth, M., Koeppel, L., Full, P. M., Pekdemir, B., Godau, P., Trofimova, D., Isensee, F., Adler, T. J., Tran, T. N., Moccia, S., Calimeri, F., Müller-Stich, B. P., Kopp-Schneider, A., & Maier-Hein, L. (2023). Beyond rankings: Learning (more) from algorithm validation. Medical Image Analysis, 102765. https://doi.org/10.1016/j.media.2023.102765

Godau, P., Kalinowski, P., Christodoulou, E., Reinke, A., Tizabi, M., Ferrer, L., Jäger, P., & Maier-Hein, L. (2023). Deployment of Image Analysis Algorithms under Prevalence Shifts. https://doi.org/10.48550/ARXIV.2303.12540

Eisenmann, M., Reinke, A., Weru, V., Tizabi, M. D., Isensee, F., Adler, T. J., Godau, P., Cheplygina, V., Kozubek, M., Ali, S., Gupta, A., Kybic, J., Noble, A., de Solórzano, C. O., Pachade, S., Petitjean, C., Sage, D., Wei, D., Wilden, E., … Maier-Hein, L. (2022). Biomedical image analysis competitions: The state of current participation practice (arXiv:2212.08568). arXiv. https://doi.org/10.48550/arXiv.2212.08568

Tran, T. N., Adler, T., Yamlahi, A., Christodoulou, E., Godau, P., Reinke, A., Tizabi, M. D., Sauer, P., Persicke, T., Albert, J. G., & Maier-Hein, L. (2022). Sources of performance variability in deep learning-based polyp detection (arXiv:2211.09708). arXiv. https://doi.org/10.48550/arXiv.2211.09708

Zimmerer, D., Full, P. M., Isensee, F., Jäger, P., Adler, T., Petersen, J., Köhler, G., Ross, T., Reinke, A., Kascenas, A., Jensen, B. S., O’Neil, A. Q., Tan, J., Hou, B., Batten, J., Qiu, H., Kainz, B., Shvetsova, N., Fedulova, I., … Maier-Hein, K. (2022). MOOD 2020: A Public Benchmark for Out-of-Distribution Detection and Localization on Medical Images. IEEE Transactions on Medical Imaging, 41(10), 2728–2738. https://doi.org/10.1109/TMI.2022.3170077

Maier-Hein, L., Reinke, A., Godau, P., Tizabi, M. D., Christodoulou, E., Glocker, B., Isensee, F., Kleesiek, J., Kozubek, M., Reyes, M., Riegler, M. A., Wiesenfarth, M., Baumgartner, M., Eisenmann, M., Heckmann-Nötzel, D., Kavur, A. E., Rädsch, T., Acion, L., Antonelli, M., … Jäger, P. F. (2022). Metrics reloaded: Pitfalls and recommendations for image analysis validation (arXiv:2206.01653). arXiv. https://doi.org/10.48550/arXiv.2206.01653

Antonelli, M., Reinke, A., Bakas, S., Farahani, K., Kopp-Schneider, A., Landman, B. A., Litjens, G., Menze, B., Ronneberger, O., Summers, R. M., van Ginneken, B., Bilello, M., Bilic, P., Christ, P. F., Do, R. K. G., Gollub, M. J., Heckers, S. H., Huisman, H., Jarnagin, W. R., … Cardoso, M. J. (2022). The Medical Segmentation Decathlon. Nature Communications, 13(1), 4128. https://doi.org/10.1038/s41467-022-30695-9

Reinke, A., Tizabi, M. D., Sudre, C. H., Eisenmann, M., Rädsch, T., Baumgartner, M., Acion, L., Antonelli, M., Arbel, T., Bakas, S., Bankhead, P., Benis, A., Cardoso, M. J., Cheplygina, V., Christodoulou, E., Cimini, B., Collins, G. S., Farahani, K., van Ginneken, B., … Maier-Hein, L. (2022). Common Limitations of Image Processing Metrics: A Picture Story (arXiv:2104.05642). arXiv. https://doi.org/10.48550/arXiv.2104.05642

Schellenberg, M., Dreher, K. K., Holzwarth, N., Isensee, F., Reinke, A., Schreck, N., Seitel, A., Tizabi, M. D., Maier-Hein, L., & Gröhl, J. (2022). Semantic segmentation of multispectral photoacoustic images using deep learning. Photoacoustics, 26, 100341. https://doi.org/10.1016/j.pacs.2022.100341

Bron, E. E., Klein, S., Reinke, A., Papma, J. M., Maier-Hein, L., Alexander, D. C., & Oxtoby, N. P. (2022). Ten years of image analysis and machine learning competitions in dementia. NeuroImage, 253, 119083. https://doi.org/10.1016/j.neuroimage.2022.119083

Reinke, A. (2021, October 6). A discovery dive into the world of evaluation — Do’s don’ts and other considerations. MICCAI Educational Initiative. https://medium.com/miccai-educational-initiative/a-discovery-dive-into-the-world-of-evaluation-dos-don-ts-and-other-considerations-4189ab46fe06

Roß, T., Reinke, A., Full, P. M., Wagner, M., Kenngott, H., Apitz, M., Hempe, H., Mindroc-Filimon, D., Scholz, P., Tran, T. N., Bruno, P., Arbeláez, P., Bian, G.-B., Bodenstedt, S., Bolmgren, J. L., Bravo-Sánchez, L., Chen, H.-B., González, C., Guo, D., … Maier-Hein, L. (2021). Comparative validation of multi-instance instrument segmentation in endoscopy: Results of the ROBUST-MIS 2019 challenge. Medical Image Analysis, 70, 101920. https://doi.org/10.1016/j.media.2020.101920

Maier-Hein, L., Wagner, M., Ross, T., Reinke, A., Bodenstedt, S., Full, P. M., Hempe, H., Mindroc-Filimon, D., Scholz, P., Tran, T. N., Bruno, P., Kisilenko, A., Müller, B., Davitashvili, T., Capek, M., Tizabi, M., Eisenmann, M., Adler, T. J., Gröhl, J., … Müller-Stich, B. P. (2021). Heidelberg Colorectal Data Set for Surgical Data Science in the Sensor Operating Room (arXiv:2005.03501). arXiv. https://doi.org/10.48550/arXiv.2005.03501

Wiesenfarth, M., Reinke, A., Landman, B. A., Eisenmann, M., Saiz, L. A., Cardoso, M. J., Maier-Hein, L., & Kopp-Schneider, A. (2021). Methods and open-source toolkit for analyzing and visualizing challenge results. Scientific Reports, 11(1), 2369. https://doi.org/10.1038/s41598-021-82017-6

Maier-Hein, L., Reinke, A., Kozubek, M., Martel, A. L., Arbel, T., Eisenmann, M., Hanbury, A., Jannin, P., Müller, H., Onogur, S., Saez-Rodriguez, J., van Ginneken, B., Kopp-Schneider, A., & Landman, B. A. (2020). BIAS: Transparent reporting of biomedical image analysis challenges. Medical Image Analysis, 66, 101796. https://doi.org/10.1016/j.media.2020.101796

Reinke, A., & Maier-Hein, L. (n.d.). Common limitations of performance metrics in biomedical image analysis. 3.

Dr. Annika Reinke

Publications

Personal Links