Dr. Annika Reinke

Postdoctoral Researcher

Annika Reinke joined the division of Intelligent Medical Systems at the German Cancer Research Center (DKFZ) to adapt mathematical concepts to societally relevant topics, like scientific benchmarking and validation.

Having published disruptive findings on biomedical image analysis challenges in Nature Communications, she is a founding member of the initiative of Biomedical Image Analysis ChallengeS (BIAS) aiming for bringing biomedical image analysis challenges to the next level of quality.

She serves as the secretary of the MICCAI special interest group on biomedical challenges and as an active member and taskforce lead of the MONAI working group on evaluation, reproducibility and benchmarking.

Publications

Christodoulou, E., Reinke, A., Houhou, R., Kalinowski, P., Erkan, S., Sudre, C. H., Burgos, N., Boutaj, S., Loizillon, S., Solal, M., Rieke, N., Cheplygina, V., Antonelli, M., Mayer, L. D., Tizabi, M. D., Cardoso, M. J., Simpson, A., Jäger, P. F., Kopp-Schneider, A., … Maier-Hein, L. (2024). Confidence intervals uncovered: Are we ready for real-world medical imaging AI? (arXiv:2409.17763). arXiv. https://doi.org/10.48550/arXiv.2409.17763

Mais, L., Hirsch, P., Managan, C., Kandarpa, R., Rumberger, J. L., Reinke, A., Maier-Hein, L., Ihrke, G., & Kainmueller, D. (2024). FISBe: A real-world benchmark dataset for instance segmentation of long-range thin filamentous structures (arXiv:2404.00130). arXiv. https://doi.org/10.48550/arXiv.2404.00130

Reinke, A., Tizabi, M. D., Baumgartner, M., Eisenmann, M., Heckmann-Nötzel, D., Kavur, A. E., Rädsch, T., Sudre, C. H., Acion, L., Antonelli, M., Arbel, T., Bakas, S., Benis, A., Buettner, F., Cardoso, M. J., Cheplygina, V., Chen, J., Christodoulou, E., Cimini, B. A., … Maier-Hein, L. (2024). Understanding metric-related pitfalls in image analysis validation. Nature Methods, 1–13. https://doi.org/10.1038/s41592-023-02150-0

Kofler, F., Möller, H., Buchner, J. A., de la Rosa, E., Ezhov, I., Rosier, M., Mekki, I., Shit, S., Negwer, M., Al-Maskari, R., Ertürk, A., Vinayahalingam, S., Isensee, F., Pati, S., Rueckert, D., Kirschke, J. S., Ehrlich, S. K., Reinke, A., Menze, B., … Piraud, M. (2023). Panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps (arXiv:2312.02608). arXiv. https://doi.org/10.48550/arXiv.2312.02608

Brandenburg, J. M., Jenke, A. C., Stern, A., Daum, M. T. J., Schulze, A., Younis, R., Petrynowski, P., Davitashvili, T., Vanat, V., Bhasker, N., Schneider, S., Mündermann, L., Reinke, A., Kolbinger, F. R., Jörns, V., Fritz-Kebede, F., Dugas, M., Maier-Hein, L., Klotz, R., … Wagner, M. (2023). Active learning for extracting surgomic features in robot-assisted minimally invasive esophagectomy: a prospective annotation study. Surgical Endoscopy, 37(11), 8577–8593. https://doi.org/10.1007/s00464-023-10447-6

Adler, T. J., Nölke, J.-H., Reinke, A., Tizabi, M. D., Gruber, S., Trofimova, D., Ardizzone, L., Jaeger, P. F., Buettner, F., Köthe, U., & Maier-Hein, L. (2023). Application-driven Validation of Posteriors in Inverse Problems (arXiv:2309.09764). arXiv. https://doi.org/10.48550/arXiv.2309.09764

Tran, T. N., Adler, T. J., Yamlahi, A., Christodoulou, E., Godau, P., Reinke, A., Tizabi, M. D., Sauer, P., Persicke, T., Albert, J. G., & Maier-Hein, L. (2023). Sources of performance variability in deep learning-based polyp detection. International Journal of Computer Assisted Radiology and Surgery, 18(7), 1311–1322. https://doi.org/10.1007/s11548-023-02936-9

Wagner, M., Müller-Stich, B.-P., Kisilenko, A., Tran, D., Heger, P., Mündermann, L., Lubotsky, D. M., Müller, B., Davitashvili, T., Capek, M., Reinke, A., Reid, C., Yu, T., Vardazaryan, A., Nwoye, C. I., Padoy, N., Liu, X., Lee, E.-J., Disch, C., … Bodenstedt, S. (2023). Comparative validation of machine learning algorithms for surgical workflow and skill analysis with the HeiChole benchmark. Medical Image Analysis, 86, 102770. https://doi.org/10.1016/j.media.2023.102770

Eisenmann, M., Reinke, A., Weru, V., Tizabi, M. D., Isensee, F., Adler, T. J., Ali, S., Andrearczyk, V., Aubreville, M., Baid, U., Bakas, S., Balu, N., Bano, S., Bernal, J., Bodenstedt, S., Casella, A., Cheplygina, V., Daum, M., de Bruijne, M., … Maier-Hein, L. (2023). Why is the winner the best? (arXiv:2303.17719). arXiv. https://doi.org/10.48550/arXiv.2303.17719

Rädsch, T., Reinke, A., Weru, V., Tizabi, M. D., Schreck, N., Kavur, A. E., Pekdemir, B., Roß, T., Kopp-Schneider, A., & Maier-Hein, L. (2023). Labelling instructions matter in biomedical image analysis. Nature Machine Intelligence, 1–11. https://doi.org/10.1038/s42256-023-00625-5

Roß, T., Bruno, P., Reinke, A., Wiesenfarth, M., Koeppel, L., Full, P. M., Pekdemir, B., Godau, P., Trofimova, D., Isensee, F., Adler, T. J., Tran, T. N., Moccia, S., Calimeri, F., Müller-Stich, B. P., Kopp-Schneider, A., & Maier-Hein, L. (2023). Beyond rankings: Learning (more) from algorithm validation. Medical Image Analysis, 102765. https://doi.org/10.1016/j.media.2023.102765

Godau, P., Kalinowski, P., Christodoulou, E., Reinke, A., Tizabi, M., Ferrer, L., Jäger, P., & Maier-Hein, L. (2023). Deployment of Image Analysis Algorithms under Prevalence Shifts. https://doi.org/10.48550/ARXIV.2303.12540

Eisenmann, M., Reinke, A., Weru, V., Tizabi, M. D., Isensee, F., Adler, T. J., Godau, P., Cheplygina, V., Kozubek, M., Ali, S., Gupta, A., Kybic, J., Noble, A., de Solórzano, C. O., Pachade, S., Petitjean, C., Sage, D., Wei, D., Wilden, E., … Maier-Hein, L. (2022). Biomedical image analysis competitions: The state of current participation practice (arXiv:2212.08568). arXiv. https://doi.org/10.48550/arXiv.2212.08568

Tran, T. N., Adler, T., Yamlahi, A., Christodoulou, E., Godau, P., Reinke, A., Tizabi, M. D., Sauer, P., Persicke, T., Albert, J. G., & Maier-Hein, L. (2022). Sources of performance variability in deep learning-based polyp detection (arXiv:2211.09708). arXiv. https://doi.org/10.48550/arXiv.2211.09708

Zimmerer, D., Full, P. M., Isensee, F., Jäger, P., Adler, T., Petersen, J., Köhler, G., Ross, T., Reinke, A., Kascenas, A., Jensen, B. S., O’Neil, A. Q., Tan, J., Hou, B., Batten, J., Qiu, H., Kainz, B., Shvetsova, N., Fedulova, I., … Maier-Hein, K. (2022). MOOD 2020: A Public Benchmark for Out-of-Distribution Detection and Localization on Medical Images. IEEE Transactions on Medical Imaging, 41(10), 2728–2738. https://doi.org/10.1109/TMI.2022.3170077

Maier-Hein, L., Reinke, A., Godau, P., Tizabi, M. D., Christodoulou, E., Glocker, B., Isensee, F., Kleesiek, J., Kozubek, M., Reyes, M., Riegler, M. A., Wiesenfarth, M., Baumgartner, M., Eisenmann, M., Heckmann-Nötzel, D., Kavur, A. E., Rädsch, T., Acion, L., Antonelli, M., … Jäger, P. F. (2022). Metrics reloaded: Pitfalls and recommendations for image analysis validation (arXiv:2206.01653). arXiv. https://doi.org/10.48550/arXiv.2206.01653

Antonelli, M., Reinke, A., Bakas, S., Farahani, K., Kopp-Schneider, A., Landman, B. A., Litjens, G., Menze, B., Ronneberger, O., Summers, R. M., van Ginneken, B., Bilello, M., Bilic, P., Christ, P. F., Do, R. K. G., Gollub, M. J., Heckers, S. H., Huisman, H., Jarnagin, W. R., … Cardoso, M. J. (2022). The Medical Segmentation Decathlon. Nature Communications, 13(1), 4128. https://doi.org/10.1038/s41467-022-30695-9

Reinke, A., Tizabi, M. D., Sudre, C. H., Eisenmann, M., Rädsch, T., Baumgartner, M., Acion, L., Antonelli, M., Arbel, T., Bakas, S., Bankhead, P., Benis, A., Cardoso, M. J., Cheplygina, V., Christodoulou, E., Cimini, B., Collins, G. S., Farahani, K., van Ginneken, B., … Maier-Hein, L. (2022). Common Limitations of Image Processing Metrics: A Picture Story (arXiv:2104.05642). arXiv. https://doi.org/10.48550/arXiv.2104.05642

Schellenberg, M., Dreher, K. K., Holzwarth, N., Isensee, F., Reinke, A., Schreck, N., Seitel, A., Tizabi, M. D., Maier-Hein, L., & Gröhl, J. (2022). Semantic segmentation of multispectral photoacoustic images using deep learning. Photoacoustics, 26, 100341. https://doi.org/10.1016/j.pacs.2022.100341

Bron, E. E., Klein, S., Reinke, A., Papma, J. M., Maier-Hein, L., Alexander, D. C., & Oxtoby, N. P. (2022). Ten years of image analysis and machine learning competitions in dementia. NeuroImage, 253, 119083. https://doi.org/10.1016/j.neuroimage.2022.119083

Reinke, A. (2021, October 6). A discovery dive into the world of evaluation — Do’s don’ts and other considerations. MICCAI Educational Initiative. https://medium.com/miccai-educational-initiative/a-discovery-dive-into-the-world-of-evaluation-dos-don-ts-and-other-considerations-4189ab46fe06

Roß, T., Reinke, A., Full, P. M., Wagner, M., Kenngott, H., Apitz, M., Hempe, H., Mindroc-Filimon, D., Scholz, P., Tran, T. N., Bruno, P., Arbeláez, P., Bian, G.-B., Bodenstedt, S., Bolmgren, J. L., Bravo-Sánchez, L., Chen, H.-B., González, C., Guo, D., … Maier-Hein, L. (2021). Comparative validation of multi-instance instrument segmentation in endoscopy: Results of the ROBUST-MIS 2019 challenge. Medical Image Analysis, 70, 101920. https://doi.org/10.1016/j.media.2020.101920

Maier-Hein, L., Wagner, M., Ross, T., Reinke, A., Bodenstedt, S., Full, P. M., Hempe, H., Mindroc-Filimon, D., Scholz, P., Tran, T. N., Bruno, P., Kisilenko, A., Müller, B., Davitashvili, T., Capek, M., Tizabi, M., Eisenmann, M., Adler, T. J., Gröhl, J., … Müller-Stich, B. P. (2021). Heidelberg Colorectal Data Set for Surgical Data Science in the Sensor Operating Room (arXiv:2005.03501). arXiv. https://doi.org/10.48550/arXiv.2005.03501

Wiesenfarth, M., Reinke, A., Landman, B. A., Eisenmann, M., Saiz, L. A., Cardoso, M. J., Maier-Hein, L., & Kopp-Schneider, A. (2021). Methods and open-source toolkit for analyzing and visualizing challenge results. Scientific Reports, 11(1), 2369. https://doi.org/10.1038/s41598-021-82017-6

Maier-Hein, L., Reinke, A., Kozubek, M., Martel, A. L., Arbel, T., Eisenmann, M., Hanbury, A., Jannin, P., Müller, H., Onogur, S., Saez-Rodriguez, J., van Ginneken, B., Kopp-Schneider, A., & Landman, B. A. (2020). BIAS: Transparent reporting of biomedical image analysis challenges. Medical Image Analysis, 66, 101796. https://doi.org/10.1016/j.media.2020.101796

Reinke, A., & Maier-Hein, L. (n.d.). Common limitations of performance metrics in biomedical image analysis. 3.

Dr. Annika Reinke

Publications

Personal Links