Dr. Annika Reinke

Postdoctoral Researcher

Annika Reinke joined the division of Intelligent Medical Systems at the German Cancer Research Center (DKFZ) to adapt mathematical concepts to societally relevant topics, like scientific benchmarking and validation.

Having published disruptive findings on biomedical image analysis challenges in Nature Communications, she is a founding member of the initiative of Biomedical Image Analysis ChallengeS (BIAS) aiming for bringing biomedical image analysis challenges to the next level of quality.

She serves as the secretary of the MICCAI special interest group on biomedical challenges and as an active member and taskforce lead of the MONAI working group on evaluation, reproducibility and benchmarking.

Publications

Reinke, A., Tizabi, M. D., Baumgartner, M., Eisenmann, M., Heckmann-Nötzel, D., Kavur, A. E., Rädsch, T., Sudre, C. H., Acion, L., Antonelli, M., Arbel, T., Bakas, S., Benis, A., Buettner, F., Cardoso, M. J., Cheplygina, V., Chen, J., Christodoulou, E., Cimini, B. A., … Maier-Hein, L. (2024). Understanding metric-related pitfalls in image analysis validation. Nature Methods, 1–13. https://doi.org/10.1038/s41592-023-02150-0
Kofler, F., Möller, H., Buchner, J. A., de la Rosa, E., Ezhov, I., Rosier, M., Mekki, I., Shit, S., Negwer, M., Al-Maskari, R., Ertürk, A., Vinayahalingam, S., Isensee, F., Pati, S., Rueckert, D., Kirschke, J. S., Ehrlich, S. K., Reinke, A., Menze, B., … Piraud, M. (2023). Panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps (arXiv:2312.02608). arXiv. https://doi.org/10.48550/arXiv.2312.02608
Brandenburg, J. M., Jenke, A. C., Stern, A., Daum, M. T. J., Schulze, A., Younis, R., Petrynowski, P., Davitashvili, T., Vanat, V., Bhasker, N., Schneider, S., Mündermann, L., Reinke, A., Kolbinger, F. R., Jörns, V., Fritz-Kebede, F., Dugas, M., Maier-Hein, L., Klotz, R., … Wagner, M. (2023). Active learning for extracting surgomic features in robot-assisted minimally invasive esophagectomy: a prospective annotation study. Surgical Endoscopy, 37(11), 8577–8593. https://doi.org/10.1007/s00464-023-10447-6
Adler, T. J., Nölke, J.-H., Reinke, A., Tizabi, M. D., Gruber, S., Trofimova, D., Ardizzone, L., Jaeger, P. F., Buettner, F., Köthe, U., & Maier-Hein, L. (2023). Application-driven Validation of Posteriors in Inverse Problems (arXiv:2309.09764). arXiv. https://doi.org/10.48550/arXiv.2309.09764
Tran, T. N., Adler, T. J., Yamlahi, A., Christodoulou, E., Godau, P., Reinke, A., Tizabi, M. D., Sauer, P., Persicke, T., Albert, J. G., & Maier-Hein, L. (2023). Sources of performance variability in deep learning-based polyp detection. International Journal of Computer Assisted Radiology and Surgery, 18(7), 1311–1322. https://doi.org/10.1007/s11548-023-02936-9
Wagner, M., Müller-Stich, B.-P., Kisilenko, A., Tran, D., Heger, P., Mündermann, L., Lubotsky, D. M., Müller, B., Davitashvili, T., Capek, M., Reinke, A., Reid, C., Yu, T., Vardazaryan, A., Nwoye, C. I., Padoy, N., Liu, X., Lee, E.-J., Disch, C., … Bodenstedt, S. (2023). Comparative validation of machine learning algorithms for surgical workflow and skill analysis with the HeiChole benchmark. Medical Image Analysis, 86, 102770. https://doi.org/10.1016/j.media.2023.102770
Eisenmann, M., Reinke, A., Weru, V., Tizabi, M. D., Isensee, F., Adler, T. J., Ali, S., Andrearczyk, V., Aubreville, M., Baid, U., Bakas, S., Balu, N., Bano, S., Bernal, J., Bodenstedt, S., Casella, A., Cheplygina, V., Daum, M., de Bruijne, M., … Maier-Hein, L. (2023). Why is the winner the best? (arXiv:2303.17719). arXiv. https://doi.org/10.48550/arXiv.2303.17719
Rädsch, T., Reinke, A., Weru, V., Tizabi, M. D., Schreck, N., Kavur, A. E., Pekdemir, B., Roß, T., Kopp-Schneider, A., & Maier-Hein, L. (2023). Labelling instructions matter in biomedical image analysis. Nature Machine Intelligence, 1–11. https://doi.org/10.1038/s42256-023-00625-5
Roß, T., Bruno, P., Reinke, A., Wiesenfarth, M., Koeppel, L., Full, P. M., Pekdemir, B., Godau, P., Trofimova, D., Isensee, F., Adler, T. J., Tran, T. N., Moccia, S., Calimeri, F., Müller-Stich, B. P., Kopp-Schneider, A., & Maier-Hein, L. (2023). Beyond rankings: Learning (more) from algorithm validation. Medical Image Analysis, 102765. https://doi.org/10.1016/j.media.2023.102765
Godau, P., Kalinowski, P., Christodoulou, E., Reinke, A., Tizabi, M., Ferrer, L., Jäger, P., & Maier-Hein, L. (2023). Deployment of Image Analysis Algorithms under Prevalence Shifts. https://doi.org/10.48550/ARXIV.2303.12540
Eisenmann, M., Reinke, A., Weru, V., Tizabi, M. D., Isensee, F., Adler, T. J., Godau, P., Cheplygina, V., Kozubek, M., Ali, S., Gupta, A., Kybic, J., Noble, A., de Solórzano, C. O., Pachade, S., Petitjean, C., Sage, D., Wei, D., Wilden, E., … Maier-Hein, L. (2022). Biomedical image analysis competitions: The state of current participation practice (arXiv:2212.08568). arXiv. https://doi.org/10.48550/arXiv.2212.08568
Tran, T. N., Adler, T., Yamlahi, A., Christodoulou, E., Godau, P., Reinke, A., Tizabi, M. D., Sauer, P., Persicke, T., Albert, J. G., & Maier-Hein, L. (2022). Sources of performance variability in deep learning-based polyp detection (arXiv:2211.09708). arXiv. https://doi.org/10.48550/arXiv.2211.09708
Zimmerer, D., Full, P. M., Isensee, F., Jäger, P., Adler, T., Petersen, J., Köhler, G., Ross, T., Reinke, A., Kascenas, A., Jensen, B. S., O’Neil, A. Q., Tan, J., Hou, B., Batten, J., Qiu, H., Kainz, B., Shvetsova, N., Fedulova, I., … Maier-Hein, K. (2022). MOOD 2020: A Public Benchmark for Out-of-Distribution Detection and Localization on Medical Images. IEEE Transactions on Medical Imaging, 41(10), 2728–2738. https://doi.org/10.1109/TMI.2022.3170077
Maier-Hein, L., Reinke, A., Godau, P., Tizabi, M. D., Christodoulou, E., Glocker, B., Isensee, F., Kleesiek, J., Kozubek, M., Reyes, M., Riegler, M. A., Wiesenfarth, M., Baumgartner, M., Eisenmann, M., Heckmann-Nötzel, D., Kavur, A. E., Rädsch, T., Acion, L., Antonelli, M., … Jäger, P. F. (2022). Metrics reloaded: Pitfalls and recommendations for image analysis validation (arXiv:2206.01653). arXiv. https://doi.org/10.48550/arXiv.2206.01653
Antonelli, M., Reinke, A., Bakas, S., Farahani, K., Kopp-Schneider, A., Landman, B. A., Litjens, G., Menze, B., Ronneberger, O., Summers, R. M., van Ginneken, B., Bilello, M., Bilic, P., Christ, P. F., Do, R. K. G., Gollub, M. J., Heckers, S. H., Huisman, H., Jarnagin, W. R., … Cardoso, M. J. (2022). The Medical Segmentation Decathlon. Nature Communications, 13(1), 4128. https://doi.org/10.1038/s41467-022-30695-9
Reinke, A., Tizabi, M. D., Sudre, C. H., Eisenmann, M., Rädsch, T., Baumgartner, M., Acion, L., Antonelli, M., Arbel, T., Bakas, S., Bankhead, P., Benis, A., Cardoso, M. J., Cheplygina, V., Christodoulou, E., Cimini, B., Collins, G. S., Farahani, K., van Ginneken, B., … Maier-Hein, L. (2022). Common Limitations of Image Processing Metrics: A Picture Story (arXiv:2104.05642). arXiv. https://doi.org/10.48550/arXiv.2104.05642
Schellenberg, M., Dreher, K. K., Holzwarth, N., Isensee, F., Reinke, A., Schreck, N., Seitel, A., Tizabi, M. D., Maier-Hein, L., & Gröhl, J. (2022). Semantic segmentation of multispectral photoacoustic images using deep learning. Photoacoustics, 26, 100341. https://doi.org/10.1016/j.pacs.2022.100341
Bron, E. E., Klein, S., Reinke, A., Papma, J. M., Maier-Hein, L., Alexander, D. C., & Oxtoby, N. P. (2022). Ten years of image analysis and machine learning competitions in dementia. NeuroImage, 253, 119083. https://doi.org/10.1016/j.neuroimage.2022.119083
Reinke, A. (2021, October 6). A discovery dive into the world of evaluation — Do’s don’ts and other considerations. MICCAI Educational Initiative. https://medium.com/miccai-educational-initiative/a-discovery-dive-into-the-world-of-evaluation-dos-don-ts-and-other-considerations-4189ab46fe06
Roß, T., Reinke, A., Full, P. M., Wagner, M., Kenngott, H., Apitz, M., Hempe, H., Mindroc-Filimon, D., Scholz, P., Tran, T. N., Bruno, P., Arbeláez, P., Bian, G.-B., Bodenstedt, S., Bolmgren, J. L., Bravo-Sánchez, L., Chen, H.-B., González, C., Guo, D., … Maier-Hein, L. (2021). Comparative validation of multi-instance instrument segmentation in endoscopy: Results of the ROBUST-MIS 2019 challenge. Medical Image Analysis, 70, 101920. https://doi.org/10.1016/j.media.2020.101920
Maier-Hein, L., Wagner, M., Ross, T., Reinke, A., Bodenstedt, S., Full, P. M., Hempe, H., Mindroc-Filimon, D., Scholz, P., Tran, T. N., Bruno, P., Kisilenko, A., Müller, B., Davitashvili, T., Capek, M., Tizabi, M., Eisenmann, M., Adler, T. J., Gröhl, J., … Müller-Stich, B. P. (2021). Heidelberg Colorectal Data Set for Surgical Data Science in the Sensor Operating Room (arXiv:2005.03501). arXiv. https://doi.org/10.48550/arXiv.2005.03501
Wiesenfarth, M., Reinke, A., Landman, B. A., Eisenmann, M., Saiz, L. A., Cardoso, M. J., Maier-Hein, L., & Kopp-Schneider, A. (2021). Methods and open-source toolkit for analyzing and visualizing challenge results. Scientific Reports, 11(1), 2369. https://doi.org/10.1038/s41598-021-82017-6
Maier-Hein, L., Reinke, A., Kozubek, M., Martel, A. L., Arbel, T., Eisenmann, M., Hanbury, A., Jannin, P., Müller, H., Onogur, S., Saez-Rodriguez, J., van Ginneken, B., Kopp-Schneider, A., & Landman, B. A. (2020). BIAS: Transparent reporting of biomedical image analysis challenges. Medical Image Analysis, 66, 101796. https://doi.org/10.1016/j.media.2020.101796
Reinke, A., & Maier-Hein, L. (n.d.). Common limitations of performance metrics in biomedical image analysis. 3.