MEDAL

Data is the driving force behind meaningful progress in research, particularly in the domain of artificial intelligence (AI) for medical imaging. The provision of high-quality, relevant datasets can be used to strategically channel international scientific and technical resources toward solving the most pressing and impactful clinical problems. However, existing benchmarks often fall short by focusing on narrow, repetitive, and clinically irrelevant tasks. To address this critical gap, the Medical Imaging AGI’s Last Exam (MEDAL) initiative introduces a paradigm-shifting, top-down approach to benchmark design. MEDAL aims to create the world’s most challenging and clinically impactful benchmark – a “final exam” that artificial general intelligence (AGI) systems in medical imaging must pass to demonstrate transformative clinical utility.

Through a global crowdsourcing campaign, MEDAL will collect high-stakes, real-world clinical questions and multimodal data that reflect the complexity and variability of everyday medical practice. A multidisciplinary, international consortium, which comprises leaders from diverse fields including AI, clinical medicine, epidemiology, global health, biomedical ethics, and regulatory science, will guide its design and implementation to ensure alignment with real clinical needs. The project tackles four core limitations of existing benchmarks: low diversity (addressed via global crowdsourcing), limited clinical relevance (countered by a top-down approach driven by diverse stakeholders), data sparsity (mitigated through incentivized contributions), and benchmark saturation (countered by using unpublished datasets). The result will be a trustworthy benchmarking framework that sets a new gold standard for evaluating and advancing AI in healthcare, steering the field away from trivial exercises toward solving the most pressing challenges in medical imaging.