Evaluation and
Benchmarking Working Group
Mission Statement
The Evaluation and Benchmarking MONAI working group aims at providing guidelines, infrastructure, and practical tools for evaluation and benchmarking of medical image analysis methods. It focuses on leading the community towards the identification and adoption of best practices for evaluation and benchmarking and on identifying practical solutions to improve reproducibility.
Initiatives
Best Practices & Infrastructure
- • Development of algorithm validation standards
- • Implementation of automated evaluation systems
- • Creation of benchmark frameworks and protocols
- • Establishment of quality control guidelines
Research & Community Standards
- • Development of novel evaluation metrics
- • Implementation of performance analysis tools
- • Establishment of data sharing practices
- • Creation of reproducibility guidelines
Group Leads
Dr. Annika Reinke
Deputy Head and Group Lead Validation of Intelligent Systems
German Cancer Research Center (DKFZ)
Benchmark Working Group Chair
Dr. Carole Sudre
Associate Professor
University College London
Benchmark Working Group Chair
Meeting Notes
GitHub Wiki
Resources
Development Resources
Collaboration Opportunities
Development Contributions
- • Contribute to evaluation metrics development
- • Help create benchmarking tools
- • Develop validation frameworks
Community Engagement
- • Join our regular surveys
- • Share your expertise in validation and benchmarking
- • Participate in standards development