Published October 2014
| Version v1
Journal article
The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)
Creators
- Menze, Bjoern
- Jakab, Andras
- Bauer, Stefan
- Kalpathy-Cramer, Jayashree
- Farahani, Keyvan
- Kirby, Justin
- Burren, Yuliya
- Porz, Nicole
- Slotboom, Johannes
- Wiest, Roland
- Lanczi, Levente
- Gerstner, Elisabeth
- Weber, Marc-Andre
- Arbel, Tal
- Avants, Brian
- Ayache, Nicholas
- Buendia, Patricia
- Collins, Louis
- Cordier, Nicolas
- Corso, Jason
- Criminisi, Antonio
- Das, Tilak
- Delingette, Hervé
- Demiralp, Cagatay
- Durst, Christopher
- Dojat, Michel
- Doyle, Senan
- Festa, Joana
- Forbes, Florence
- Geremia, Ezequiel
- Glocker, Ben
- Golland, Polina
- Guo, Xiaotao
- Hamamci, Andac
- Iftekharuddin, Khan
- Jena, Raj
- John, Nigel
- Konukoglu, Ender
- Lashkari, Danial
- Antonio Mariz, Jose
- Meier, Raphael
- Pereira, Sergio
- Precup, Doina
- Price, S. J.
- Riklin-Raviv, Tammy
- Reza, Syed
- Ryan, Michael
- Schwartz, Lawrence
- Shin, Hoo-Chang
- Shotton, Jamie
- Silva, Carlos
- Sousa, Nuno
- Subbanna, Nagesh
- Szekely, Gabor
- Taylor, Thomas
- Thomas, Owen
- Tustison, Nicholas
- Unal, Gozde
- Vasseur, Flor
- Wintermark, Max
- Hye Ye, Dong
- Zhao, Liang
- Zhao, Binsheng
- Zikic, Darko
- Prastawa, Marcel
- Reyes, Mauricio
- van Leemput, Koen
Contributors
Others:
- Analysis and Simulation of Biomedical Images (ASCLEPIOS) ; Centre Inria d'Université Côte d'Azur (CRISAM) ; Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)
- Computational Image Analysis and Radiology (CIR lab) ; Medizinische Universität Wien = Medical University of Vienna
- Institute for Surgical Technology and Biomechanics [Bern] (ISTB) ; Universität Bern = University of Bern = Université de Berne (UNIBE)
- Massachusetts General Hospital [Boston]
- Department of Health and Human Services ; National Institutes of Health [Bethesda, MD, USA] (NIH)
- Inselspital Bern
- University of Debrecen
- Division of Medical Physics in Radiology [Heidelberg] ; German Cancer Research Center - Deutsches Krebsforschungszentrum [Heidelberg] (DKFZ)
- Diagnostic and Interventional Radiology [Heidelberg] ; Heidelberg University Hospital [Heidelberg]
- Centre for Intelligent Machines (CIM) ; McGill University = Université McGill [Montréal, Canada]
- Penn Image Computing & Science Lab [Philadelphia] (PICSL) ; University of Pennsylvania
- INFOTECH Soft
- McConnell Brain Imaging Centre (MNI) ; Montreal Neurological Institute and Hospital ; McGill University = Université McGill [Montréal, Canada]-McGill University = Université McGill [Montréal, Canada]
- State University of New York (SUNY)
- Microsoft Research [Cambridge] (Microsoft) ; Microsoft Research
- Cambridge University Hospitals - NHS (CUH) ; University of Cambridge [UK] (CAM)
- Computer Science Department [Stanford] ; Stanford University
- Department of radiology and medical imaging [Charlottesville] ; University of Virginia
- [GIN] Grenoble Institut des Neurosciences (GIN) ; Université Joseph Fourier - Grenoble 1 (UJF)-Institut National de la Santé et de la Recherche Médicale (INSERM)
- Modelling and Inference of Complex and Structured Stochastic Systems (MISTIS) ; Centre Inria de l'Université Grenoble Alpes ; Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Jean Kuntzmann (LJK) ; Université Pierre Mendès France - Grenoble 2 (UPMF)-Université Joseph Fourier - Grenoble 1 (UJF)-Institut polytechnique de Grenoble - Grenoble Institute of Technology (Grenoble INP)-Centre National de la Recherche Scientifique (CNRS)-Université Pierre Mendès France - Grenoble 2 (UPMF)-Université Joseph Fourier - Grenoble 1 (UJF)-Institut polytechnique de Grenoble - Grenoble Institute of Technology (Grenoble INP)-Centre National de la Recherche Scientifique (CNRS)
- Universidade do Minho = University of Minho [Braga]
- Computer Science and Artificial Intelligence Laboratory [Cambridge] (CSAIL) ; Massachusetts Institute of Technology (MIT)
- Columbia University [New York]
- Faculty of Engineering and Natural Sciences (Sabanci University) ; Sabanci University [Istanbul]
- Old Dominion University [Norfolk] (ODU)
- Addenbrooke's Hospital ; Cambridge University NHS Trust
- Bangor University
- Department of Surgery ; Université de Genève = University of Geneva (UNIGE)
- McGill University = Université McGill [Montréal, Canada]
- Bristol Glaciology Centre ; School of Geographical Sciences
- Molecular Carcinogenesis [Sutton] ; Institute of cancer research
- Center for Neuroscience and Cell Biology (CNC) (CNC) ; Universidade de Coimbra = University of Coimbra [Portugal] (UC)-Neuroscience Research Domain
- Computer Vision Laboratory - ETHZ [Zurich] ; Eidgenössische Technische Hochschule - Swiss Federal Institute of Technology [Zürich] (ETH Zürich)
- Department of Radiology [San Francisco] ; University of California [San Francisco] (UC San Francisco) ; University of California (UC)-University of California (UC)
- Purdue University [West Lafayette]
- Department of Computer Science [New York] ; Columbia University [New York]
- Scientific Computing and Imaging Institute (SCI Institute) ; University of Utah
- Department of radiology (Massachusetts General Hospital) ; Massachusetts General Hospital [Boston]
- European Project: 291080,EC:FP7:ERC,ERC-2011-ADG_20110209,MEDYMA(2012)
Description
In this paper we report the set-up and results of the Multimodal Brain Tumor Image Segmentation (BRATS) benchmark organized in conjunction with the MICCAI 2012 and 2013 conferences. Twenty state-of-the-art tumor segmentation algorithms were applied to a set of 65 multi-contrast MR scans of low- and high-grade glioma patients - manually annotated by up to four raters - and to 65 comparable scans generated using tumor simulation software. Quantitative evaluations revealed considerable disagreement between the human raters in segmenting various tumor sub-regions (Dice scores in the range 74-85%), illustrating the difficulty of this task. We found that different algorithms worked best for different sub-regions (reaching performance comparable to human inter-rater variability), but that no single algorithm ranked in the top for all subregions simultaneously. Fusing several good algorithms using a hierarchical majority vote yielded segmentations that consistently ranked above all individual algorithms, indicating remaining opportunities for further methodological improvements. The BRATS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource.
Abstract
International audienceAdditional details
Identifiers
- URL
- https://inria.hal.science/hal-00935640
- URN
- urn:oai:HAL:hal-00935640v2
Origin repository
- Origin repository
- UNICA