Context:
In the field of 2D or 3D region of interest (ROI) delineation in medical imaging, combining segmentations of anatomical structures from different sources proves beneficial. This is particularly relevant given the development of multimodal and multi-parametric imaging acquisition devices. Furthermore, combining multiple expert tracings of the same ROI can be useful for evaluating segmentation methods to estimate a consensus ground truth, thereby accounting for intra- or inter-expert variability. Finally, combining multiple segmentation methods can help improve results and their reproducibility. Thus, even though deep learning segmentation methods offer highly accurate results, it is sometimes necessary to combine several architectures or parameterizations to pool segmentation results and reduce variability.
To address these various challenges in segmentation method evaluation and fusion, the CREATIS lab and its collaborators have been developing a method [1,2,3] for several years to compute a mutual shape from n masks corresponding to n segmentations of the same region. Within this working group, we initially proposed a variational formulation of the foundational STAPLE algorithm [4], enabling the evaluation of segmentation methods without a gold standard. The mathematical framework we established, based on domain optimization tools, subsequently allowed us to study different shape metrics, including an original shape metric based on information theory concepts [1]. This metric also proved relevant for the robust fusion of 2D or 3D segmentation methods [1,2,3].
Objectives:
Based on this mathematical framework we aim for developing a deep learning based model to combine several segmentation maps and compare it with our analytical methods.
The aim of this internship is to design a new deep learning architecture similar to [5]. It would be composed of a (convolutional) auto-encoder that is trained to reconstruct segmentation maps combined with a contrastive learning framework to structure the embedding space according to the desired properties of the shape distance (e.g. mutual shape). The advantage of such a model is that it may compute distances between shapes in a Euclidean space (i.e. the distance between the embeddings) and thus more easily compute the final reference shape of the n input segmentation maps by combining the n embeddings and reconstructing the corresponding segmentation map.
There are several scientific challenges that need to be addresses, e.g. the way to constrain the latent space according to the target distance measure, the architecture of the neural network and loss function and the way to combine several embeddings.
In the first place, we will work with binary segmentation maps and later, potentially, extend the approach to probability maps. An advantage would be that a combination based on probability maps would better incorporate uncertainty and, in theory, lead to a better fused reference shape. In this regard, the intern will collaborate with another intern working on the mathematical aspects of the extension of the existing method (of mutual shape) to probability maps.
Environnement:
This internship will be conducted at INSA Lyon and supervised by Stefan Duffner from the LIRIS lab (IMAGINE team) and Stéphanie Jehan-Besson and Patrick Clarysse from the CREATIS lab (MYRIAD team).
Contact:
stefan dot duffner at insa-lyon dot fr
Bibliography
[1] « A Mutual Reference Shape for Segmentation Fusion and Evaluation », S. Jehan-Besson, R. Clouard, C. Tilmant, A. de Cesare, A. Lalande, J. Lebenberg, P. Clarysse, L. Sarry, F. Frouin, M. Garreau, arXiv:2102.08939 (2021).
[2] « Optimization of a mutual shape based on the Fréchet-Nikodym metric for 3D shapes fusion », . S.Jehan-Besson, P. Clarysse, R. Clouard, F. Frouin. International Conference on Curves and Surfaces (2022).
[3] « Optimization of a shape metric based on information theory applied to segmentation fusion and evaluation in multimodal MRI for DIPG tumor analysis », S. Jehan-Besson, R. Clouard, N. Boddaert, J. Grill, F. Frouin, Int. Conf . on Geometric Science of Information GSI (2021).
[4] «Simultaneous truth and performance level estimation (STAPLE) : an algorithm for the validation of image segmentation » S.K.Warfield, K. H. Zou et W. M. Wells III In : IEEE TMI (2004)
[5] « Learning Wasserstein Embeddings », N. Courty, R. Flamary, M. Ducoffe, ICLR (2018)
