Réunion
Unrolling and un/self/*/supervised learning for inverse problems
Axes scientifiques :
- Théorie et méthodes
Organisateurs :
- - Matthieu Kowalski (LISN)
- - Thomas Moreau (INRIA Saclay)
Nous vous rappelons que, afin de garantir l'accès de tous les inscrits aux salles de réunion, l'inscription aux réunions est gratuite mais obligatoire.
Inscriptions
68 personnes membres du GdR IASIS, et 49 personnes non membres du GdR, sont inscrits à cette réunion.
Capacité de la salle : 120 personnes. 3 Places restantes
Inscriptions closes pour cette journée
Annonce
Inverse problems, where hidden variables are reconstructed from indirect measurements, often rely on iterative optimization methods that become computationally expensive as data size grows. This thematic day will focus on the emerging paradigm of algorithm unrolling, as a tool for designing state-of-the-art deep neural network architectures. By unrolling the iterations of traditional optimization algorithms, we can learn their parameters as if they were neural network weights, allowing for faster, more efficient solutions that exploit the forward model. More generally, the program will cover the interest of deep (un/self/*/supervised) learning for solving inverse problems.
NB. La date limite pour la prise en charge de mission (orateurs ou participants) est le 18 avril, 12h. A cette date, un dossier de demande complet doit être disponible (sur Notilus, une fois la mission validée sur Etamine, ou bien transmis à gestion-gdriasis@services.cnrs.fr, selon le statut du demandeur).
Programme
- 09h00 – 09h20: Welcome
- 09h20 – 10h00: Thomas Moreau (Inria MIND) « Unrolling algorithms for inverse problems: the critical role of warm starts in bilevel optimization«
- 10h00 – 10h20: Christophe Kervazo (Télécom Paris) « Unrolled Multiplicative Updates for Nonnegative Matrix Factorization applied to Hyperspectral Unmixing«
- 10h20 – 10h40: Jonathan Kern (CEA – IRFU) « Efficient Learned Reconstruction of Radio Interferometric Images Using Deep Unrolling«
- 10h40 – 11h00: Pause
- 11h00 – 11h40: Emilie Chouzenoux (Inria OPIS) « Deep Unfolding Approach for Limited-Angle Computed Tomography Image Reconstruction«
- 11h40 – 12h00: Sarah Reynaud (IMT Atlantique) « Lower-Level Solvers in Bi-Level Approaches for the EEG source imaging Problem«
- 12h00 – 12h20: Félix Riedel (Univ. St Étienne) « Méthode d’apprentissage auto-supervisée informée par la physique pour la reconstruction en microscopie holographique«
- 12h20 – 15h00: Déjeuner + Session Posters
- 15h00 – 15h40: Julian Tachella (CNRS & ENS Lyon) « UNSURE: Unknown Noise level Stein’s Unbiased Risk Estimator«
- 15h40 – 16h00: Jérémy Scanvic (ENS Lyon) « Scale-Equivariant Imaging: Self-Supervised Learning for Image Super-Resolution and Deblurring«
- 16h00 – 16h20: Vincent Lostanlen (CNRS LS2N) « Understanding equivariant self-supervised learning in musical pitch space«
- 16h20 – 17h00: Pause + Posters
- 17h00 – 17h20: Nathan Buskulic (Machine Learning Genoa Center) « Implicit Regularization of the Deep Inverse Prior via Inertial Gradient Flow«
- 17h20 – 17h40: Can Pouliquen (Inria OCKHAM) « Schur’s Positive-Definite Network: Deep Learning in the SPD cone with structure«
- 17h40 • Discussions & Clôture
Keynote Speakers:
Emilie Chouzenoux (Inria OPIS)
Deep Unfolding Approach for Limited-Angle Computed Tomography Image Reconstruction
This talk presents recent developments made in the context of a collaboration between University Paris Saclay, and GE Healthcare R&D department. We addressed the inverse problem arising in computational imaging, of the regions of interest (ROI) reconstruction from a limited number of computed tomography (CT) measurements. Classical model-based iterative reconstruction methods lead to images with predictable features. Still, they often suffer from tedious parameterization and slow convergence. On the contrary, deep learning methods are fast, and they can reach high reconstruction quality by leveraging information from large datasets, but they lack interpretability. At the crossroads of both methods, deep unfolding networks have been recently proposed. Their design includes the physics of the imaging system and the steps of an iterative optimization algorithm.
Motivated by the success of these networks for various applications, we introduced in [1] an unfolding neural network designed for ROI CT reconstruction from limited data. Few-view truncated data are effectively handled thanks to a robust non-convex data fidelity term combined with a sparsity-inducing regularization function. We unfold a block alternating proximal algorithm, embedded in an iterative reweighted scheme, allowing the learning of key parameters in a supervised manner. Our experiments showcase an improvement over several state-of-the-art methods, including a model-based iterative scheme, a multi-scale deep learning architecture, and other deep unfolding networks.
[1] M. Savanier, E. Chouzenoux, J.-C. Pesquet, C. Riddell. Deep Unfolding of the DBFB Algorithm with Application to ROI CT Imaging with Limited Angular Density. IEEE Transactions on Computational Imaging, vol. 9, pp. 502-516, 2023
Julian Tachella (CNRS & ENS Lyon)
UNSURE: Unknown Noise level Stein’s Unbiased Risk Estimator
Recently, many self-supervised learning methods for signal reconstruction have been proposed that can learn from noisy data alone, bypassing the need for ground-truth references. Most existing methods cluster around two classes: i) Noise2Self and similar cross-validation methods that require very mild knowledge about the noise distribution, and ii) Stein’s Unbiased Risk Estimator (SURE) and similar approaches that assume full knowledge of the distribution. The first class of methods is often suboptimal compared to supervised learning, and the second class is often impractical, as the noise level is generally unknown in real-world applications. In this talk, I will provide a theoretical framework that characterizes this expressivity-robustness trade-off and propose a new approach based on SURE, but unlike the standard SURE, does not require knowledge about the noise level. Finally, I will present applications on various imaging inverse problems.
Thomas Moreau (Inria Mind)
Unrolling algorithms for inverse problems: the critical role of warm starts in bilevel optimization
Algorithm unrolling, a method that parameterizes classical optimization algorithms as differentiable procedures, has emerged as a powerful tool for solving inverse problems. These unrolled methods allow for the learning of problem-specific parameters, often leading to improved performance in the early iterations of optimization.
In this talk, I explore the links between algorithm unrolling and bilevel optimization. First, I will discuss results highlighting unrolled algorithms’ asymptotic limitations. These findings emphasize the advantages of using unrolling with limited iterations. I will then discuss some of my recent works on combining unrolled algorithms with dictionary learning to capture data-driven structures in inverse problem solutions. These results highlight the non-robustness of the gradient estimation obtained with unrolling. A possible way to limit this drawback is to rely on warm starting, which has been known to be critical to deriving converging bilevel optimization algorithms. This offers new insights into designing efficient and robust plug-and-play algorithms based on unrolled denoisers for solving challenging inverse problems.
Résumés des contributions orales
Christophe Kervazo (Télécom Paris) « Unrolled Multiplicative Updates for Nonnegative Matrix Factorization applied to Hyperspectral Unmixing«
HyperSpectral Unmixing (HSU), the problem of separating mixed spectra of overlapping materials in a hyperspectral image, has motivated dedicated algorithmic developments in the last two decades, which can be roughly categorized as « model-based » and « deep learning-based ». On the one hand, traditional model-based algorithms frequently guarantee interpretable results. On the other hand, deep-learning-based approaches are often faster at inference time and may obtain better empirical results. This work utilizes the strengths of both approaches by exploring the deep unrolling paradigm. Our contribution is twofold. First, we revisit the Non-Adaptive Learned Multiplicative Updates (NALMU) algorithm, which is based on deep unrolling of the well-known Multiplicative Updates. We relate the NALMU to the minimization of an explicit cost function under some assumptions. Such guarantees are unique in the HSU field. Second, we propose a new unrolling-based algorithm coined RALMU, which overcomes NALMU shortcomings and considerably improves its practical performance. RALMU is tested on astrophysics and remote sensing datasets, exhibiting superior performances compared to other unrolled HSU algorithms and vanilla Multiplicative Updates.
Jonathan Kern (CEA – IRFU) « Efficient Learned Reconstruction of Radio Interferometric Images Using Deep Unrolling«
Many existing image deconvolution methods rely on iterative optimization algorithms which lack of computational efficiency for new emerging applications such as radio-interferometric imaging with large scale data provided by telescopes like the future Square Kilometer Array Observatory (SKAO). In this work, we propose a new deconvolution algorithm building upon algorithm unrolling, which allows to merge model-based iterative methods and deep learning ones. Our method significantly reduces computational complexity while preserving reconstruction quality. Additionally, it is designed to be flexible enough to handle variations in UV coverage and noise levels, thereby minimizing the need for retraining the neural network under different conditions.
Sarah Reynaud (IMT Atlantique) « Lower-Level Solvers in Bi-Level Approaches for the EEG source imaging Problem«
Resolving inverse problems is a long-standing and still challenging topic in research. This importance stems from two main reasons: (1) many real-world problems can be formulated as inverse problems, and (2) most of them are ill-posed, making them difficult to solve.
The classical approach to solving inverse problems relies on optimization, which depends on the choice of cost function, constraints, data representation, and hyperparameters. An alternative that is gaining traction today involves using machine learning models. While this approach has shown impressive results in various applications, it still has limitations: it may suffer from overfitting or underfitting, depending heavily on the quality and distribution of the training data, and often fails to incorporate all the physical properties of the problem.
In this work, we propose a bi-level method that combines a physics-based optimization process (lower-level) with a supervised learning approach for selecting suitable data representations and hyperparameters (upper-level). We solve this bi-level problem using unrolling techniques, which approximate the lower-level iterative optimization process in a differentiable and learnable way.
In this work we focus on different lower-level solvers based on long short term (LSTM) neural networks, and designed to fine-tune the gradient while considering the actual state, thereby improving the descent direction for updates, and having a faster convergence in only a few number of iterations. This method is applied to the problem of EEG source imaging (ESI), an inverse problem which consists in estimating brain activity from electrical signals recorded on the scalp. We demonstrate the potential of integrating LSTM-based models into optimization frameworks, paving the way for advancements in both machine learning and biomedical applications.
Félix Riedel (Univ. St Étienne) « Méthode d’apprentissage auto-supervisée informée par la physique pour la reconstruction en microscopie holographique«
Les techniques d’imagerie de phase permettent de caractériser des objets transparents par leurs propriétés morphologiques et réfringentes. Parmi celles-ci, l’holographie en ligne est une modalité d’imagerie computationnelle consistant à enregistrer le motif de diffraction d’un échantillon observé sous illumination cohérente et à reconstruire numériquement le déphasage introduit par celui-ci.
Les approches classiques basées sur des modèles physiques (reconstructions itératives, approches problèmes inverses, etc.) permettent d’obtenir des résultats quantitatifs mais nécessitent souvent un temps de calcul important. Dans ce contexte, l’usage des réseaux de neurones permet une diminution considérable de ces temps de calcul mais leur apprentissage supervisé nécessite une bonne base d’apprentissage, ce qui pose la question de la représentativité des objets d’intérêt si la base est construite par simulation ou celle de la constitution de vérités terrains pour des images expérimentales.
Nous proposons une méthodologie d’apprentissage auto-supervisée d’un réseau fϑ basée sur la minimisation d’une fonction de coût L(ϑ) mesurant la distance entres deux modèles physiques des observations mz et mz+∆z et les hologrammes expérimentaux dz et dz+∆z correspondant. Cette approche exploite le principe de diversité de phase afin de mieux contraindre l’apprentissage. Après la phase d’entraînement, le réseau peut reconstruire un plan de transmittance t à partir d’un seul hologramme observé dz .
Nous montrons que cette méthode permet d’entraîner un réseau de neurones dont les performances de reconstruction sur un unique hologramme approchent celles d’une reconstruction Gerchberg-Saxton exploitant 2 hologrammes, pour un temps de calcul réduit.
Jérémy Scanvic (ENS Lyon) « Scale-Equivariant Imaging: Self-Supervised Learning for Image Super-Resolution and Deblurring »
Self-supervised methods have recently proved to be nearly as effective as supervised ones in various imaging inverse problems, paving the way for learning-based approaches in scientific and medical imaging applications where ground truth data is hard or expensive to obtain. These methods critically rely on invariance to translations and/or rotations of the image distribution to learn from incomplete measurement data alone. However, existing approaches fail to obtain competitive performances in the problems of image super-resolution and deblurring, which play a key role in most imaging systems. In this presentation, I will explain why invariance to roto-translations is insufficient to learn from measurements that only contain low-frequency information. Instead, I will present scale-equivariant imaging as an alternate self-supervised method that leverages the fact that many image distributions are approximately scale-invariant, and explain why it enables recovering the high-frequency information lost in the measurement process. Our experiments on real datasets show that scale-equivariant imaging performs favorably against other self-supervised approaches, and close to fully supervised learning
Vincent Lostanlen (CNRS LS2N) « Understanding equivariant self-supervised learning in musical pitch space«
Although deep neural networks can estimate the key of a musical piece, their supervision incurs a massive annotation effort. Against this shortcoming, we present STONE, the first self-supervised tonality estimator. The architecture behind STONE, named ChromaNet, is a convnet with octave equivalence which outputs a key signature profile (KSP) of 12 structured logits. First, we train ChromaNet to regress artificial pitch transpositions between any two unlabeled musical excerpts from the same audio track, as measured as cross-power spectral density (CPSD) within the circle of fifths (CoF). We observe that this self-supervised pretext task leads KSP to correlate with tonal key signature. Based on this observation, we extend STONE to output a structured KSP of 24 logits, and introduce supervision so as to disambiguate major versus minor keys sharing the same key signature. Applying different amounts of supervision yields semi-supervised and fully supervised tonality estimators: i.e., Semi-TONEs and Sup-TONEs. We evaluate these estimators on FMAK, a new dataset of 5489 real-world musical recordings with expert annotation of 24 major and minor keys. We find that Semi-TONE matches the classification accuracy of Sup-TONE with reduced supervision and outperforms it with equal supervision.
https://arxiv.org/abs/2407.07408
Nathan Buskulic (Machine Learning Genoa Center) « Implicit Regularization of the Deep Inverse Prior via Inertial Gradient Flow«
While neural networks proved to be very powerful tools for the resolution of inverse problem, even in the self-supervised case, they benefit from very few theoretical guarantees, especially when it comes to the recovery guarantees. In this talk, I will provide convergence and recovery guarantees for self-supervised neural networks applied to inverse problems, such as Deep Image/Inverse Prior, and trained with inertia featuring both viscous and geometric Hessian-driven dampings, which extends previous results obtained for gradient flow and gradient descent. I will focus mainly on the continuous case, i.e., the trajectory of a dynamical system, where we can show that the network can be trained with an optimal accelerated exponential convergence rate compared to the rate obtained with gradient flow. I will also show some experimental results using Deep Inverse Prior on imaging inverse problems that demonstrate this acceleration when the optimization parameters are chosen properly.
Can Pouliquen (Inria OCKHAM) « Schur’s Positive-Definite Network: Deep Learning in the SPD cone with structure«
Estimating matrices in the symmetric positive-definite (SPD) cone is of interest for many applications ranging from computer vision to graph learning. While there exist various convex optimization-based estimators, they remain limited in expressivity due to their model-based approach. The success of deep learning motivates the use of learning-based approaches to estimate SPD matrices with neural networks in a data-driven fashion. However, designing effective neural architectures for SPD learning is challenging, particularly when the task requires additional structural constraints, such as element-wise sparsity. Current approaches either do not ensure that the output meets all desired properties or lack expressivity. In this paper, we introduce SpodNet, a novel and generic learning module that guarantees SPD outputs and supports additional structural constraints. Notably, it solves the challenging task of learning jointly SPD and sparse matrices. Our experiments illustrate the versatility and relevance of SpodNet layers for such applications.
https://arxiv.org/pdf/2406.09023
Résumé des présentations Posters
Luc LE MAGOAROU (INSA Rennes): « Model-based learning for the wireless channel«
Many aspects of wireless communication systems involve estimating, leveraging, or optimizing the propagation channel. While physical models exist to describe the channel, their accuracy often depends on assumptions that may not hold in practice. Machine learning methods can mitigate this limitation. Specifically, model-based learning approaches utilize the prior knowledge of physical models, even if imperfect, to structure, train, and initialize learning methods. This talk will introduce various model-based learning methods applied to several wireless channel-related applications, such as channel estimation, channel-based localization, and radio environment compression.
Sébastien Herbreteau (CREST/ENSAI): « débruitage d’image faiblement supervisé«
Supervised deep learning has become the method of choice for image denoising. It involves the training of neural networks on large datasets composed of pairs of noisy and clean images. However, the necessity of training data that are specific to the targeted application constrains the widespread use of denoising networks. Recently, several approaches have been developed to overcome this difficulty by whether artificially generating realistic clean/noisy image pairs, or training exclusively on noisy images. In this presentation, we show that, contrary to popular belief, denoising networks specialized in the removal of Gaussian noise can be efficiently leveraged in favor of real-world image denoising, even without additional training. For this to happen, an appropriate variance-stabilizing transform (VST) has to be applied beforehand. We propose an algorithm termed Noise2VST for the learning of such a model-free VST. Our approach requires only the input noisy image and an off-the-shelf Gaussian denoiser. We demonstrate through extensive experiments the efficiency and superiority of Noise2VST in comparison to existing methods trained in the absence of specific clean/noisy pairs.
Vitor TUCCI RAMOS (Orange – Centrale Supélec/L2S): « Review of Deep Unfolding Algorithms for Symbol Detection in MIMO Communication Systems«
Abstract: This presentation introduces and compares various iterative algorithms unfolded in deep neural networks model architectures for the problem of Multiple-Input Multiple-Output (MIMO) symbol detection in the field of radio mobile communication (physical layer). The deep learning models are promising for tackling this telecommunication task, due to their ability to handle existing MIMO nonlinear detection algorithms. This work proposes a benchmark of existing state-of-the-art algorithms and models developed to solve the MIMO detection, which can be represented as an inverse noisy problem. The resulting deep learning models are evaluated based on performance (symbol error rates versus signal-to-noise ratios), computational complexity (numbers of trainable parameters and of floating operations per second) and their robustness to communication impairments (namely correlated channels and imperfect channel-state information). The results obtained are promising, illustrating and demonstrating the potential advantages of these solutions over traditional detection methods, the minimum mean squared error detection estimate, especially in the case of MIMO configurations with large numbers of antennas at the transmitter and the receiver stages.
Kannara Mom (Univ Grenoble Alpes): « Deep Gauss-Newton for X-ray phase contrast imaging«
The in-line X-ray phase contrast imaging technique relies on the measurement of Fresnel diffraction intensity patterns due to the phase shift and the absorption induced by the object. The recovery of both phase and absorption is an ill-posed non-linear inverse problem. Several algorithms have been proposed for the phase (and absorption) retrieval problem, among them are algorithms that rely on the linearization of the Fresnel integral to obtain fast reconstructions. These methods all have limitations and are only valid under some restrictive assumptions about the imaging conditions or the object. The reconstructions obtained from these are often used as initial estimates for iterative methods that refine the reconstruction, but they can be time consuming.
Recently, deep learning methods have shown potential for solving ill-posed inverse problem such as the phase problem. Several architectures of Convolutional Neural Networks (CNNs) have been proposed to recover the phase and absorption directly from the intensity measurement.
We present a learned iterative scheme, the Deep Gauss-Newton algorithm, which is obtained by unrolling a Gauss-Newton scheme. The proposed method combines Convolutional Neural Network (CNN) and knowledge of the imaging physics given by the forward operator and its Fréchet derivative.
We show that taking into account the knowledge of the forward model enhances the quality of the reconstructions and allows a better generalization. Compared to the standard Gauss-Newton method, the DGN method both substantially improved the reconstruction and reduced the calculation time. We demonstrate that, compared with other unrolled optimization schemes, this also improves resolution.
Matthieu Muller (Univ Grenoble Alpes): « Deep Unrolled Method for Pattern-Robust Demosaicking«
Most commercially available cameras and low-cost image sensors capture color images using Color Filter Arrays (CFAs) patterns of color filters placed over the sensor’s photodetectors. Traditional demosaicking methods are typically designed for specific CFA layouts, leading to limited adaptability across different patterns. A promising approach to solve the demosaicking problem with multiple CFAs is algorithm unrolling. It transforms iterative optimization algorithms into trainable models. However, despite employing the knowledge of the used CFA to guide the reconstruction, it tends to learn CFA-specialized solutions, struggling to generalize beyond the CFA seen during training.
We introduce DRUID (Demosaicking with Robust UnrollIng Design), a general demosaicking algorithm designed to overcome this limitation. DRUID is trained using a novel loss function that applies geometric transformations to CFA patterns. This forces the model to learn a robust prior that is less dependent on the specific CFAs encountered during training.
Sébastien Valette (Creatis Insa Lyon): « Représentation neuronale implicite régularisée pour la reconstruction tomographique en microscopie électronique«
Nous explorons une nouvelle approche pour la reconstruction tomographique en microscopie électronique. La microscopie électronique en transmission (MET) est une technique de pointe pour l’analyse des matériaux à l’échelle nanométrique, mais elle est confrontée à des défis significatifs, notamment les artefacts d’élongation dus aux angles d’acquisition limités. Les contraintes mécaniques des microscopes restreignent l’inclinaison des échantillons à environ 120 degrés, ce qui entraîne des distorsions dans les reconstructions obtenues par les méthodes classiques.
Pour surmonter ces limitations, nous avons développé une méthode inspirée des champs de radiance neuronaux (NeRF) qui utilise une représentation neuronale implicite du volume à reconstruire. Cette représentation permet de capturer les détails complexes du volume de manière continue, contrairement aux représentations discrètes comme les voxels. Nous avons également intégré une régularisation de type Variation Totale (TV) pour améliorer la qualité de la reconstruction en réduisant les artefacts et le bruit. Nous avons comparé notre approche, que nous appelons T-NeF-TV, avec des méthodes traditionnelles telles que SIRT et MLEM. Les résultats obtenus sur des données simulées et expérimentales montrent que notre méthode offre une meilleure qualité de reconstruction, en particulier dans des conditions difficiles avec des angles manquants et un bruit élevé. Les expériences ont été réalisées sur un fantôme de Shepp-Logan et sur des données réelles de microscopie électronique, démontrant ainsi le potentiel de notre approche pour des applications pratiques.
Younès MOUSSAOUI (Centrale/CHU Nantes): « Implicit Neural Representations for End-to-End PET Reconstruction«
Implicit neural representations (INRs) have demonstrated strong capabilities in various medical imaging tasks, such as denoising, registration, and segmentation, by representing images as continuous functions, allowing complex details to be captured. For image reconstruction problems, INRs can also reduce artifacts typically introduced by conventional reconstruction algorithms. However, to the best of our knowledge, INRs have not been studied in the context of PET reconstruction. In this paper, we propose an unsupervised PET image reconstruction method based on the implicit SIREN neural network architecture using sinusoidal activation functions. Our method incorporates a forward projection model and a loss function adapted to perform PET image reconstruction directly from sinograms, without the need for large training datasets. The performance of the proposed approach was compared with that of conventional penalized likelihood methods and deep image prior (DIP) based reconstruction using brain phantom data and realistically simulated sinograms. The results show that the INR-based approach can reconstruct high-quality images with a simpler, more efficient model, offering improvements in PET image reconstruction, particularly in terms of contrast, activity recovery, and relative bias.
Cristiano Ulondu Mendes (Telecom Paris): « tomographie radar en milieu urbain«
La tomographie radar en milieu urbain consiste à séparer des réflecteurs situés à des hauteurs différentes mais vus dans un même pixel car situés à une distance similaire du radar. Les méthodes d’apprentissage profond proposées récemment pour résoudre cette tâche sont basées sur le déroulement d’algorithmes de poursuites de base avec contrainte de parcimonie. Ils dépendent d’une discrétisation des hauteurs et ne permettent pas un contrôle simple du nombre de réflecteurs détectés.
On présente dans cet article une approche alternative permettant d’estimer la position des cibles sur un intervalle continu. Notre approche s’inspire des itérations des algorithmes gloutons de reconstruction parcimonieuse tels que Matching Pursuit ou RELAX. Nous montrons des résultats de reconstruction satisfaisants sur des données simulées et sur une pile d’images satellitaires.
Maxime BOUTON (Centrale Lille) : « A distributed PnP sampler for high-dimensional inverse problems«
To solve inverse problems, Markov Chain Monte Carlo (MCMC) algorithms allow for the building of estimators and the quantification of their uncertainty, which is essential in the absence of calibration data. Inspired by recent optimization algorithms known as Plug-and-Play (PnP) methods, samplers such as PnP-ULA and PnP-SGS use neural network denoisers to encode rich prior information, enabling the construction of high-quality estimators. For very high-dimensional problems (beyond 106 parameters), such as in high-resolution imaging, implementing such samplers raises technical challenges related to data storage and computational cost. We propose a distributed ”Single Program Multiple Data” (SPMD) implementation of the PnP-ULA sampler deployed across multiple GPUs. By distributing a denoiser in the shape of an unrolled algorithm and encoded by a small number of parameters compared to DRUNet, our approach enables the application of PnP-ULA to very high-dimensional inverse problems. Computational efficiency and scalability are demonstrated for the deconvolution of large images (2048 × 2048).
Hubert Leterme (Unicaen): « Plug-and-play weak lensing mass mapping with fast uncertainty quantification«
In this talk, I will present a plug-and-play (PnP) approach for estimating the dark matter distribution from weak gravitational lensing data, using noisy shear measurements. Our method is designed to provide accurate and efficient mass maps without the need to retrain deep learning models for each new galaxy survey or sky region. Instead, a single model is trained on simulated mass maps corrupted by Gaussian white noise. We show that a well-chosen data fidelity term accelerates convergence to the algorithm’s fixed point. Additionally, we adapt a fast uncertainty quantification (UQ) method to the PnP framework. Unlike existing UQ approaches in this context, this method does not rely on posterior sampling, which is often computationally intensive. We benchmark our method against both model-driven and data-driven mass mapping techniques, and show that it achieves state-of-the-art reconstruction accuracy while producing smaller error bars, all with increased flexibility.
Romain Vo (ENS Lyon): « Comprehensive review of deep learning based methods for low-dose and sparse-view X-ray Computed Tomography«
In recent years, deep learning approaches to reconstruct X-ray computed tomography (CT) images have become widespread. In the meantime, it has become challenging for the practitioner to navigate the vast landscape of methods and draw a rapid conclusion on which approach to use given a specific problem or set of constraints. Similarly, the lack of common standards for benchmarking and implementing these methods makes it difficult to compare them. In this work, we present a comprehensive and quantitative review of deep learning-based CT reconstruction methods, focusing on the most recent and impactful works. We categorize the methods based on the strategy they use to address the ill-posedness of the CT reconstruction problem (post-processing, plug-and-play, learned regularization, unrolling, diffusion …), and not so much on the specific network architecture they use. We also provide a detailed comparison of the methods based on their performance and stability on benchmark datasets, namely we focus on low-dose CT and sparse-view CT applications in 2D. We also present a set of techniques from the literature to easily mitigate the complexity of unrolled techniques.
Léo Paillet (LAAS): « optimisation de systèmes hyperspectraux codés. L’une des applications est la reconstruction de cubes hyperspectraux à partir d’acquisitions faites par ces systèmes«
Coded hyperspectral reconstruction consists in reconstructing 3D hyperspectral cubes from 2D SD-CASSI coded acquisitions of these cubes. Deep learning has emerged as the go-to method in this field, allowing for impressive results and a small inference time. More recently, algorithm unrolling methods have been used in these algorithms, in order to take into account the image formation model to help the reconstruction process. Based on these networks, we simulated 2D coded acquisitions with a ray-tracing simulator rendering realistic SD-CASSI acquisitions, and used the forward model given implicitly by our ray-tracer to build a mapping function between the scene and the detector, representing the adjoint operator. This mapping function is then used in the unrolled algorithms to guide the reconstruction process with our realistic image formation model, instead of only relying on data. It allows for better results with prototypes, and the ability to consider a wider range of systems in coded hyperspectral reconstruction.
Alessandro Pasqui (College de France): « VertAX for effortless inverse vertex-based modeling«
Confluent tissues are commonly described as energy-based physical systems characterized by an internal optimization process, in which the energy function minimized by the system depends on microscopic parameters associated with each cell. Appropriate choices of these microscopic parameters give rise to a diverse expression of macroscopic behaviors, such as tissue softness or rigidity, or the formation of patterns. To solve the inverse problem of obtaining specific material properties, these systems can be associated with an external loss function dependent on the microscopic parameters explicitly and implicitly through the minimum energy state. Here we present VertAX, a novel framework written in JAX to quickly and easily address the inverse problem of tuning these microscopic cellular parameters to achieve a given target behavior for confluent tissues. VertAX enables users to effortlessly define any energy or cost function on a vertex-based mesh and seamlessly compute its derivatives with respect to any system parameter using automatic differentiation. This capability allows users to apply bilevel optimization techniques to determine optimal parameters, either to achieve a desired macroscopic behavior or to fit real microscopic data. Additionally, we benchmark equilibrium propagation as an alternative to automatic differentiation, demonstrating its potential for solving inverse problems in confluent tissues when used alongside non-differentiable frameworks.
Corentin Constanza (Creatis): « U-MAPEM: unrolled MAPEM for SPECT deep reconstruction »
Single Photon Emission Computed Tomography (SPECT) images are limited by their relatively low spatial resolution. This limitation significantly impacts quantitative studies, such as those for Targeted Radionuclide Therapy dosimetry. The primary factors contributing to this poor resolution are the collimator-detector response, attenuation, and photon scatter. Traditional SPECT image reconstruction relies on the Maximum Likelihood Expectation Maximization (MLEM) method and prior knowledge incorporation. However, this can introduce artifacts and necessitates careful tuning of priors and parameters to prevent biased reconstructions. We proposed an unrolled version of the MAPEM algorithm. We use a convolutional neural network (CNN) denoiser as a regularization term within the MAPEM iterative scheme.
Organisateurs
- Matthieu Kowalski (Inria Tau, Université Paris-Saclay, matthieu.kowalski@universite-paris-saclay.fr)
- Thomas Moreau (Inria Mind, thomas.moreau@inria.fr)