The whole PhD subject, including equations and figures, can be found at: https://partage.imt.fr/index.php/s/GQcfXmjY58Zkbbq
1. Context
Artificial intelligence (AI) and deep learning approaches have produced impressive results on many problems of computer vision and image processing. The field of remote sensing, which exploits satellite or aerial images for Earth observation, also benefits from the efficiency of these approaches. This has a strong societal impact, as these methods are applied to crucial issues such as environment monitoring (e.g. urban monitoring, deforestation, crop and agriculture monitoring).
Nevertheless, one of the issue with deep learning methods is their lack of interpretability. This is an important matter in remote sensing, since the images are often ultimately used for strategic purposes. To circumvent this issue in the more specific context of hyperspectral remote sensing images, this project aims at combining deep neural networks and classical interpretable iterative optimization algorithms by building on the so-called plug-and-play paradigm.
2. Hyperspectral imaging and unmixing
In contrast to classical Red-Green-Blue (RGB) images, which possess only three color channels, HyperSpectral Images (HSIs) are acquired in numerous spectral bands (typically 200), each of them acquiring the scene in a very narrow wavelength interval. This rich spectral information enables in principle to identify the materials (water, sand, vegetation…) present in the scene. However, HSIs suffer from a low spatial resolution. Consequently, each pixel of HSIs corresponds to a large area in the scene (typically 30 × 30 m for satellite images) and therefore contains several materials. The spectrum acquired in each pixel is thus a mixture of the spectra of the individual materials, which prohibits a direct identification of the materials by a mere comparison with a spectrum database. This calls for hyperspectral unmixing (HSU) methods [1], enabling to recover both the spectra of the individual materials present in the scene (the so-called endmembers) and their relative concentration in each pixel (the abundances).
Nonetheless, finding both endmembers and abundances from the sole knowledge of the considered HSI image unfortunately admits an infinite amount of (potentially spurious) solutions. To solve this severely ill-posed posed problem, many methods of the literature write HSU as an optimization problem in which some regularization (priors) are imposed over the abundances and endmembers. The optimization problem is then solved by iterative solvers.
3. Research axes
Over the last decade, different handcrafted regularization terms have been proposed. Nevertheless, such terms are often too simplistic to model the complex nature of the considered images. Therefore, Plug-and-Play (PnP) methods [3] rather propose to replace them, within iterative optimization algorithms, by an off-the-shelf denoiser, which can potentially be data-driven and thus more sophisticated.
During this PhD, and in contrast to existing algorithms, one of our priorities will be to propose new PnP schemes with mathematical convergence guarantees. Indeed, depending on both the choice of denoiser and the considered optimization algorithm, the considered PnP algorithm may have no guarantee of convergence and might even diverge. While convergence guarantees of PnP have been explored when only a single factor (e.g. S) is learnt [3], the problem of estimating simultaneously both endmembers and abundances remains largely open.
Additionally, beyond the theoretical aspects, applying PnP schemes to HSU rises practical issues. In particular, it calls for dedicated-to-hyperspectral denoisers, which in turns requires proper datasets for their training (unfortunately unavailable in practice). In consequence, we plan to resort to the use of synthetic dataset training, which has not been attempted so far in HSU.
The last part of this PhD will focus on going further the linear mixture model to take into account non-linear light interactions in real-world HSIs.
4. Candidates
The candidate must have completed a Master 2 (or equivalent) with a strong knowledge of signal/image processing and mathematics (in particular, optimization). Alternatively, we might also consider students actually enrolled in a Master 2, who could first be hired for an internship prior to starting the PhD. Basic machine learning knowledge is at least required. Ideally, the candidate will be familiar with the Python language (and in particular with Pytorch). Knowledge in hyperspectral imaging is a plus.
The candidate will acquire an expertise in remote sensing, deep learning, inverse problems and plug-and-play methods. Such skills are largely valuable, both in academia and in private companies, and are transferable to many other applications such as biomedical imaging, astrophysics, etc.
5. Contact
The PhD (3 years) will take place in the IMAGES team (T ́el ́ecom Paris) under the supervision of Christophe Kervazo and Arthur Leclaire.
— Contact : christophe.kervazo@telecom-paris.fr ; arthur.leclaire@telecom-paris.fr ;
— More informations on : https://sites.google.com/view/christophekervazo/
References
[1] Bioucas-Dias, J. M., Plaza, A., Dobigeon, N., Parente, M., Du, Q., Gader, P., and Chanussot, J. Hyperspectral unmixing overview : Geometrical, statistical, and sparse regression-based approaches. IEEE journal of selected topics in applied earth observations and remote sensing 5, 2 (2012), 354–379.
[2] Bolte, J., Sabach, S., and Teboulle, M. Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Mathematical Programming 146, 1-2 (2014), 459–494.
[3] Hurault, S., Leclaire, A., and Papadakis, N. Gradient step denoiser for convergent plug-and-play. In International Conference on Learning Representations (2021).
[4] Zhao, M., Chen, J., and Dobigeon, N. Ae-red : A hyperspectral unmixing framework powered by deep autoencoder and regularization by denoising. arXiv preprint arXiv :2307.00269 (2023).
[5] Zibulevsky, M., and Pearlmutter, B. A. Blind source separation by sparse decomposition in a signal dictionary. Neural Computation 13, 4 (2001), 863–882.
