Application: Send motivation letter, CV, track record and recommendations by email at jean-baptiste.thomas@u-bourgogne.fr AND Abdelhamid-Nour-Eddine.Fsian@u-bourgogne.fr before the 15th of January 2025.
Context: This project takes place at Lab IMVIA (https://imvia.u-bourgogne.fr/) at Université de Bourgogne. The Spectral Filter Arrays camera technology was developed over a decade at IMVIA, and recent research works open new perspectives in AI-based image reconstruction methods. This internship is funded for 5 months, during Spring 2025. Starting date is upon discussion between early February and late March.
Supervision: Abdelhamid Fsian, Jean-Baptiste Thomas, and Pierre Gouton
Summary: Spectral Filter Array (SFA) [1] cameras offer a promising, low-cost alternative to RGB cameras by capturing more spectral bands, though they suffer from inherently low spatial resolution. This project investigates the application of diffusion models [2,5], a recent class of generative models that iteratively refine noisy inputs into high-quality outputs by learning the underlying data distribution, to improve the spatial resolution of spectral images acquired with an SFA camera. To address the challenge of limited spectral datasets, we propose to use RGB2Spectral models [3] to reconstruct synthetic spectral data from RGB images, which will serve as training resources for a new super-resolution (SR) model based on diffusion models [5]. The goal is to enhance spatial resolution while preserving spectral fidelity. We will assess the effectiveness of synthetic data in training the SR model [4], and compare its performance against state-of-the-art models, using both synthetic data and real spectral data. This approach aims to provide valuable insights into the use of synthetic data for advancing the super-resolution of spectral images.
Activities: The project will begin with a literature review to explore existing techniques in spectral SR, diffusion models [5], and RGB2Spectral techniques [4]. Based on this, suitable architectures will be selected, and datasets will be prepared, including synthetic spectral data generated from RGB images. The SR diffusion model will be trained using both real and synthetic data, with performance evaluated using standard metrics. Comparative analysis will be conducted to assess the efficacy of synthetic data in improving spectral SR. Finally, findings will be disseminated through scientific publication.
References:
[1] Lapray, P. J., Wang, X., Thomas, J. B., & Gouton, P. (2014). Multispectral filter arrays: Recent advances and practical implementation. Sensors, 14(11), 21626-21659.
[2] Wang, Y., Yang, W., Chen, X., Wang, Y., Guo, L., Chau, L. P., … & Wen, B. (2024). SinSR: diffusion-based image super-resolution in a single step. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 25796-25805).
[3] Cai, Y., Lin, J., Lin, Z., Wang, H., Zhang, Y., Pfister, H., … & Van Gool, L. (2022). Mst++: Multi-stage spectral-wise transformer for efficient spectral reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 745-755).
[4] Fsian, A. N., Thomas, J. B., Hardeberg, J. Y., & Gouton, P. (2024). Spectral Reconstruction from RGB Imagery: A Potential Option for Infinite Spectral Data?. Sensors, 24(11), 3666.
[5] Moser, B. B., Shanbhag, A. S., Raue, F., Frolov, S., Palacio, S., & Dengel, A. (2024). Diffusion models, image super-resolution, and everything: A survey. IEEE Transactions on Neural Networks and Learning Systems.
