Réunion

Self-Supervised Visual Learning: Methods, Applications and Challenges

Annonce

Les inscriptions (en présentiel, en distanciel) effectuées cet été ont été prises en compte.

Seules les inscriptions en distanciel sont dorénavant possibles. Les dernières inscriptions en présentiel sont réservées aux personnes donnant une présentation

Les demandes de prise en charge de mission par le GdR IASIS sont à formuler avant le 30 septembre.

En raison des fermetures probables d’universités nous sommes à la recherche d’une nouvelle salle pouvant accueillir la journée (60 personnes en présentiel) à Paris. Si vous avez connaissance d’une telle salle merci de contacter les organisateurs.

Over the past six years, Self-Supervised Learning (SSL) has driven major advances in computer vision.
These methods have enabled new possibilities across a wide range of domains, including robotics, industrial applications, biological applications, and video generation. This workshop aims to bring together the regional community interested in SSL to review recent progress and discuss its current limitations. Despite impressive successes, SSL approaches remain highly data-hungry, often struggle to generalize to out-of-distribution tasks, and raise ethical concerns. To foster discussion around these challenges, the workshop will feature talks from established keynote speakers alongside presentations from junior researchers. The event offers an opportunity to identify promising research directions and encourage new collaborations within the community.

The topics include (not limited):

• Self-supervised learning
• Generative models
• Multi-modal models
• Industrial or interdisciplinary applications
• Embodied and active learning

Invited speakers (confirmed) 

• Diane Larlus (Naver labs)
• Jean Ponce (DIENS, ENS-PSL)
• Camille Couprie (Meta-FAIR)
• Rufin VanRullen (CerCo, CNRS)

Call for contributions : Extension

We welcome contributions from early-career researchers, including Bachelor/Master/Ph.D. students, postdocs, or recently promoted assistant professors. The titles and short abstracts (1/2 pages max) should be written in English and sent to gautier.claisse@uca.fr and arthur.aubret@irit.fr by September 8th. September 21st
The contribution can present recently published research, on-going research, or a research project. We will give confirmation of oral/poster presentation for each contribution by September 15th 25th. We aim to provide a friendly atmosphere for students to practice presenting their research in english. Registration is free but mandatory

Program

[9:00-9:30] Welcoming coffee
[9:30-9:45] Introduction to Self Supervised Learning
[9:45-10:30] Prof. Diane Larlus, Naver Labs
[10:30-10:45] Coffee break 
[10:45-11:30] Dr. Camille Couprie, Meta-FAIR
[11:30-11:55] Oral contribution 1 - Paul Rongieras, INRIA Rennes
[11:55-12:20] Oral contribution 2 - Brayan Monroy, Universidad Industrial de Santander, ENS, CNRS
[12:20-14:00] Lunch break + Poster session
[14:00-14:45] Prof. Jean Ponce, ENS-PSL
[14:45-15:30] Prof. Rufin VanRullen, CerCo, CNRS
[15:30-15:45] Coffee break
[15:45-16:10] Oral contribution 3 - Mohamad Jouni, FLORALIS, Gipsa-lab, Université Grenoble Alpes
[16:10-16:35] Oral contribution 4 - Keryan Chelouche, ETIS, CY Cergy Paris Université, ENSEA, CNRS
[16:30] Discussion

Organizers

• Arthur Aubret (IRIT)
• Gautier Claisse (Institut Pascal/Lapso)
• Mehdi Gharbage (Institut Pascal/Michelin)
• Céline Teulière (Institut Pascal)

Acknowledgment

This workshop day received financial support from the ANR MESMRise ANR-23-CE23-0021 project.

Abstracts of presentations

Diane Larlus: From Many Models to One: Multi-Teacher Distillation and Model Merging for Universal Visual Representations

Abstract: Self-supervised learning has produced a rich ecosystem of powerful pretrained vision models, each capturing different aspects of visual understanding. A natural question then arises: can we consolidate the complementary strengths of multiple pretrained models into a single, unified encoder? In this talk, I will present a line of work that addresses this challenge through two complementary strategies: multi-teacher distillation and model merging. First, I will briefly touch on how visual representations, whether trained with supervised or self-supervised objectives, generalize across concepts, motivating the need for richer, more universal encoders. I will then present UNIC, a multi-teacher distillation framework that transfers knowledge from several complementary pretrained models into a single student, yielding representations that match or surpass the best individual teacher. Next, I will discuss DUNE, which extends distillation to a heterogeneous setting spanning 2D and 3D visual perception tasks. Despite differences in training objectives and expected outputs, the resulting encoder successfully integrates these diverse capabilities. Finally, I will introduce a model merging approach where task-specific models derived from a shared backbone can be efficiently combined using task alignment as a lightweight hyperparameter selection proxy, avoiding costly retraining while preserving strong multi-task performance.

Jean Ponce: Dual Perspectives on Non-Contrastive Self-Supervised Learning

Abstract: The stop gradient and exponential moving average iterative procedures are commonly used in non-contrastive approaches to self-supervised learning to avoid representation collapse, with excellent performance in downstream applications in practice. I will discuss in this presentation these procedures from the dual viewpoints of optimization and dynamical systems, and show that, in general, although they do not optimize the original objective, or any other smooth function, they do avoid collapse. I will then show using a dynamical system perspective that, in the linear case, minimizing the original objective function without the use of a stop gradient or exponential moving average always leads to collapse. Conversely, I will give an explicit characterization of the equilibria of the dynamical systems associated with these two procedures in this linear setting as algebraic varieties in their parameter space, and show that they are, in general, asymptotically stable. After illustrating these theoretical findings with empirical experiments with real and synthetic data, I will conclude by presenting extensions of our results to nonlinear encoders and applications of world models to planning.

Joint work with Michael Arbel (Inria), Basile Terver (AMI Labs) and Martial Hebert (CMU).

Camille Couprie: Scaling self-supervised learning with DINOv3: from natural images to satellite and cell microscopy.

Abstract: Self-supervised learning is the most scalable route to general-purpose visual representations: it learns from raw images, needs no annotation, and transfers across domains. I will present DINOv3, which scales the DINO recipe to a 7B-parameter ViT trained on 1.7B curated images, and the ingredient that made this scale usable — Gram anchoring, an objective that prevents the degradation of patch-level features observed in long trainings. The resulting frozen backbone yields sharp dense features, strong results on dense prediction and tracking, and easy alignment with text. DINOv3 also adapts beyond natural images — to satellite imagery for canopy height mapping, and to scientific domains such as cell microscopy. I’ll close with FINO, which leverages freely available metadata as weak supervision to learn domain-adapted yet task-agnostic representations that are more robust to distribution shifts and surpass fully supervised and domain-specific methods.

Rufin VanRullen: To be defined.

Abstract: To be defined.

Paul Rongieras: Preventing Dimensional Collapse in Self-Supervised Learning through Maximum Mean Discrepancy

Abstract: Self-Supervised Learning (SSL) has become a leading paradigm for learning visual representations without labels, but it faces two persistent difficulties: the large diversity of independently designed methods relying on similar underlying ideas, and the risk of collapse, in which the encoder produces trivial or low-rank representations. This work addresses both issues through a unifying, distributional perspective. We first review the main SSL methods (contrastive and non-contrastive, e.g. SimCLR, Barlow Twins, VICReg, LeJEPA) through the Alignment and Uniformity framework, and show that the regularization term of a wide range of these losses can be rewritten as a Maximum Mean Discrepancy (MMD) between the embedding distribution and a reference distribution, either uniform on the hypersphere or an isotropic Gaussian. This reformulation
identifies, for each method, an implicit kernel, and clarifies which methods guarantee full moment-matching against collapse and which only constrain a finite number of moments. Building on this framework, we study dimensional collapse in a linear setting: a linear encoder trained with an MMD regularization term induced by a quadratic kernel. We derive closed-form, interpretable thresholds on the regularization strength required to avoid collapse, and relate them explicitly to the level of noise introduced by data augmentation and to the natural variability present in the data. These theoretical predictions are validated empirically, and we further show that both dimensional collapse and excessive regularization degrade downstream performance

Brayan Monroy: Learning to Recorrupt: Noise Distribution Agnostic Self-Supervised Image Denoising

Abstract: Self-supervised image denoising methods have traditionally relied on architectural constraints, pseudo-pair constructions, or specialized loss functions to avoid the trivial identity mapping. Among these, approaches such as Noisier2Noise or R2R create training pairs by adding synthetic noise to noisy images. While effective, these recorruption-based approaches require precise knowledge of the noise distribution, which is often unavailable. We present Learning to Recorrupt (L2R), a self-supervised framework that does not require exact knowledge of the noise distribution. Our method introduces a learnable recorruptor jointly optimized with the denoiser through a min-max saddle-point objective. The proposed method achieves state-of-the-art performance among methods without prior knowledge of the noise distribution across unconventional and heavy-tailed noise distributions, such as log-gamma and Laplace, as well as spatially correlated noise, while obtaining competitive results on real-world denoising benchmarks characterized by signal-dependent noise.

Mohamad Jouni: SHARP: Self-Supervised Hyperspectral Alignment and Plug-and-Play Reconstruction for RGB-Guided Super-Resolution

Abstract: RGB-guided hyperspectral super-resolution (HSR) aims to recover a high-spatial-resolution hyperspectral image (HSI) by combining the spectral information of a low-resolution HSI with the spatial detail of a high-resolution RGB observation. In real dual-camera systems, however, the two modalities are rarely pixel-wise aligned. Separate viewpoints and optics induce depth-dependent parallax, occlusions, and spatially varying misregistration, while spectral, radiometric, and resolution differences make cross-modal registration particularly challenging. Consequently, inaccurate alignment may transfer inconsistent or spurious RGB structures into the reconstructed HSI. This problem is further challenged in real snapshot acquisitions by hyperspectral sensor noise and the absence of clean high-resolution hyperspectral targets.
We introduce SHARP, a self-supervised framework that couples learned cross-modal alignment with Plug-and-Play hyperspectral reconstruction for real RGB-HSI acquisitions. Our approach combines cross-modal dense registration with iterative Plug-and-Play reconstruction and adapts the registration component without clean supervision. Rather than relying on clean high-resolution hyperspectral targets, the framework exploits two repeated noisy HSI measurements of the same static scene while sharing the same RGB guidance, inspired on Noise2Noise objective. We also investigate a further-recorruption strategy to generate two conditionally independent hyperspectral observations from a single noisy measurement, enabling self-supervised reconstruction when repeated acquisitions are unavailable. Experiments on real data acquired with our snapshot RGB-HSI dual-camera system, show that self-supervised adaptation can improve denoising and stabilize cross-modal registration without clean references. Our ongoing work focuses on integrating an explicit HSR forward model with self-supervised registration and reconstruction, and on studying robustness to noise, radiometric mismatch, and varying spectral configurations, beyond the commonly used 31-band [400-700] nm spectral coverage.

Keryan Chelouche: What Transfers to Radar? Self-Supervised Representations for Privacy-Preserving Fall Anticipation.

Abstract: Falls are a leading cause of injury-related death among older adults. Preventing them requires moving beyond fall detection toward anticipation: from gait deterioration indicating rising risk to the short-term instability preceding a fall. This requires continuous monitoring at home, where cameras and wearables are poorly accepted. Micro-Doppler radar is contactless, robust to lighting and occlusion, and privacy-preserving by construction. Labelled data, however, are scarce, particularly because the target population is one from which data cannot be freely collected. This makes self-supervision a particularly natural fit for the modality. Yet large-scale self-supervised pretraining remains largely unexplored for micro-Doppler, while the field has so far imported only supervised vision encoders. Learning self-supervised representations on micro-Doppler itself is the objective of this project; it begins by asking how far representations learned on other modalities carry into this signal, and where they stop. A first study treats the time–frequency signature as an image and adapts DINOv3 with low-rank updates, under a subject-disjoint protocol with three deployment shifts. Natural-image features transfer remarkably well: with parameter-efficient adaptation they hold up even under a compound change of radar, environment and population. The choice of pretraining matters, and with the adaptation held fixed, replacing a supervised backbone by a self-supervised one gains 4.7 points when the sensing geometry changes and 8.2 under that compound shift. A second observation complicates the picture: audio encoders, pretrained on the same kind of time–frequency input, do not transfer better. In preliminary experiments the strongest of them, at matched size, only matches frozen DINOv3 in-distribution (75.1 vs 74.9) and, once both are adapted with low-rank updates, falls well behind it under sensing-geometry shift (69.2 vs 81.3). Fall anticipation is intrinsically temporal, however. The next stage therefore moves to self-supervised pretraining on unlabelled micro-Doppler sequences, borrowing predictive objectives and architectures from video. In particular, JEPA-style future-latent prediction offers a natural way to align the pretraining objective with the forecasting structure of fall anticipation.

Posters by

• Peyman Adibi,
• Abdelmalek Belghorami,
• Enzo Bonneau
• Pierre Falconnier,
• Yassine Fekif-hassen,
• Kenza Kitout.