Keywords : vision foundation models, physics-grounding, computer vision, scene understanding
Supervisors : Raoul de Charette (Inria, Paris) and Tuan-Hung Vu (Inria / Valeo.ai, Paris)
Description:
The objective of the PhD is to improve the explicit understanding of physics in Vision Foundation Models (VFM). While the latter are typically trained on reconstruction objectives (e.g., future or masked patch reconstruction), they have shown some emergence of physics but still fail at accurately modeling basic physics. In this context, the goal is to enable physics-grounded VFM capable of understanding Newtonian dynamics from real-world videos, for example being able to predict pixel-wise rigid body dynamics (e.g., gravity, forces, motion, etc.) that model the macro interactions between objects in the scene. The resulting physics-grounded VFM are expected to showcase better capability at downstream applications – especially those relying on implicit physics notions such as forecasting, motion estimation, scene parsing, etc.
More details : https://astra-vision.github.io/assets/pdf/inria_prairie_phd-physics-vfm.pdf
Hosting institution
The candidate will join Inria Paris, a dynamic and internationally-renowned Inria Paris centre which is established as a scientific and technological leader.
They will be part of the Astra project-team and will work in the Astra-Vision group (https://astra-vision.github.io) addressing robust visual scene understanding. The group research appears in all top-tier venues (CVPR, ICCV, ECCV, etc.) and the group comits to produce open source research.
Application : Apply asap and at the latest May 20th 2025 (included). Instructions for applications are in the PDF.
