Appel Choose France
L’appel Choose France est ouvert jusqu’au 31 mars. C’est une belle opportunité pour attirer en France des talents...
Annonce
stage Master 2 : Discovering new objects for visual localization
27 Novembre 2023
Catégorie : Stagiaire
Title: Discovering new objects for visual localization
Location: LORIA/INRIA, 615 rue du jardin botanique, 54600 Villers les Nancy
Research team : TANGRAM https://team.inria.fr/tangram
Supervisor: Marie-Odile Berger Gilles Simon
Expected starting date: March or April 2023
Duration: 5 to 6 months
Context :Recent work by the TANGRAM team has demonstrated the value of using object detection to aid visual relocation in a SLAM (Simultaneous Localisation And Mapping) context. The method, called OA-SLAM for object-assisted SLAM, is described in [1] and a video showing the system in use is available at https://www.youtube.com/watch?v=L1HEL4kLJ3g. However, this work is limited by the fact that the detectable objects must be known to the detector, i.e. the vocabulary used (defined by the object categories) is closed. The aim of this project is to extend the method to discover new objects in a variety of environments. For example, a factory may have many objects (valves, sensors, etc.) that are not recognised by the usual networks such as YOLO, which prevents the current localisation method from working.
Objectives : The aim of this internship is therefore to investigate both recent advances in image segmentation (Segment Anything -- SA [2]) and methods for extending a vocabulary based on proposals for boxes likely to contain objects [3,4].
Compared to existing work, the proposed method will be able to take advantage of (i) the possibility of reconstructing objects or specific points on objects via SLAM to reduce the number of tracking hypotheses, (ii) the fact that the OA-SLAM method is based on an approximate modelling of objects by ellipsoids and therefore does not require perfectly delimited segmentations.
Finally, we will study to what extent the labelling obtained automatically using the proposed method allows new learning guided by the model [5].
Qualifications :Currently enrolled in a Master's (M2) program or in the final year of a 5-year Engineering degree in electrical engineering, computer sciences, applied maths or a related field. Preliminary skills in computer vision and machine learning will be appreciated.
How to Apply: Interested candidates are invited to submit the following application materials to marie-odile.berger@inria.fr and gilles.simon@loria.fr :Curriculum Vitae (CV), Motivation Letter, andAcademic Transcripts. The position will remain open until filled.
References
[1] M. Zins, G. Simon, M.-O. Berger. OA-SLAM: Leveraging Objects for Camera Relocalization in Visual SLAM. ISMAR 2022
[2] A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A.C. Berg, W.-Y. Lo, P. Dollar, R. Girshick. Segment Anything. Proceedings of the IEEE/CVFInternational Conference on Computer Vision (ICCV), 2023, pp. 4015-4026.
[3] A. Osep, W. Mehner, P. Voigtlaender, B. Leibe. Track, Then Decide: Category-Agnostic Vision-Based Multi-Object Tracking. ICRA, 2018.
[4] Y. Du, Y. Xiao, V. Lepetit. Learning to Better Segment Objects from Unseen Classes withUnlabeled Videos. IEEE/CVF International Conference on Computer Vision (ICCV), 2021
[5] M. Zins, G. Simon, M.-O. Berger. Object-Based Visual Camera Pose Estimation From Ellip-soidal Model and 3D-Aware Ellipse Prediction. International Journal of Computer Vision, 2022,130, pp. 1107-1126.