Réunion

Abstract: The route from machine learning to artificial intelligence remains uncharted. The goal of this talk is to investigate how much progress is possible by framing machine learning beyond learning correlations: that is, by uncovering and leveraging causal relations. To this end, we will first identify multiple failure cases in modern machine learning pipelines and try to understand such failures as instances of mistaking correlation by causation. If convinced, we will explore two different ways to reveal causation from data, with some preliminary results. I hope to motivate further research by relating how advances in understanding causation from data would allow machines to ignore confounding effects and spurious correlations, generalize across distributions, leverage structure to reason, design efficient interventions, benefit from compositionality, and build causal models of the world in an unsupervised way.

Bio: David Lopez-Paz is a research scientist at Facebook AI Research, where he studies how to leverage principles from causality to transition from machine learning to artificial intelligence. Pior to that, David completed his PhD at the Max Planck Institute for Intelligent Systems and the University of Cambridge, advised by Bernhard Schölkopf and Zoubin Ghahramani. His list of publications is available at https://lopezpaz.org.

== 11h Abdellatif Zaidi, "Variational Representation Learning"

Auteurs : Abdellatif Zaidin (Université Paris-Est, Huawei), Inaki Estella Aguerri (Huwai)

== 11h25 Fabien Baradel, "Object Level Visual Reasoning in Videos"

Auteurs : Fabien Baradel (LIRIS, INSA-Lyon), Natalia Neverova (Facebook AI Research), Christian Wolf (LIRIS, INSA-Lyon, INRIA-Chroma, CITI), Julien Mille (LIFAT, INSA-CVL), Greg Mori (Simon Fraser University, Vancouver, Canada)

== 11h50 Stéphane Herbin, "Semantic bottleneck for computer vision tasks"

Auteurs: Maxime Bucher (ONERA/DTIS, Université de Caen), Stéphane Herbin (ONERA/DTIS), Frédéric Jurie (Université de Caen)

12h15 - 13h30 Pause midi

== 13h30 Keynote : Zied Bouraouin (Centre de Recherche en Informatique / CRIL),"Inductive Reasoning with Conceptual Space Representations"

Considerable attention has recently been devoted to the problem of automatically extending knowledge bases by applying some form ofinductive reasoning. In this talk, I will present some methods forknowledge base completion. At the centre of this work are conceptualspaces where objects from a domain of interest are represented aspoints in a metric space, and concepts are modelled as convex regions.I will first present how to learn conceptual space representationsfrom data, and then introduce some inductive reasoning techniques thatuse conceptual spaces together with an efficient Bayesian inferencemachinery that allows us to find plausible missing facts and rulesfrom a given knowledge base.

== 14h30 C. Hudelot, "Apport des logiques de description pour le raisonnement spatial dans le domaine de l'image"

Auteurs: Marc Aiguier (CentraleSupelec, MICS), Jamal Atif (Université Paris Dauphine, Laboratoire MICS), Isabelle Bloch (Télécom ParisTech, LTCI) et Céline Hudelot (CentraleSupélec, Laboratoire MICS)

== 14h55 Régis Pierrard, "Apprentissage de contraintes floues pour l'annotation automatique d'images justifiée"

Auteurs: Régis Pierrard (CEA), Céline Hudelot (CentraleSupélec), Jean-Philippe Poli (CEA)

15h20 - 15h50 Pause café

== 15h50 Hedi Ben-younes, "Fusion multimodale et raisonnement pour le VQA"

== 16h15 Martin Engilberge, "Deep Inside Visual-Semantic Embeddings"

Auteurs: Martin Engilberge (Technicolor, Sorbonne Université), Louis Chevallier (Technicolor), Patrick Pérez (Valeo), Matthieu Cord (Sorbonne Université)

==16h40 Yannick Le Cacheux, "Adaptation d'un modèle de zero-shot learning à un contexte de "generalized" zero-shot learning"

Auteurs: Yannick Le Cacheux (CEA LIST et CEDRIC Lab - CNAM), Hervé Le Borgne (CEA LIST), Michel Crucianu (CEDRIC Lab - CNAM)

17h05 Fin de la journée

RESUMES DES PRESENTATIONS

== Keynote : David Lopez-Paz (Facebook AI Research), "Causal Challenges in AI"

Abstract: The route from machine learning to artificial intelligence remains uncharted. The goal of this talk is to investigate how much progress is possible by framing machine learning beyond learning correlations: that is, by uncovering and leveraging causal relations. To this end, we will first identify multiple failure cases in modern machine learning pipelines and try to understand such failures as instances of mistaking correlation by causation. If convinced, we will explore two different ways to reveal causation from data, with some preliminary results. I hope to motivate further research by relating how advances in understanding causation from data would allow machines to ignore confounding effects and spurious correlations, generalize across distributions, leverage structure to reason, design efficient interventions, benefit from compositionality, and build causal models of the world in an unsupervised way.

Bio: David Lopez-Paz is a research scientist at Facebook AI Research, where he studies how to leverage principles from causality to transition from machine learning to artificial intelligence. Pior to that, David completed his PhD at the Max Planck Institute for Intelligent Systems and the University of Cambridge, advised by Bernhard Schölkopf and Zoubin Ghahramani. His list of publications is available at https://lopezpaz.org.

=== Abdellatif Zaidi, Université Paris-Est, "Variational Representation Learning"

We connect the information flow in a neural network to sufficient statistics; and show how techniques that are rooted in information theory, such as the source-coding based information bottleneck method can lead to improved architectures, as well as a better understanding of the theoretical foundation of neural networks, viewed as a cascade compression network. We illustrate our results and view through some numerical examples.

=== Fabien Baradel, "Object Level Visual Reasoning in Videos"

https://arxiv.org/abs/1806.06157

Human activity recognition is typically addressed by detecting key concepts like global and local motion, features related to object classes present in the scene, as well as features related to the global context. The next open challenges in activity recognition require a level of understanding that pushes beyond this and call for models with capabilities for fine distinction and detailed comprehension of interactions between actors and objects in a scene. We propose a model capable of learning to reason about semantically meaningful spatiotemporal interactions in videos. The key to our approach is a choice of performing this reasoning at the object level through the integration of state of the art object detection networks. This allows the model to learn detailed spatial interactions that exist at a semantic, object-interaction relevant level. We evaluate our method on three standard datasets (Twenty-BN Something-Something, VLOG and EPIC Kitchens) and achieve state of the art results on all of them. Finally, we show visualizations of the interactions learned by the model, which illustrate object classes and their interactions corresponding to different activity classes.

==== Stéphane Herbin, "Semantic bottleneck for computer vision tasks"

We introduce a novel method for the representation of images

that is semantic by nature, addressing the question of computation intelligibility in computer vision tasks. More specifically, our proposition is to introduce what we call a semantic bottleneck in the processing pipeline, which is a crossing point in which the representation of the image is entirely expressed with natural language, while retaining the efficiency of numerical representations. We show that our approach is able to generate semantic representations that give state-of-the-art results on semantic content-based image retrieval and also perform very well on image classification tasks. Intelligibility is evaluated through user centered experiments for failure detection.

== Keynote : Zied Bouraoui (Centre de Recherche en Informatique / CRIL)

Considerable attention has recently been devoted to the problem of automatically extending knowledge bases by applying some form of
inductive reasoning. In this talk, I will present some methods for
knowledge base completion. At the centre of this work are conceptual
spaces where objects from a domain of interest are represented as
points in a metric space, and concepts are modelled as convex regions.
I will first present how to learn conceptual space representations
from data, and then introduce some inductive reasoning techniques that
use conceptual spaces together with an efficient Bayesian inference
machinery that allows us to find plausible missing facts and rules
from a given knowledge base.

== C. Hudelot, "Apport des logiques de description pour le raisonnement spatial dans le domaine de l'image"

Dans le domaine de l?interpre?tation d?images, les relations spatiales jouent un ro?le important dans la description et la reconnaissance des objets : elles permettent, en effet, de lever l?ambigui?te? entre des objets d?apparence similaire et sont souvent plus stables que les caracte?ristiques des objets eux-me?mes. Nous avons donc propose? une ontologie ge?ne?rique de relations spatiales et nous avons montre? comment enrichir cette ontologie, par des repre?sentations floues des relations spatiales, afin d?en pre?ciser la se?mantique et de faire le lien entre les concepts spatiaux, souvent exprime?s sous forme linguistique, et les informations que l?on peut extraire des images, contribuant ainsi a? re?duire le fosse? se?mantique [Hudelot et al, 2008].

Nous avons ensuite propose? d?e?tendre ce formalisme pour le doter de capacite?s de repre?sentation et de raisonnement plus puissantes.

Nous avons propose? d?utiliser la morphologie mathe?matique, combine?e aux logiques de description et a? la logique floue, pour de?terminer de manie?re automatique les repre?sentations concre?tes des concepts spatiaux. Les ope?rateurs de morphologie mathe?matique, gra?ce a? leurs fondements alge?briques reposant sur la the?orie des treillis, deviennent ainsi partie inte?grante de la logique de description en tant que pre?dicats. Ces deux formalismes n?avaient pas e?te? combine?s jusqu?alors et cette combinaison permet d?une part de repre?senter de manie?re unifie?e des concepts vagues et incertains, et d?autre part de fournir des outils pour un nouveau raisonnement spatial qualitatif et quantitatif [Hudelot et al, 2014].

Nous avons aussi e?tudie? l?inte?gration de raisonnements non-monotones, tels que l?abduction, dans les logiques de description [Atif et al, 2014]. En particulier, gra?ce au cadre alge?brique de la morphologie mathe?matique et a? l?analyse formelle de concepts, nous avons propose? un nouveau formalisme permettant de mode?liser le proble?me de l?interpre?tation d?images, guide?e par un mode?le, comme un proble?me d?abduction. En effet, les informations extraites de l?image (re?sultant d?algorithmes de traitement et de segmentation d?images) peuvent e?tre vues comme des observations et l?interpre?tation revient a? trouver la meilleure explication, e?tant donne?e une connaissance a priori sur le domaine d?application, a? la manifestation de ces observations.

== Régis Pierrard, "Apprentissage de contraintes floues pour l'annotation automatique d'images justifiée"

Les récentes avancées en intelligence artificielle et les interactions de plus en plus fréquentes entre humains et IA soulèvent des questions de transparence, d'explicabilité et de confiance. Dans le cadre de nos travaux, nous proposons une approche dont le but est de résoudre un problème d'annotation ou de classification d'images tout en fournissant à l'utilisateur une explication à la solution fournie. Sur un ensemble d'images, cette approche s'appuie sur l'extraction de relations pertinentes entre entités (objets, personnes,...). Il s'agit de relations floues afin de tirer profit de la théorie des ensembles flous pour gérer l?imprécision des données (prétraitement et segmentation des images) ainsi que celle du langage (génération des explications).

Étant donné un ensemble d'images d?entraînement, les relations les plus pertinentes sont extraites afin d'apprendre un ensemble de règles ou de contraintes nous permettant de résoudre le problème. Nous nous intéressons ici à deux problèmes en particulier: classification et annotation d'images.

Un exemple d'annotation automatique justifiée d'organes sur un ensemble d'images médicales sera présenté. Il repose notamment sur la résolution d'un problème de satisfaction de contraintes floues qui ont été apprises sur un ensemble d'images.

== Hedi Ben-younes, "Fusion multimodale et raisonnement pour le VQA"

Nous commencerons par rappeler le contexte du VQA, en présentant la tâche et problèmes qu'elle soulève. Nous parlerons d'abord de la fusion multimodale: comment faire interagir les deux sources d'information que sont l'image et la question posée à son sujet, dans un contexte d'apprentissage supervisé ? Nous verrons que les modèles de fusion bilinéaire constituent une solution efficace à ce problème. Enfin, nous présenterons l'état de l'art ainsi que nos travaux en cours sur les architectures de raisonnement visuel guidé par la question, qui prennent en compte la structure spatiale de l'image et permettent d'améliorer significativement les performances des systèmes.

== Martin Engilberge, "Deep Inside Visual-Semantic Embeddings"

Jointly aligning multiple modality in a single embedding space is at the core of many research problems. In this presentation, we introduce a deep network to learn a cross-

modal mapping between images and texts. It is based on two-path neural network combining a visual path that leverages recent space-aware pooling mechanisms with a textual path.

Jointly trained from scratch, our semantic-visual embedding offers a versatile model. Once trained under the supervision of captioned images, it yields new state-of-the art performance on cross-modal retrieval. It also allows the localization of new concepts from the embedding space into any input image.

== Yannick Le Cacheux, ?Adaptation d'un modèle de zero-shot learning à un contexte de "generalized" zero-shot learning?