Location: Ensta, Palaiseau, France
Profile required: M2 student or engineer-student
Duration: 5-6 months
1 Motivation
The fields of activity recognition and sequential event-based decision making allow multiple applications
ranging from monitoring and assisting in smart environments to enhance user convenience and safety,
to automating cybersecurity protocols for real-time threat detection and response, and applying con-
trol in industrial settings to improve operational efficiency. However, the development of robust and
reliable models for these domains has been challenged by the inherent complexity of their associated
environments and the limited availability of suitable test-beds for evaluation. Furthermore, in critical
environments, it is crucial to guarantee functionality and trust. This internship builds upon clear rea-
soning and interpretability so as to ensure that a system’s behavior can be understood and trusted by
operators and regulators, promoting reliability and compliance.
2 Summary
Understanding external events to interpret the behaviour of humans is crucial for smart environments
to provide automated services such as automated power management in smart homes [1] or protection
against threats in secure buildings. These services are related to the fields of activity recognition and
sequential event-based decision making, which identify and interpret behaviors as they occur and the
optimization of subsequent choices of reaction.
While recent artificial intelligence algorithms seem to have improved the performance of these systems,
their robustness and reliability, as well as the guarantee of functionality and trust still needs to be
improved. Our approach combines reinforcement learning of data-efficient learning and rule-based meth-
ods for better explainability. For the latter, this project focuses on temporal-logic methods for better
understanding and reasoning of complex sequence models.
The intern will implement algorithms for event-based environments, with a focus on capturing and
managing complex historical dependencies between events and actions. The intern will work on testing
and improving these models using real smart home data, where a range of activities and events are
monitored over time.
3 Internship Objectives
The goal of this internship is to explore the application of interpretable temporal-logic-based algorithms
on real data. Specifically, it will involve using algorithms based on Allen’s Temporal Interval Algebra
[2] to learn interpretable rules, which enable the recognition of activities and the selection of actions in
a structured manner. We will work on 2 application cases : recognition of activity in smart homes with
the Smart Home dataset collected by the team in a living lab [3], and data generated by a simulator for sequential event-based decision making, designed to facilitate Reinforcement Learning experiments
in simulated event-based environments, OpenTheChests [4].
For our comparison, we aim to focus on three categories of algorithms: First, algorithms that integrate
temporal logic rule learning with reinforcement learning, inspired by approaches such as those in [5].
Next, algorithms designed for Partially Observable Markov Decision Processes (POMDPs), such as
POMCP [6], which excel in handling uncertainty and partial observability in dynamic environments.
Finally, methods incorporating Transofrmer architectures, such as Deep Q-Networks (DTQN) [7], which
are particularly adept at capturing long-term dependencies and modeling complex temporal relationships
in sequential data.
The intern will be responsible for evaluating these algorithms and assessing their performance across
various metrics. The objective is to highlight the advantages of interpretable algorithms, especially
in settings where transparency and trustworthiness are paramount, and to understand setbacks such
as potential limitations in the flexibility or scalability of these interpretable models when applied to
complex, real-world scenarios.
4 Context
Principal Investigator (PI): Sao Mai Nguyen is an Associate Professor at the U2IS laboratory at ENSTA
Paris, part of IP Paris, a world-class Institute of science and technology encompassing 5 prestigious
´
French engineering Schools:
Ecole Polytechnique, ENSTA Paris, ENSAE Paris, T´el´ecom Paris, T´el´ecom
SudParis. ENSTA Paris research laboratory “Computer Science and System Engineering Department”
(U2IS) studies mobile robot navigation, perception, embedded vision, machine learning and human-robot
interaction. Its emphasis is on applying machine learning to real-world applications, such as assistive and
service robotics, intelligent vehicles, defence and security. Sao Mai Nguyen specialises in robot learning,
active learning, imitation learning and reinforcement learning with applications in assistive technologies.
Co-Investigators :
- David Filliat, Autonomous Systems and Robotics team, Computer Science and Systems Engineer-
ing Laboratory (U2IS), ENSTA. - Ivelina Stoyanova, Thales.
To apply, please send to Sao Mai Nguyen (sao-mai.nguyen@ensta-paris.fr) an email with the subject
containing the tag [ Temporal Rules intership ]: your CV, a cover letter and a link to a git repository of
an example code project.
References
[1]Damien Bouchabou et al. “A survey of human activity recognition in smart homes based on IoT
sensors algorithms: Taxonomies, challenges, and opportunities with deep learning”. In: Sensors 21.18
(2021), p. 6037.
[2]James F Allen and George Ferguson. “Actions and events in interval temporal logic”. In: Journal
of logic and computation 4.5 (1994), pp. 531–579.
[3] Damien Bouchabou et al. “A Smart Home Digital Twin to Support the Recognition of Activities of
Daily Living”. In: Sensors 23.17 (Sept. 2023), p. 7586.
[4] Ivelina Stoyanova et al. “Open the Chests: An environment for Activity Recognition and Sequential
Decision Problems Using Temporal Logic”. In: (2024).
[5] Chengzhi Cao et al. “Discovering intrinsic spatial-temporal logic rules to explain human actions”.
In: Advances in Neural Information Processing Systems 36 (2024).
[6] David Silver and Joel Veness. “Monte-Carlo planning in large POMDPs”. In: Advances in neural
information processing systems 23 (2010).
[7] Kevin Esslinger, Robert Platt, and Christopher Amato. “Deep transformer q-networks for partially
observable reinforcement learning”. In: arXiv preprint arXiv:2206.01078 (2022).
[8] Damien Bouchabou and Sao Mai Nguyen. “Generative Pretrained Embedding and Hierarchical
Irregular Time Series Representation for Daily Living Activity Recognition”. In: IOS Press, Oct.
2024, pp. 4764–4771.
