Lab : Center for Robotics
Place : Ecole des Mines, 60 Bvd Saint Michel, Paris Duration and dates : 6 months , april/september 2025 Funded by : Projet ReSource PIA4
Supervisor: Alina Glushkova
Keywords: Human motion data, time series, machine learning, feature selection, hybrid model
Context: The progress of AI and data science contributes to the development of new methods and tools for capturing/analyzing/modeling and evaluating human motion in diQerent contexts and applications such as sports, rehabilitation etc. Data analysis and decision support tools for coaches/therapists are becoming increasingly popular. Crafts could benefit from this since it has common characteristics with the aforementioned domains: over the years the craftsman “trains”, he/she mobilizes complex cognitive processes and learning mechanisms, he/she makes mistakes and learns from them by reducing the variability of these errors. Motion capture technologies make it possible to capture digital signals and transform gestures into data, which encodes the tacit knowledge of experts. This data can be used to analyze/model/evaluate the quality of performance, estimate error gradients by using machine/deep learning methods.
However, machine-interpretable data is often not the same as human-interpretable data. Creating a hybrid approach, which would combine automatic preselection of parameters to be analyzed with expert feedback, would benefit from both data science and human expertise, contributing to the acceptability of the system.
Our teams has been working for years on the study of craft professions. Several scenarios were captured “in the wild” with multimodal technologies (egocentric, exocentric video, inertial sensors, contact microphone and stereo) contributing to the creation of the AIMove database.
Goals: The intern will develop a hybrid approach, combining the automatic evaluation of motor performance, using machine learning with an evaluation based on a set of rules defined by the human expert.
- Dynamic selection of features to be evaluated for motor performance analysis: To achieve this first goal the student will use the data (time series representing the angulations/rotations of the subject’s joints) and identify, using appropriate methods, the salient movement features to generate a specific analysis by profession. Reinforcement learning, the « forward feature selection » method or other discriminative deep learning methods such as eXtreme Gradient Boosting make it possible to dynamically select the set of optimal features of multidimensional data.
- Prediction of motor performance quality: Then it is necessary to predict the quality of performance, by estimating the “accuracy score” of the pre-selected features based on supervised machine learning methods (SVM, DT etc.). The automatic evaluation will then be completed by the prediction of the quality of performance based on rules (“if-then”) predefined by the experts through a series of interviews and elicitation process.
- Proposal of a hybrid model: Finally, a hybrid model will be developed to merge the 2 evaluation perspectives. The intern will choose the appropriate method for this hybridization (e.g. Bayesian networks, MaxEnt, attention network, weighted “Ensemble” methods, etc.).
- Evaluation of results: The intern will also contribute to the analysis of the results by comparing the evaluation carried out by the automatic approach, the rule-based approach and the hybrid approach. If possible, he will also conduct a qualitative evaluation with the experts/trainers to define the relevance of the hybrid approach.
References:
Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A., & Bermúdez i Badia, S. B. (2021, May). A human-ai collaborative approach for clinical decision making on rehabilitation assessment. In Proceedings of the 2021 CHI conference on human factors in computing systems (pp. 1-14).
X. Li, J. Yao, J. Ren and L. Wang, « A New Feature Selection Algorithm Based on Deep Q-Network, » 2021 40th Chinese Control Conference (CCC), Shanghai, China, 2021, pp. 7100-7105, doi: 10.23919/CCC52363.2021.9550745.
Frangoudes, F., Matsangidou, M., Schiza, E. C., Neokleous, K., & Pattichis, C. S. (2022). Assessing human motion during exercise using machine learning: A literature review. IEEE Access, 10, 86874-86903.
Bouchlaghem, Y., Akhiat, Y., Touchanti, K., & Amjad, S. (2024). A novel feature selection method with transition similarity measure using reinforcement learning. Decision Analytics Journal, 11, 100477.
