Members: J. Françoise, V. Belissen, Y. Bellik, H. Bull, H. Chaaban, N. DelpratM. Gouiffès

Collaborators : A. Braffort (ILES), M. Filhol (ILES), É. Prigent (CPU), S. Fdili Alaoui (LRI)

Automatic Analysis of Sign Language Videos

lsf• Characterization of the articulators involved in French sign language (LSF): look, mouthing, movement and shape of the hands. Study of their impact in the distinction between lexical signs and illustrative signs.
• Characterization of manual movement (location, symmetry, mono and bimanual motion local and global movement)
• Recognition of lexical signs and identification of linguistic structures: pointing, buoys
• LSF video segmentation and annotation, pause detection
• Alignment of text (written French) and LSF video for the creation of a written French-LSF concordancer and for automatic subtitling LSF videos.

Methods : use of learning estimators such as Deep Hand, Deep Face, OpenPose, RNN LSTM recurrent networks, graph-based networks.

Live-coding for Movement Analysis and Sonification


The design of expressive movement-based interactions for contemporary dance and the performing arts requires the development of new design and methodologies and support tools. The movement theme contributes to :
• Developing a new approach to the design of movement-based interactions design relying on improvisation techniques.
• Enabling rapid prototyping and live-coding of movement-based interactions, in particular of movement sonification.
• Developing new improvisation practices in augmented dance where the interactions between movement and sound are programmed on the fly.
• Studying the influence of continuous auditory feedback on movement perception, in particular regarding the anticipation of a person's movement.

Methods : user-centred design, live-coding, reactive programming, motion signal processing, gesture recognition, experimental studies

Movement Learning in HCI


• Study and modeling of motor learning in HCI, in particular concerning the acquisition of coarticulation in the learning of bimanual gesture sequences
• Studies of "learning" and "learnable" interactions that take into account the reciprocal learning mechanisms between the user and an interactive machine learning system.
• Development of human-centered machine learning tools allowing the personalization of gestural interactions and adaptation to the sensory-motor abilities of different users.
• Application to the learning of movement in artistic contexts (dance) and for assistance to people with disabilities.

Methods : experimental studies of movement learning, statistical models for motion modeling (HMMs and extensions)

Emotion, avatar-matter and sense of the self

ELEMENTA delprat

  • Research-creation project on the emotional feelings related to a virtual transformation of the body density into classical elements (air, fire, water, earth)
  • Study of the relationships between body awareness and virtual materiality through the paradigm of Augmented Reverie, taking into account the imaginary dimension in the action-perception loop and the motionless interaction with a material avatar.
  • Evaluation of the impact of the different modalities (sound/movement/image) on the body's feeling and sense of the self (postures, psychomotor behaviours, explicitation interviews)
  • Design of experimental protocols for the development of an alternative and original method for cognitive and emotional remediation, focused on the modulation of the attentional processes involved in the perception of the internal body state and the regulation of emotions.

Methods : RêvA interactive device, developed at LIMSI-CNRS within the framework of the VIDA theme (collaboration N. Ladevèze and L. Bolot, P2I), used as a support for therapeutic mediation, for the study of the feeling of the self and as a tool for artistic works.

Campus universitaire bât 507
Rue du Belvédère
F - 91405 Orsay cedex
Tél +33 (0) 1 69 15 80 15


Scientific report

LIMSI in numbers

8 Research Teams
100 Researchers
40 Technicians and Engineers
60 Doctoral Students
70 Trainees


Paris-Saclay University new window

Logo DataIA