A GAN-based Approach for Generating Culture-Aware Co-Speech Gestures

Embedding social robots with the capability of accompanying their sentences with natural gestures may be the key to increasing their acceptability and their usage in real contexts. However, it could be argued that the definition of natural communicative gestures is not trivial, since it strictly depends on the culture of the person interacting with the robot. The proposed work investigates the usage of Generative Adversarial Networks (GANs) for generating culture-dependent communicative gestures based on speech audio features. To this aim, a custom dataset, only composed of persons belonging to the same culture, has been created, to extract all keypoints and audio features needed to train the network. Then, a generative model, also consisting of a voice conversion module, has been implemented and tested with the humanoid robot Pepper. Preliminary results, obtained through objective measurements and subjective evaluation, show that the proposed approach may be promising for generating culture-dependent communicative gestures for social robots.