Degree of Human Perception of Facial Emotions Based on Audio and Video Information

概要

論文の詳細を見る
A large number of research papers can be found which are devoted purely to visual or purely to auditory facial emotion detection. However we found that it is very interesting to consider both these auditory and visual informations together, for processing, since we hope this kind of multi modal information processing will become a datum of information processing in future multimedia era. By several intensive subjective evaluation studies we found that human beings recognize some emotions (out of the 6 basic emotions identified by Ekman et. al.) based on their visual content while some other emotions are recognized by their auditory content or by using both. The subjects were Japanese under-graduates (ages between 18-24) while the speakers were Spanish or Sinhalese to make the context in-comprehensible to the subjects. We found that, most of the subjects easily identified Anger, happiness, Surprise and Dislike by their visual appearance, compared to voice only detection. When the audio track of each emotion clip is dubbed with a different type of emotional expression, still Anger, Happiness and Surprise were video dominant. If the audio track were dubbed with Sad or Fear emotional expressions then the subjects tended to identify those emotions as Sad or as Fear irrespective of the visual appearance. However Dislike emotion gave mixed responses. Then we analyzed the percentage correct detection of negative and non-negative emotions based on sound and vision, and found some similarities in different speakers. As a conclusion to the paper we propose a method of facial emotion detection by using a hybrid approach, which uses audio and video information in a hierarchical manner.
1996-10-14

Degree of Human Perception of Facial Emotions Based on Audio and Video Information

スポンサーリンク

概要

著者

関連論文

スポンサーリンク