異なる言語による音声と人物口形状との対応付け : 画像通信システム

概要

論文の詳細を見る
When the speech spoken by a person on the screen is translated into the one spoken in a different language, the mismatching between the speech and the mouth shapes will be observed. This type of problem also occurs in the multilingual broadcasting of TV programs. This paper proposes a novel method to reduce the mismatching between the speech spoken in a different language and the mouth shapes in the moving facial images. Two basic techniques are utilized to carry out the conversion of mouth shapes. One is the rule-based conversion from text to mouth shapes and speech. The other is the 3-D motion estimation of whole head using the distribution of 2-D motion vectors detected from the sequence of input frames and a 3-D-shape model of head. After obtaining the converted mouth shapes represented by the 3-D-shape model, the original texture of mouth area is mapped onto the 3-D-shape model. Then, this mouth image replaces the mouth area in the input frame. Some experimental results are shown to demonstrate the effect and usefulness of the proposed method.
一般社団法人映像情報メディア学会の論文
1992-10-15