Development of a Lip-Sync Algorithm Based on an Audio-Visual Corpus
スポンサーリンク
概要
- 論文の詳細を見る
In this paper, wo propose a corpus-based lip-syne algorithm for natural face animation. For this purpose, we constructed a Korean audio-visual (AV) corpus. Based on this AV corpus, we propose a concatenation method of AV units, which is similar to a corpus-based text-to-speech system. For our AV corpus, lip-related parameters were extracted from every video-recorded facial shot which of speaker reads the given texts selected from newspapers. The spoken utterances were labeled with HTK and such prosodic information as duration, pitch and intensity was extracted as lip-sync parameters. Based on the constructed AV corpus, basic synthetic units are set by CVC-syllable units. For the best concatenation performance, based on the phonetic environment distance and the prosodic distance, the best path is estimated by a general Viterbi search algorithm. From the computer simulation results, we found that the information concerned with not only duration but also pitch and intensity is useful to enhance the lip-sync performance. And the reconstructed lip parameters have almost equal values to those of the original parameters.
- 社団法人電子情報通信学会の論文
- 2003-02-01
著者
-
Shirai Katsuhiko
School Of Science And Engineering Waseda University
-
Shirai Katsuhiko
Department Of Information And Computer Science Waseda University
-
Lee J
Gyeongin National Univ. Education Inchon Kor
-
Kim Jinyoung
Department Of Electronics Computer And Information Eng. And Rrc Hecs Chon-nam National University
-
LEE Joohun
Department of Internet Broadcasting. Dong-Ah College
-
Lee Joohun
Department Of Electronics Engineering Seoul National University
-
Lee JooHun
Department of Biology, Yonsei University
関連論文
- Differential Expression of ADC mRNA during Development and upon Acid Stress in Soybean (Glycine max) Hypocotyls
- Purification and Characterization of Arginine Decarboxylase from Soybean (Glycine max) Hypocotyls
- A Synchronous Completion Prediction Adder (SCPA)
- Development of a Lip-Sync Algorithm Based on an Audio-Visual Corpus
- An Efficient Lip-Reading Method Robust to Illumination Variations
- Phrase Recognition in Conversational Speech Using Prosodic and Phonemic Information (Special Issue on Speech and Discourse Processing in Dialogue Systems)
- A Robust Recursive Least Square Algorithm against Impulsive Noise(Digital Signal Processing)
- Recognizing Reverberant Speech Based on Amplitude and Frequency Modulation
- Sounds of Speech Based Spoken Document Categorization : A Subword Representation Method(Speech Dynamics by Ear, Eye, Mouth and Machine)
- Extraction of Human Face and Transformable Region by Facial Expression Based on Extended Labeled Graph Matching
- The Skipping Technique : A Simple and Fast Algorithm to Find the Pitch in CELP Vocoder
- Linguistic Intelligent CAI System Using Speech Data-Base
- ANALYSIS OF PATH FLOW CHANGES CAUSED BY TRAFFIC INFORMATION PROVISION USING DYNAMIC PATH FLOW ESTIMATION