FUSING MULTIMODAL INPUTS

概要

論文の詳細を見る
The study reported here is an attempt to understand human verbal-gestural behavior in a multimodal bilingual setting. Specific issues addressed are: 1) What kind of deictic gestures people use in a machine-mediated condition, and how these differ from those used in a human-mediated condition, 2) How significant the use of gestures is in each condition, 3) How verbal and gestural behaviors are interrelated, and 4) What the implications of our findings are for a multimodal spoken language interpretation system. In this paper, we attempt to answer these questions, and introduce the architecture of a prototype multimodal user interface system. This system takes spoken language and deictic gestures and produces a combined semantic representation of the inputs.
社団法人電子情報通信学会の論文
1995-10-16