| 周小舟,宗承龙,郭一冰,贾乐松,杜晓茜,薛澄岐.多模态交互中的目标选择技术[J].包装工程,2022,43(4):36-44. |
| 多模态交互中的目标选择技术 |
| A Review of Target Selection Techniques in Multimodal Interaction |
| 投稿时间：2021-09-10 |
| DOI：10.19554/j.cnki.1001-3563.2022.04.004 |
| 中文关键词: 多模态交互 目标选择技术 自然交互 交互意图 |
| 英文关键词: multimodal interaction target selection technology natural interaction interactive intention |
| 基金项目: |
|
| 摘要点击次数: |
| 全文下载次数: |
| 中文摘要: |
| 目的 伴随着各类传感器、计算机识别算法与计算机网络的发展，人机交互已不再拘泥于应用键盘、鼠标等传统输入模态，而是进一步拓展为应用人的多种行为模态。方法 多模态交互可以捕获多个模态下人的显性行为信息，组合并分辨多个模态信息的不同形式，现已应用在虚拟现实、增强现实、混合现实、遥操作、普适交互等场景下，可以优化这些场景下的人机交互。在自然交互中，多点触摸、自然语音、手势体感、眼动追踪是常用的交互模态，展现出了较优的效果。而多模态交互技术按照选择、相继、并发和互补的方式组合两个及以上的输入模式，借助多种非侵入式的传感器，识别天然形成的人类语言和行为。该技术的核心是基于各种传感器所捕获的多模态信息来预测人的交互意图，提升机器对人的行为理解，这可以很好地应用到目标选择任务之中。结论 在人机交互系统中，目标选择任务是一种基础任务，已经有菲兹定律模型、优化初始脉冲模型和层叠效应模型等行为模型在多个模态交互下对其进行了描述，这些理论都有助于开发多模态交互技术。综上所述，对多模态交互的目标选择技术研究可以有效地推动人机交互向自然交互迈进。 |
| 英文摘要: |
| With the development of various sensors, computer recognition algorithms and computer networks, human-computer interaction (HCI) is no longer confined to the application of traditional input modes such as keyboards and mouse, but further expanded to the application of various modes driven by human behavior. Multimodal interaction can capture the explicit behavior information of multiple behavior modes and distinguish different forms of multimodal information after combination. Nowadays it has been applied in virtual reality, augmented reality, mixed reality, teleoperation, universal interaction and other settings to optimize human-computer interaction. Multi-touch, natural speech, gesture motion sensing and eye movement tracking are commonly used in natural interaction. And all of them show their advantages in specific fields. Multimodal interaction technology combines two or more input modes in selective, sequential, concurrent or complementary ways, using a variety of non-invasive sensors to identify naturally occurring human language and behavior. The core of this technology is to predict human interaction intention based on multi-modal information captured by various sensors and improve the understanding of machine to human behavior, which can be well applied to target selection tasks. In human-computer interaction system, target selection task is a basic task, which has been described by Fitz’s law model, optimized initial pulse model and cascade effect model under multiple modal interaction. These theories are helpful for the development of multi-modal interaction technology. To sum up, the research on target selection technology of multimodal interaction can effectively push human-computer interaction forward to natural interaction. |
|
查看全文
查看/发表评论 下载PDF阅读器 |
| 关闭 |
|
|
|
