A Review of Target Selection Techniques in Multimodal Interaction

ZHOU Xiao-zhou, ZONG Cheng-long, GUO Yi-bing, JIA Le-song, DU Xiao-xi, XUE Cheng-qi

Packaging Engineering ›› 2022, Vol. 43 ›› Issue (4) : 36-44.

PDF(490 KB)
PDF(490 KB)
Packaging Engineering ›› 2022, Vol. 43 ›› Issue (4) : 36-44. DOI: 10.19554/j.cnki.1001-3563.2022.04.004

A Review of Target Selection Techniques in Multimodal Interaction

  • ZHOU Xiao-zhou, ZONG Cheng-long, GUO Yi-bing, JIA Le-song, DU Xiao-xi, XUE Cheng-qi
Author information +
History +

Abstract

With the development of various sensors, computer recognition algorithms and computer networks, human-computer interaction (HCI) is no longer confined to the application of traditional input modes such as keyboards and mouse, but further expanded to the application of various modes driven by human behavior. Multimodal interaction can capture the explicit behavior information of multiple behavior modes and distinguish different forms of multimodal information after combination. Nowadays it has been applied in virtual reality, augmented reality, mixed reality, teleoperation, universal interaction and other settings to optimize human-computer interaction. Multi-touch, natural speech, gesture motion sensing and eye movement tracking are commonly used in natural interaction. And all of them show their advantages in specific fields. Multimodal interaction technology combines two or more input modes in selective, sequential, concurrent or complementary ways, using a variety of non-invasive sensors to identify naturally occurring human language and behavior. The core of this technology is to predict human interaction intention based on multi-modal information captured by various sensors and improve the understanding of machine to human behavior, which can be well applied to target selection tasks. In human-computer interaction system, target selection task is a basic task, which has been described by Fitz’s law model, optimized initial pulse model and cascade effect model under multiple modal interaction. These theories are helpful for the development of multi-modal interaction technology. To sum up, the research on target selection technology of multimodal interaction can effectively push human-computer interaction forward to natural interaction.

Key words

multimodal interaction; target selection technology; natural interaction; interactive intention

Cite this article

Download Citations
ZHOU Xiao-zhou, ZONG Cheng-long, GUO Yi-bing, JIA Le-song, DU Xiao-xi, XUE Cheng-qi. A Review of Target Selection Techniques in Multimodal Interaction[J]. Packaging Engineering. 2022, 43(4): 36-44 https://doi.org/10.19554/j.cnki.1001-3563.2022.04.004
PDF(490 KB)

Accesses

Citation

Detail

Sections
Recommended

/