Research on Multimodal Interaction Design Strategies for ICH APPs Based on VACP Model

LAI Shiwen; KUANG Yuxiang; CHENG Zuling; XU Liping

doi:10.19554/j.cnki.1001-3563.2025.20.021

PDF(2977 KB)

Packaging Engineering ›› 2025, Vol. 46 ›› Issue (20) : 225-236. DOI: 10.19554/j.cnki.1001-3563.2025.20.021

Industrial Design

Research on Multimodal Interaction Design Strategies for ICH APPs Based on VACP Model

LAI Shiwen¹, KUANG Yuxiang², CHENG Zuling³, XU Liping^1,*

Author information +

History +

Abstract

The work aims to explore the theoretical foundation of multimodal interaction design based on the VACP model and apply it to optimize the interaction of intangible cultural heritage (ICH) APPs, so as to enhance user experience. Starting from the four core dimensions of visual (V), auditory (A), cognitive (C), and psychomotor (P), a multimodal interaction analysis framework based the VACP model was constructed. By examining users' resource load characteristics across different tasks, targeted interaction design strategies were proposed. The model effectively identified interaction problems encountered by users when using ICH applications and provided a clear basis for system optimization. In conclusion, the application of the VACP model in multimodal interaction design for ICH APPs demonstrates its effectiveness. It offers a practical design path for improving user experience in complex interaction scenarios and provides theoretical and practical references for the design of other intelligent interactive systems.

Key words

multimodal / VACP model / ICH APP / interaction design

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

LAI Shiwen, KUANG Yuxiang, CHENG Zuling, XU Liping. Research on Multimodal Interaction Design Strategies for ICH APPs Based on VACP Model[J]. Packaging Engineering. 2025, 46(20): 225-236 https://doi.org/10.19554/j.cnki.1001-3563.2025.20.021

References

[1] 徐俊, 孟刚. 基于双钻+PBL模式的非遗技艺学习类APP交互设计研究[J]. 包装工程, 2024, 45(18): 352-357.
XU J, MENG G.Research on The Interaction Design of Non-Heritage Skills Learning APP Based on Double Drill+PBL Model[J]. Packaging Engineering: 2024, 45(18): 352-357.
[2] 吴兰. 基于UCD视角的非物质文化遗产APP的痛点分析[J]. 出版广角, 2017(9): 60-62.
WU L.Analysis on the Pain Point of Intangible Cultural Heritage APP from the Perspective of UCD[J]. View on Publishing, 2017(9): 60-62.
[3] 沈宏扬, 黄玉, 万长林. 面向学龄儿童的非遗信息交互设计路径探析——以“识·绣”交互设计为例[J]. 湖南包装, 2023, 38(6): 132-136.
SHEN H Y, HUANG Y, WAN C L.Exploring the Path of Intangible Cultural Heritage Information Interaction Design for School Aged Children: Taking "Recognition Embroidery" Interaction Design as an Example[J]. Hunan Packaging, 2023, 38(6): 132-136.
[4] WICKENS C D.Multiple Resources and Mental Workload[J]. Human Factors, 2008, 50(3): 449-455.
[5] OVIATT S, COULSTON R, LUNSFORD R.When Do we Interact Multimodally? : Cognitive Load and Multimodal Communication Patterns[C]// Proceedings of the 6th International Conference on Multimodal Interfaces. State College: ACM, 2004: 129-136.
[6] 黄爱涵, 孙宁娜, 李明珠. 基于EEI模型的非遗手工艺APP设计研究[J]. 大众文艺, 2024(5): 43-45.
HUANG A H, SUN N N, LI M Z.Research on the Design of Intangible Handicraft APP Based on EEI Model[J]. Popular Literature and Art, 2024(5): 43-45.
[7] 徐世博, 陆野, 王子健, 等. 服务设计视角下非遗手工艺活化平台APP设计研究[J]. 包装工程, 2024, 45(6): 326-337.
XU S B, LU Y, WANG Z J, et al.APP Design of Intangible Cultural Heritage Handicraft Revitalization Platform from the Perspective of Service Design[J]. Packaging Engineering, 2024, 45(6): 326-337.
[8] 王子楷, 杨弃. 交互式叙事在非遗数字博物馆设计中的应用与探索[J]. 设计, 2024, 37(4): 64-67.
WANG Z K, YANG Q.The Application and Exploration of Interactive Narration in the Design of Intangible Cultural Heritage Digital Museum[J]. Design, 2024, 37(4): 64-67.
[9] 李建, 田崑. 互联网产品设计中用户体验与交互设计优化对策[J]. 包装工程, 2024, 45(14): 399-402.
LI J, TIAN K.Optimization of User Experience and Interaction Design in Internet Product Design[J]. Packaging Engineering, 2024, 45(14): 399-402.
[10] 姚奕, 陈朝阳, 杜晓明, 等. 多模态知识图谱构建技术及其在军事领域的应用综述[J]. 计算机工程与应用, 2024, 60(22): 18-37.
YAO Y, CHEN Z Y, DU X M, et al.Survey of Multimodal Knowledge Graph Construction Technology and Its Application in Military Field[J]. Computer Engineering and Applications, 2024, 60(22): 18-37.
[11] 戴妍妍, 金赟, 马勇, 等. 基于高效通道注意力机制的语音情感识别方法[J]. 信号处理, 2021, 37(10): 1835-1842.
DAI Y Y, JIN Y, MA Y, et al.Speech Emotion Recognition Based on Efficient Channel Attention[J]. Journal of Signal Processing, 2021, 37(10): 1835-1842.
[12] 陈囿任, 李勇, 温明, 等. 多模态知识图谱融合技术研究综述[J]. 计算机工程与应用, 2024, 60(13): 36-50.
CHEN Y R, LI Y, WEN M, et al.Research and Comprehensive Review on Multi-Modal Knowledge Graph Fusion Techniques[J]. Computer Engineering and Applications, 2024, 60(13): 36-50.
[13] 丁美荣, 陈鸿业, 曾碧卿. 跨模态语义对齐和信息细化的多模态情感分析[J]. 计算机工程与应用, 2024, 60(22): 114-125.
DING M R, CHEN H Y, ZENG B Q.Cross-Modal Semantic Alignment and Informant-ion Refinement for Multimodal Sentiment Analysis[J]. Computer Engineering and Applications, 2024, 60(22): 114-125.
[14] 周柏男, 李旭, 范丰龙, 等. 基于交互注意力机制的多模态情感分析模型[J]. 大连工业大学学报, 2023, 42(5): 378-384.
ZHOU B N, LI X, FAN F L, et al.Multi-Modal Sentiment Analysis Model Based on Interactive Attention Mechanism[J]. Journal of Dalian Polytechnic University, 2023, 42(5): 378-384.
[15] 吴思思, 马静. 基于感知融合的多任务多模态情感分析模型[J]. 数据分析与知识发现, 2023, 7(10): 74-84.
WU S S, MA J.Multi-Task & Multi-Modal Sentiment Analysis Model Based on Aware Fusion[J]. Data Analysis and Knowledge Discovery, 2023, 7(10): 74-84.
[16] 潘家辉, 何志鹏, 李自娜, 等. 多模态情绪识别研究综述[J]. 智能系统学报, 2020, 15(4): 633-645.
PAN J H, HE Z P, LI Z N, et al.A Review of Multimodal Emotion Recognition[J]. CAAI Transactions on Intelligent Systems, 2020, 15(4): 633-645.
[17] 金磊, 吉翔, 邓丽云, 等. 手机端多模态低头族危险感知与预警[J]. 电子测量技术, 2024, 47(9): 172-183.
JIN L, JI X, DENG L Y, et al.Multimodal Hazard Sensing and Warning for Bowed-Head Tribe Based on Mobile Terminals[J]. Electronic Measurement Technology, 2024, 47(9): 172-183.
[18] 胡小强, 桑小十. 多模态交互在教学中的应用[J]. 中国信息技术教育, 2022(13): 89-93.
HU X Q, SANG X S.Application of Multi-Modal Interaction in Teaching[J]. China Information Technology Education, 2022(13): 89-93.
[19] 黄锦飞. 自动驾驶背景下的电动车HMI交互设计研究[D]. 上海: 华东师范大学, 2022.
HUANG J F.Research on Hmi Interactive Design of Electric Vehicle Under the Background of Automatic Driving[D]. Shanghai: East China Normal University, 2022.
[20] 田寒宇. 基于雷达技术的手势识别系统在人机交互中的应用[J]. 信息技术与信息化, 2020(4): 105-107.
TIAN H Y.Application of Gesture Recognition System Based on Radar Technology in Human-Computer Interaction[J]. Information Technology and Informatization, 2020(4): 105-107.
[21] 张轶, 韩宇翃. 基于多模态信息的交互景观设计研究[J]. 包装工程, 2024, 45(S1): 393-399.
ZHANG Y, HAN Y H.Research on Interactive Landscape Design Based on Multimodal Information[J]. Packaging Engineering, 2024, 45(S1): 393-399.
[22] 殷兵, 周良, 何山, 等. 多模态虚拟人交互的技术进展和应用[J]. 人工智能, 2024, 11(3): 15-26.
YIN B, ZHOU L, HE S, et al.Technical Progress and Application of Multimodal Virtual Human Interaction[J]. AI-View, 2024, 11(3): 15-26.
[23] 张永年, 赵英杰. 多模态交互包装设计评价体系构建与应用[J]. 包装工程, 2024, 45(20): 366-374.
ZHANG Y N, ZHAO Y J.Construction and Application of Evaluation System for M-ultimodal Interactive Packaging Design[J]. 包装工程, 2024, 45(20): 366-374.
[24] WICKENS C D, HELTON W S, HOLLANDS J G, et al.Engineering Psychology and Human Performance[M]. New York: Routledge, 2021.
[25] Wickens C D, Sandry D L, Vidulich M.Compatibility and Resource Competition between Modalities of Input, Central Processing, and Output[J]. Human Factors, 1983, 25(2): 227-248.
[26] Wickens C D.Multiple Resources and Performance Prediction[J]. Theoretical Issues in Ergonomics Science, 2002, 3(2): 159-177.
[27] Hancock P A.A Dynamic Model of Stress and Sustained Attention[J]. Human Factors, 1989, 31(5): 519-537.
[28] Wickens C D, Liu Y.Codes and Modalities in Multiple Resources: A Success and a Qualification[J]. Human Factors, 1988, 30(5): 599-616.
[29] Salvucci D D, Tatge N A.Threaded Cognition: an Integrated Theory of Concurrent Multitasking[J]. Psychological Review, 2008, 115(1): 101.
[30] Treisman A M, Gelada G.A Feature-Integration Theory of Attention[J]. Cognitive Psychology, 1980, 12(1): 97-136.
[31] SIMONS D J, LEVIN D T.Change Blindness[J]. Trends in Cognitive Sciences, 1997, 1(7): 261-267.
[32] HENDERSON J M, HOLLINGWORTH A.High-Level Scene Perception[J]. Annual Review of Psychology, 1999, 50(1): 243-271.
[33] WARE C.Information Visualization: Perception for Design[M]. Cambridge: Morgan Kaufmann, 2019.