虚拟装配实训具身交互技术应用研究

颜羽鹏; 姜可; 石星辰; 张靖宇; 明世杰; 何自强

doi:10.19554/j.cnki.1001-3563.2025.18.008

PDF(1138 KB)

包装工程（设计栏目） ›› 2025, Vol. 46 ›› Issue (18) : 84-95. DOI: 10.19554/j.cnki.1001-3563.2025.18.008

工业设计

虚拟装配实训具身交互技术应用研究

颜羽鹏, 姜可^*, 石星辰, 张靖宇, 明世杰, 何自强

作者信息 +

Application of Embodied Interaction Technology in Virtual Assembly Training

YAN Yupeng, JIANG Ke^*, SHI Xingchen, ZHANG Jingyu, MING Shijie, HE Ziqiang

Author information +

文章历史 +

摘要

目的为应对未来装配训练人机交互中的诸多挑战,梳理和分析了虚拟装配实训具身交互技术的研究现状并提出了研究思路,为人机交互理论和实践的发展提供支撑。方法分析虚拟装配实训系统现状及人机交互技术需求,结合具身交互技术特性,对系统的实施路径与关键技术进行归纳与总结。结果虚拟装配实训中多通道交互反馈融合是当前学界所面临的难题。具身交互技术促进了视听触多感官协同,为学员带来了更为高效和生动的认知体验。结论针对具身交互技术的应用研究,以多通道交互指令整合、交互反馈调控为核心目标。通过多通道交互实操、交互反馈融合及反馈信息调节,提升学员的认知效率,对促进虚拟装配实训及具身交互技术的未来发展具有至关重要的作用。

Abstract

The work aims to sort out and analyze the research status of embodied interaction technology in virtual assembly training and put forward research ideas to provide support for the development of human-machine interaction theory and practice, so as to deal with the challenges of human-machine interaction in assembly training in the future. The status quo of virtual assembly training systems and the technical requirements of human-machine interaction were analyzed, and the characteristics of embodied interaction technology were combined to conclude and summarize the implementation path and key technologies of the system. As a result, multi-channel interactive feedback fusion in virtual assembly training was a difficult problem facing the current academic community. Embodied interaction technology promoted audio-visual touch multi-sensory collaboration and brought more efficient and vivid cognitive experience to students. In conclusion, aiming at the application research of embodied interaction technology, multi-channel interactive instruction integration and interactive feedback regulation are the core goals. It is crucial to promote the future development of virtual assembly training and embodied interaction technology by improving students' cognitive efficiency through multi-channel interactive practice, interactive feedback fusion and feedback information adjustment.

导出引用1

颜羽鹏, 姜可, 石星辰, 张靖宇, 明世杰, 何自强. 虚拟装配实训具身交互技术应用研究[J]. 包装工程. 2025, 46(18): 84-95 https://doi.org/10.19554/j.cnki.1001-3563.2025.18.008

YAN Yupeng, JIANG Ke, SHI Xingchen, ZHANG Jingyu, MING Shijie, HE Ziqiang. Application of Embodied Interaction Technology in Virtual Assembly Training[J]. Packaging Engineering. 2025, 46(18): 84-95 https://doi.org/10.19554/j.cnki.1001-3563.2025.18.008

中图分类号： TB482 TP391

参考文献

[1] 教育部. 教育信息化2.0行动计划[J]. 中华人民共和国教育部公报, 2018(4): 118-125.
The Ministry of Education on Printing and Distributing. The "Education Informatization 2.0 Action Plan"[J]. Bulletin of the Ministry of Education of the People's Republic of China, 2018(4): 118-125.
[2] 李鹏宇. 试论虚拟装配关键技术及其发展[J]. 山东工业技术, 2016(18): 242.
LI P Y.On the Key Technologies and Development of Virtual Assembly[J]. Shandong Industrial Technology, 2016(18): 242.
[3] DOURISH P.Where the Action is: the Foundations of Embodied Interaction[M]. Cambridge: The MIT Press, 2001:17.
[4] SPACKMAN J S, YANCHAR S C. Embodied Cognition, Repre-sentationalism,and Mechanism: A Reviewand Analysis[J]. Journal for the Theory of Social Behaviour, 2014, 44(1):46-79.
[5] DE PREESTER H.Technology and the Body:The (im) Possibilities of Re-embodiment[J]. Foundations of Science, 2011, 16(2): 119-137.
[6] 董占勋, 李亚鸿, 贺星涛, 等. 增强现实技术驱动儿童教育积木产品创新设计流程研究[J]. 装饰, 2024(4): 133-135.
DONG Z X, LI Y H, HE X T, et al.Research on the Innovative Design Process of Children's Educational Building Block Products Driven by Augmented Reality Technology[J]. Decoration, 2024(4): 133-135.
[7] 杨婧岚, 欧阳宏生. 具身认知视域下的主流价值传播创新[J]. 湖南师范大学社会科学学报, 2021(3): 65-73.
YANG J L, OUYANG H S.Innovation of Mainstream Value Communication from the Perspective of Embodied Cognition[J]. Journal of Social Sciences of Hunan Normal University, 2021(3): 65-73.
[8] 杨易轩. VR影像的具身性交互叙事转向[J]. 新媒体研究, 2020(14): 137-138.
YANG Y X.Embodied Interactive Narrative Turning of VR Images[J]. New Media Research, 2020 (14): 137-138.
[9] 戴逸君. 浅析具身认知与一体化情景式体验[C]//上海交通大学, 国际设计科学学会(ISDS). 第七届东方设计论坛暨2021乡村振兴与东方设计国际研讨会论文集. 上海交通大学设计学院, 2021: 98-105.
DAI Y J.Analysis of Embodied Cognition and Integrated Situational Experience[C]//Shanghai Jiao Tong University, International Society for Design Science (ISDS). Proceedings of the 7th Oriental Design Forum and 2021 International Symposium on Rural Revitalization and Oriental Design. School of Design, Shanghai Jiao Tong University, 2021:98-105.
[10] 陆吉健, 周美美, 张霞, 等. 基于MR实验的“多模态+人机协同”教学及应用探索[J]. 远程教育杂志, 2021(6): 58-66.
LU J J, ZHOU M M, ZHANG X, et al."Multimodal + Human-Machine Collaboration" Teaching and Application Exploration Based on MR Experiment[J]. Journal of Distance Education, 2021(6): 58-66.
[11] 颜羽鹏, 姜可, 程健鹏, 等. 作战筹划系统多模态人机交互技术应用进展[J]. 机械设计, 2024, 41(7): 175-181.
YAN Y P, JIANG K, CHENG J P, et al.Application Progress of Multimodal Human-machine Interaction Technology in Combat Planning System[J]. Mechanical Design, 2024, 41(7): 175-181.
[12] 肖瑶, 吴耀辉, 王之纲. 从映射到吸纳: 具身交互中“沉浸”的技术意向探究[J]. 传媒, 2021(17): 87-90.
XIAO Y, WU Y H, WANG Z G.From Mapping to Absorption: An Exploration of the Technical Intention of "Immersion" in Embodied Interaction[J]. Media, 2021(17): 87-90.
[13] 张春华, 吴莎莎. 沉浸式学习背景下虚拟现实技术的教育应用[J]. 中国现代教育装备, 2018, 300(20): 72-76.
ZHANG C H, WU S S.Educational Application of Virtual Reality Technology in the Context of Immersive Learning[J]. China Modern Educational Equipment, 2018, 300(20): 72-76.
[14] 米哈赖M C. 心流: 最优体验心理学[M]. 张定绮, 译. 北京: 中信出版社,2017.
MIHARAI M C.Heart Flow: the Psychology of Optimal Experience[M]. ZHANG D Q, translated. Beijing: CITIC Press, 2017.
[15] WAMBECKE J, GOGUEY A, NIGAY L, et al.M[Eye] Cro: Eye-gaze+Microgestures for Multitasking and Interruptions[J]. Proceedings of the ACM on Human- Computer Interaction, 2021, 5(EICS):1-22.
[16] DEBIE E, ROJAS R F, FIDOCK J, et al.Multi-modal Fusion for Objective Assessment of Cognitive Workload: A Review[J]. IEEE Transactions on Cybernetics, 2019, 51(3): 1542-1555.
[17] 张俊峰, 薛青, 吴堃, 等. 军事行动推演系统通用建模机制研究[J]. 计算机仿真, 2018, 35(6): 13-17.
ZHANG J F, XUE Q, WU K, et al.Research on the General Modeling Mechanism of Military Operation Deduction System[J]. Computer Simulation, 2018, 35(6): 13-17.
[18] 冷志成, 薛凤桐, 张灏龙, 等. 基于智能仿真推演的导弹防御指挥控制系统研究[J]. 软件工程, 2021, 24(8): 39-43.
LENG Z C, XUE F T, ZHANG H L, et al.Research on Missile Defense Command and Control System Based on Intelligent Simulation and Deduction[J]. Software Engineering, 2021, 24(8): 39-43.
[19] 张晓海, 操新文. 基于深度学习的军事智能决策支持系统[J]. 指挥控制与仿真, 2018, 40(2): 1-7.
ZHANG X H, ZAO X W.Military Intelligent Decision Support System Based on Deep Learning[J]. Command and Control and Simulation, 2018, 40(2): 1-7.
[20] RICHARDS D, TAYLOR M.A Comparison of Learning Gains when Using a 2D Simulation Tool Versus a 3D Virtual World: An Experiment to Find the Right Representation Involving the Marginal Value Theorem[J]. Computers & Education, 2015(8): 157-171.
[21] 丁楠, 汪亚珉. 虚拟现实在教育中的应用:优势与挑战[J]. 现代教育技术, 2017(2): 19-25.
DING N, WANG Y M.The Application of Virtual Reality in Education: Advantages and Challenges[J]. Modern Educational Technology, 2017(2): 19-25.
[22] COOPER N, MILELLA F, PINTO C, et al.The Effects of Substitute Multisensory Feedback on Task Performance and the Sense of Presence in a Virtual Reality Environment[J]. Plos One, 2018(2):1-25.
[23] 高媛, 刘德建, 黄真真, 等. 虚拟现实技术促进学习的核心要素及其挑战[J]. 电化教育研究, 2016(10): 77-87.
GAO Y, LIU D J, HUANG Z Z, et al.The Core Elements and Challenges of Virtual Reality Technology to Promote Learning[J]. Research on Electronic Education, 2016(10): 77-87.
[24] 祝士明, 陈静潇. 虚拟现实学习环境的作用、挑战以及应对策略[J]. 现代教育技术, 2019(2): 39-45.
ZHU S M, CHEN J X. The Role, Challenges and Coping Strategies of Virtual Reality Learning Environment[J]. Modern Educational Technology, 2019(2): 39-45.
[25] 郭继武, 李邢西. 虚拟现实技术在高校思政课的应用[J]. 中国高等教育, 2017(23): 20-21.
GUO J W, LI X X.Application of Virtual Reality Technology in Ideological and Political Courses in Colleges and Universities[J]. China Higher Education, 2017(23): 20-21.
[26] 谭旸, 袁勤俭. 认知负荷理论及其在信息系统研究中的应用与展望[J]. 现代情报, 2019, 39(12): 160-169.
TAN Y, YUAN Q J.Cognitive Load Theory and Its Application and Prospect in Information System Research[J]. Modern Intelligence, 2019, 39(12): 160-169.
[27] 孙凌云, 王可幸, 陈培, 等. 多通道人机交互中感觉刺激的模拟方法研究综述[J]. 包装工程, 2023, 44(8): 81-94.
SUN L Y, WANG K X, CHEN P, et al.A Review of Research on Simulation Methods of Sensory Stimulation in Multi-channel Human-machine Interaction[J]. Packaging Engineering, 2023, 44(8): 81-94.
[28] 陈晓皎, 薛澄岐, 陈默, 等. 基于眼动追踪实验的数字界面质量评估模型[J]. 东南大学学报(自然科学版), 2017, 47(1): 38-42.
CHEN X J, XUE C Q, CHEN M, et al.Quality Assessment Model of Digital Interface Based on Eye-Tracking Experiments[J]. Journal of Southeast University (Natural Science Edition), 2017, 47(1): 38-42.
[29] 吴丹, 梁少博, 董品. 查询式序列视角下跨设备搜索信息准备与重用阶段的眼动变化研究[J]. 情报学报, 2019, 38(7): 760-770.
WU D, LIANG S B, DONG J.Change of Gaze Behavior InInformation Preparation and Resumption of Crss-device SearchBased on Query Lists[J]. Journal of the China Society for Scientificand Technical Information, 2019, 38(7): 760-770.
[30] 赵佳宝, 张丙辰, 韦懿洋, 等. 基于视觉刺激的ASD儿童干预图卡角色设计研究[J]. 包装工程, 2022, 43(10): 23-40.
ZHAO J B, ZHANG B C, WEI Y Y, et al.Role Design OlIntervention Picture Card for ASD Children Based on Visual Stimulation[J]. Packaging Engineering, 2022, 43(10): 23-40.
[31] 尤洋, 王以宁, 张海. 智慧课堂环境下教学视频复杂度与学习者认知负荷关系研究[J]. 现代远距离教育, 2020(2): 91-96.
YOU Y, WANG Y N, ZHANG H.Research on the Relationship between the Complexity of Teaching Video and LearnersCognitive Load in Intelligent Classroom Environment[J]. ModernDistance Education, 2020(2): 91-96.
[32] CHEN S Y, EPPS J.Using Task-induced Pupil Diameter and BlinkRate to Infer Cognitive Load[J]. Human- Computer Interaction, 2014, 29(4): 390-413.
[33] 薛耀锋, 李卓玮. 基于眼动追踪技术的在线学习认知负荷量化模型研究[J]. 现代教育技术, 2019, 29(7): 59-65.
XUE Y F, LI Z W.Research on Online Learning CognitiveLoad Quantitative Model Based on Eye Tracking Technology[J]. Modern Educational Technology, 2019, 29(7): 59-65.
[34] 戴敏丽. 基于虚拟现实的设备装配训练系统设计与实现[D]. 武汉: 华中科技大学, 2019.
DAI M L.Design and Implementation of Equipment Assembly Training System Based on Virtual Reality[D]. Wuhan: Huazhong University of Science and Technology, 2019.
[35] 邱煜健. 虚拟现实技术在中职汽车维修教育的应用前景[J]. 南方农机, 2018, 49(18): 148.
QIU Y J.The Application Prospect of Virtual Reality Technology in Secondary Vocational Automobile Maintenance Education[J]. Southern Agricultural Machinery, 2018, 49(18): 148.
[36] 梁剑斌, 周志豪, 曾秋玮. 面向增强现实虚拟装配细小零件视觉反馈模型[J]. 现代计算机, 2021, 27(26): 62-66.
LIANG J B, ZHOU Z H, ZENG Q W.Visual Feedback Model for Augmented Reality-oriented Virtual Assembly of Small Parts[J]. Modern Computer, 2021, 27(26): 62-66.
[37] JOSEPH A W, MURUGESH R.Potential Eye Tracking Metrics and Indi-cators to Measure Cognitive Oad in Human-Computer InteractionResearch[J]. Journal of Scientifc Research, 2020, 64(1): 168-175.
[38] LORENZ S, HELMERT J R, ANDERS R, et al.UUX Evaluation of a Digitally Advanced Human-Machine Interface for Excavators[J]. Multi-modal Technologies and Interaction, 2020, 4(3): 57.
[39] 潘飞, 姜可, 王东琦. 基于眼动追踪技术的购票网站可用性设计研究[J]. 包装工程,2020, 41(24): 243-247.
PAN F, KE J, WANG D Q.Research on Usability Design of Ticketing Websites Based on Eye Tracking Technology[J]. Packaging Engineering, 2020, 41(24): 243-247.
[40] DALKEY N C.Delphi[M]//London: Routledge, 2018: 25-30.
[41] CHEN X, JIANG X Q, LU Y.Study on the Rural Eeotourism Re-source Evaluation System[J]. Environmental Technology & Inno-vation, 2020(20): 101131.