基于AR头显设备的巡检场景手势交互设计研究

张帆; 黄晓峰; 娄焕志; 李甜

doi:10.19554/j.cnki.1001-3563.2026.04.004

PDF(1926 KB)

包装工程（设计栏目） ›› 2026, Vol. 47 ›› Issue (4) : 39-52. DOI: 10.19554/j.cnki.1001-3563.2026.04.004

工业设计

基于AR头显设备的巡检场景手势交互设计研究

张帆, 黄晓峰, 娄焕志, 李甜^*

作者信息 +

Designing and Evaluating Gesture Interaction for AR-Based Industrial Inspections

ZHANG Fan, HUANG Xiaofeng, LOU Huanzhi, LI Tian^*

Author information +

文章历史 +

摘要

目的为解决增强现实头显在工业管廊巡检中存在的交互认知断层与长时间作业生理疲劳问题,本研究旨在构建并验证一套兼顾直觉认知与工效舒适度的自然手势交互方案。方法融合基于现实的交互与工效学理论,构建了“直觉-工效”双重约束的交互设计模型。通过任务分析明确管廊巡检的核心需求,采“绿野仙踪”（Wizard of Oz）实验提取用户本能意向。针对大幅度挥臂导致的疲劳效应,引入“微手势”修正策略,确立了包含7个核心动作的手势集。搭建包含实体管道道具的“情境模拟”实验平台,在虚实融合环境下对方案的直观性、舒适度及任务鲁棒性进行实证评估。结果实验数据显示,该方案获得了较高的用户认可,整体可用性平均分达到4.35（S_D值为0.65）,“使用意愿”评分高达4.49。其中,“指尖微操”策略显著提升了感知舒适度,有效调和了操作直观性与生理负荷之间的矛盾。结论本研究提出的基于情境适应性的微手势交互方案,成功验证了从理论构建、策略修正到情境化评估的完整设计范式,为AR技术在工业巡检领域的落地提供了理论与实证支持。

Abstract

The work aims to construct and verify a natural gesture interaction scheme that balances intuitive cognition with ergonomic comfort to address the cognitive disconnect in interaction and the physiological fatigue associated with prolonged operations in industrial pipe gallery inspections using Augmented Reality (AR) headsets. Firstly, integrating Reality-Based Interaction (RBI) and ergonomics theories, an "Intuition-Ergonomics" dual-constraint interaction design model was constructed. Then, through task analysis, the core requirements of pipe gallery inspection were defined, and a "Wizard of Oz" experiment was conducted to elicit users instinctive intents. To address the fatigue effects caused by large-amplitude arm movements, a "micro-gesture" correction strategy was introduced, establishing a gesture set containing seven core actions. Finally, a "context simulation" experimental platform featuring physical pipe props was developed to empirically evaluate the scheme's intuitiveness, comfort, and task robustness within a mixed reality environment. Experimental data indicated high user acceptance, with an average overall usability score of 4.35 (SD=0.65) and a "willingness to use" score reaching 4.49. Notably, the "fingertip micro-manipulation" strategy significantly enhanced perceived comfort, effectively reconciling the conflict between operational intuitiveness and physiological load. The context-adaptive micro-gesture interaction scheme proposed in this study successfully validates a complete design paradigm spanning theoretical construction, strategy revision, and contextualized evaluation, providing theoretical and empirical support for the practical application of AR technology in the field of industrial inspection.

导出引用1

张帆, 黄晓峰, 娄焕志, 李甜. 基于AR头显设备的巡检场景手势交互设计研究[J]. 包装工程. 2026, 47(4): 39-52 https://doi.org/10.19554/j.cnki.1001-3563.2026.04.004

ZHANG Fan, HUANG Xiaofeng, LOU Huanzhi, LI Tian. Designing and Evaluating Gesture Interaction for AR-Based Industrial Inspections[J]. Packaging Engineering. 2026, 47(4): 39-52 https://doi.org/10.19554/j.cnki.1001-3563.2026.04.004

中图分类号： TB482

参考文献

[1] 王宇希, 张凤军, 刘越. 增强现实技术研究现状及发展趋势[J]. 科技导报, 2018, 36(10): 75-83.
WANG Y X, ZHANG F J, LIU Y.Augmented Reality Technology[J]. Science & Technology Review, 2018, 36(10): 75-83.
[2] 何明, 陈莹莹, 张斌, 等. AR技术在配电站房巡检业务中的应用研究[J]. 现代信息科技, 2019, 3(1): 175-176.
HE M, CHEN Y Y, ZHANG B, et al.Research on Application of AR Technology in Power Distribution Station Inspection[J]. Modern Information Technology, 2019, 3(1): 175-176.
[3] 李子阳, 张东胜, 杜邦. 基于AR的智能安全巡检系统[J]. 北京化工大学学报(自然科学版), 2022, 49(5): 59-66.
LI Z Y, ZHANG D S, DU Y B.An Intelligent Safety Inspection System Based on AR Technology[J]. Journal of Beijing University of Chemical Technology(Natural Science Edition), 2022, 49(5): 59-66.
[4] 李大勇, 杨畅, 张永伍, 等. 基于AR技术的变电站智能巡检系统设计与实现[J]. 微型电脑应用, 2020, 36(8): 92-94.
LI D Y, YANG C, ZHANG Y W, et al.Application of Remote Assistance System for Intelligent Inspection of Substation Based on the AR Technology[J]. Microcomputer Applications, 2020, 36(8): 92-94.
[5] 李仕, 卢俊阳, 张明威, 等. 大场景下自由手势交互技术[J]. 印刷与数字媒体技术研究, 2023(4): 244-251.
LI S, LU J Y, ZHANG M W, et al.Free Gesture Interaction Technology in Large Scenes[J]. Printing and Digital Media Technology Research, 2023(4): 244-251.
[6] 瞿珏, 焦浩洋, 王庆力, 等. 直感交互人机界面研究综述[J]. 兵器装备工程学报, 2023, 44(12): 141-150.
QU J, JIAO H Y, WANG Q L, et al.Review of Direct Interaction Human-Machine Interface Research[J]. Journal of Ordnance Equipment Engineering, 2023, 44(12): 141-150.
[7] 易心武, 薛锦云, 游珍, 等. 沉浸式虚拟现实多模态交互模型研究[J]. 江西师范大学学报(自然科学版), 2024, 48(1): 52-58.
YI X W, XUE J Y, YOU Z, et al.Research on Multimodal Interaction Model of Immersive Virtual Reality[J]. Journal of Jiangxi Normal University (Natural Science Edition), 2024, 48(1): 52-58.
[8] VERTEGAAL R.A Fitts Law Comparison of Eye Tracking and Manual Input in the Selection of Visual Targets[C]// Proceedings of the 10th International Conference on Multimodal Interfaces. New York, NY, USA: Association for Computing Machinery, 2008: 241-248.
[9] HURTIENNE J, KLÖCKNER K, DIEFENBACH S,et al. Designing with Image Schemas: Resolving the Tension between Innovation,Inclusion and Intuitive Use[J]. Interacting with Computers, 2015, 27(3): 235-255.
[10] YU C, SUN K, ZHONg M Y, et al.One-dimensional Handwriting: Inputting Letters and Words on Smart Glasses[C]// Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. San Jose: CHI, 2016: 71-82
[11] 魏嘉焜, 王家润. 手势识别与交互综述[J]. 计算机与现代化, 2024(8): 67-76.
WEI J K, WANG J R.Survey on Gesture Recognition and Interaction[J]. Computer and Modernization, 2024(8): 67-76.
[12] QIAN X Y, HAN L,ZHANG Y L, et al.An Object Tracking Method Using Deep Learning and Adaptive Particle Filter for Night Fusion Image[C]// Proceedings of the 2017 International Conference on Progress in Informatics and Computing. Paris: IEEE, 2017: 138-142.
[13] 丛玉华, 何啸, 邢长达, 等. 基于计算机视觉手势识别的人机交互技术研究[J]. 兵器装备工程学报, 2022, 43(1): 152-160.
CONG Y H, HE X, XING C D, et al.Research on Human-Computer Interaction Technology Based on Computer Vision Gesture Recognition[J]. Journal of Ordnance Equipment Engineering, 2022, 43(1): 152-160.
[14] 王丽园, 赵陶, 王文, 等. 具有姿态变化鲁棒性的行人检测跟踪算法[J]. 计算机工程与设计, 2022, 43(10): 2877-2881.
WANG L Y, ZHAO T, WANG W, et al.Pedestrian Detection and Tracking Algorithm with Robustness to Pose Changes[J]. Computer Engineering and Design, 2022, 43(10): 2877-2881.
[15] 方新国, 王赞强, 高健权. 基于Leap Motion的手势交互系统设计[J]. 机械设计与研究, 2020, 36(2): 128-132.
FANG X G, WANG Z Q, GAO J Q.Design of Gesture Interaction System Based on Leap Motion[J]. Machine Design and Research, 2020, 36(2): 128-132.
[16] HINCAPIE-RAMOS J D, GUO X, MOGHADASIAN P, et al. Consumed Endurance: A Metric to Quantify Arm Fatigue of Mid-air Interactions[C]// CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.New York: Association for Computing Machinery, 2014: 1063-1072.
[17] WOBBROCK J O, MORRIS M R, WILSON A D.User-Defined Gestures for Surface Computing[C]// CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York: Association for Computing Machinery, 2009: 1083-1092.
[18] PIUMSOMBOON T, CLARK A, BILLINGHURST M, et al.User-defined Gestures for Augmented Reality[C]// Proceedings of the CHI '13 Extended Abstracts on Human Factors in Computing Systems.New York: Association for Computing Machinery, 2013: 955-960.
[19] VILLARREAL-NARVAEZ S, VANDERDONCKT J, VATAVU R D, et al.A Systematic Review of Gesture Elicitation Studies: What can We Learn from 216 Studies?[C]// Proceedings of the 2020 ACM Designing Interactive systems Conference. New York: Association for Computing Machinery, 2020: 855-872.
[20] 钱晓松, 窦金花, 杜芃. 无人驾驶汽车行驶引导交互手势设计研究[J]. 包装工程, 2024, 45(8): 40-47.
QIAN X S, DOU J H, DU P.Research on Design of Interactive Gestures for Driving Guidance of Autonomous Vehicles[J]. Packaging Engineering, 2024, 45(8): 40-47.
[21] 鲁艺, 王晓烨, 王煜琳, 等. 面向混合现实教学的空中手势交互体验设计应用研究[J]. 包装工程, 2024, 45(4): 181-190.
LU Y, WANG X Y, WANG Y L, et al.Application of Mid-air Gesture Interactive Experience Design for Mixed Reality Teaching[J]. Packaging Engineering, 2024, 45(4): 181-190.
[22] 孙效华, 周博, 李彤. 隔空手势交互的设计要素与原则[J]. 包装工程, 2015, 36(8): 10-13.
SUN X H, ZHOU B, LI T.Design Elements and Principles of Mid-Air Gesture Interaction[J]. Packaging Engineering, 2015, 36(8): 10-13.
[23] 钱晓松, 张帆, 杨新. 基于全自动驾驶情境下驾驶导航需求的手势交互设计研究[J]. 装饰, 2020(4): 104-107.
QIAN X S, ZHANG F, YANG X.Study on Gesture Interaction for Navigation in The Autonomous Driving Situation[J]. Art & Design, 2020(4): 104-107.
[24] JACOB R J K, GIROUARD A,, HIRSHFIELD L M, et al. Reality-based Interaction: a Framework for Post-WIMP Interfaces[C]// Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York: Association for Computing Machinery, 2008: 201-210.
[25] NORMAN D A.The Design of Everyday Things[M]. New York: Basic Books, 2013.
[26] BOWMAN D A, KRUIJFF E, LAVIOLA J J J, et al. 3D User Interfaces: Theory and Practice[M]. Boston: Addison-Wesley, 2004.
[27] 陶建华, 巫英才, 喻纯, 等. 多模态人机交互综述[J]. 中国图象图形学报, 2022, 27(6): 1956-1987.
TAO J H, WU Y C, YU C, et al.A Survey on Multi-modal Human-computer Interaction[J]. Journal of Image and Graphics, 2022, 27(6): 1956-1987.
[28] MORRIS M R, WOBBROCK J O, WILSON A D.Understanding Users' Preferences for Surface Gestures[C]// Proceedings of Graphics Interface 2010. Toronto: Canadian Information Processing Society,2010: 261-268.