Design and Implementation of AR Visual Monitoring Systems for Industrial Manipulators

WANG Yan

doi:10.19554/j.cnki.1001-3563.2025.22.002

PDF(753 KB)

Packaging Engineering ›› 2025, Vol. 46 ›› Issue (22) : 9-15. DOI: 10.19554/j.cnki.1001-3563.2025.22.002

Special Subject: Product Design, Manufacturing and Service Coor曲nation Optimization Driven by Data and Model Int，电ra世on

Design and Implementation of AR Visual Monitoring Systems for Industrial Manipulators

WANG Yan^*

Author information +

History +

Abstract

To address the issues of non-intuitive information and weak interactivity in the remote monitoring of industrial manipulators, the work aims to develop a real-time monitoring system based on augmented reality to achieve precise superimposition and integration of virtual information with the real manipulators, thereby enhancing operation and maintenance efficiency. The ORB algorithm was adopted for rapid extraction and matching of image feature points, and combined with the LK optical flow method to improve tracking robustness. By analyzing the conversion relationship between camera imaging model and coordinate system, the camera pose was solved to complete three-dimensional tracking registration. Finally, the virtual model of the manipulator's end tool was superimposed onto the real video stream, achieving an augmented reality effect of virtual and real integration. This technical solution effectively enhances the intuitiveness and interactivity of monitoring, providing reliable technical support for the intelligent operation and maintenance of industrial equipment, and achieving the expected research goals.

Key words

augmented reality / 3D registration / feature matching

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

WANG Yan. Design and Implementation of AR Visual Monitoring Systems for Industrial Manipulators[J]. Packaging Engineering. 2025, 46(22): 9-15 https://doi.org/10.19554/j.cnki.1001-3563.2025.22.002

References

[1] PALMARINI R, ERKOYUNCU J A, ROY R, et al.A Systematic Review of Augmented Reality Applications in Maintenance[J]. Robotics and Computer-Integrated Manufacturing, 2018(49): 215-228.
[2] 刘一帆, 吕建峰, 于忠良. AR技术在航天领域中的应用综述[J]. 计算机仿真, 2022, 39(12): 1-6.
LIU Y F, LYU J F, YU Z L.Review on Application of AR Technology in Aerospace Field[J]. Computer Simulation, 2012, 39(12): 1-6.
[3] NEB A, BRANDT D, RAUHÖFT G, et al. A Novel Approach to Generate Augmented Reality Assembly Assistance Automatically from CAD Models[J]. Procedia CIRP, 2021(104): 68-73.
[4] 蒋勇, 朱力量. 5G技术在工业物联网中应用研究[J]. 中国新通信, 2023, 25(2): 72-74.
JIANG Y, ZHU L L.Research on Application of 5G Technology in Industrial Internet of Things[J]. China New Communications, 2023, 25(2): 72-74.
[5] 方维岚, 吴奕龙, 陈翀. 基于AR/MR技术的智能穿戴设备在工业巡检中的应用[J]. 数字技术与应用, 2021, 39(7): 150-152.
FANG W L, WU Y L, CHEN C.Smart Wear Based on AR/MR Technology Equipment in Industrial Inspection Application[J]. Journal of Digital Technology and Application, 2021, 33(7):150-152.
[6] 刘强. 智能制造概论[M]. 北京: 机械工业出版社, 2021.
LIU Q.Introduction to Intelligent Manufacturing[M]. Beijing: China Machine Press, 2021.
[7] FU H H, YANG H M, CHEN C Y, et al.3D Registration Based on V-SLAM and Application in Augmented Reality[J]. Journal of Computational Methods in Sciences and Engineering, 2022, 22(3): 911-923.
[8] 王祯. 基于增强现实的船体分段辅助装配系统研究[D]. 大连: 大连理工大学, 2020.
WANG Z.Research on Auxiliary Assembly System of Ship Hull Segmentation Based on Augmented Reality[D]. Dalian: Dalian University of Technology, 2020.
[9] MARCHAND E, UCHIYAMA H, SPINDLER F.Pose Estimation for Augmented Reality: A Hands-on Survey[J]. IEEE Transactions on Visualization and Computer Graphics, 2015, 22(12): 2633-2651.
[10] RUBLEE E, RABAUD V, KONOLIGE K, et al.ORB: An Efficient Alternative to SIFT or SURF[C]//Proceedings of the 2011 International Conference on Computer Vision. Paris: IEEE, 2011: 2564-2571.
[11] ROSTEN E, DRUMMOND T.Machine Learning for High-speed Corner Detection[C]//Proceedings of the Computer Vision-ECCV 2006: 9th European Conference on Computer Vision, Graz: Springer, 2006: 430-443.
[12] BAY H, TUYTELAARS T, VAN GOOL L.Surf: Speeded up Robust Features[C]//Proceedings of the Computer Vision-ECCV 2006: 9th European Conference on Computer Vision, Graz: Springer, 2006: 404-417.
[13] LOWE D G.Distinctive Image Features from Scale-invariant Keypoints[J]. International Journal of Computer Vision, 2004(60): 91-110.
[14] 贾丹. 摄像机现场标定算法研究[D]. 哈尔滨: 哈尔滨工程大学, 2007.
JIA D.Research on Camera Field calibration Algorithm[D]. Harbin: Harbin Engineering University, 2007.
[15] 任霞. 基于无人机视频的运动目标检测与跟踪方法研究[D]. 成都: 电子科技大学, 2020.
REN X.Research on Moving Target Detection and Tracking Method Based on UAV Video[D]. Chengdu: University of Electronic Science and Technology of China, 2020.
[16] WANG Z B, ONG S K, NEE A.Augmented Reality Aided Interactive Manual Assembly Design[J]. The International Journal of Advanced Manufacturing Technology, 2013(69): 1311-1321.
[17] 徐迟. 增强装配系统的虚实融合技术研究[D]. 武汉: 华中科技大学, 2007.
XU C.Research on Virtual-Real Fusion Technology of Enhanced Assembly System[D]. Wuhan: Huazhong University of Science and Technology, 2007.
[18] 魏巍, 冯蓬勃, 陈峥廷, 等. 增强现实辅助装配技术综述[J]. 包装工程, 2021,42(14): 108-118.
WEI W, FENG P B, CHEN Z T, et al.Review of Augmented Reality Assisted Assembly Technology[J]. Packaging Engineering, 2021, 42(14): 108-118.