基于人工智能的沉浸式设计综述

刘莹莹

doi:10.19554/j.cnki.1001-3563.2025.20.001

PDF(1662 KB)

包装工程（设计栏目） ›› 2025, Vol. 46 ›› Issue (20) : 1-15. DOI: 10.19554/j.cnki.1001-3563.2025.20.001

工业设计

基于人工智能的沉浸式设计综述

刘莹莹

作者信息 +

Review of AI-Based Immersive Design

LIU Yingying

Author information +

文章历史 +

摘要

目的揭示人工智能技术在沉浸式设计中的显著进展,改进人工智能技术赋能沉浸式设计的生成方式与生成质量。方法基于Web of Science数据库,系统回顾2015—2024年人工智能赋能沉浸式设计领域的最新研究成果,并将文献划分为基础理论、用户体验、技术驱动、场景应用四大类别进行分析。结果重点探讨了计算机视觉、自然语言处理、机器学习与深度学习等人工智能技术如何通过多模态交互、个性化与智能化、实时反馈与动态调整等路径提升沉浸感的核心影响因素,并在娱乐与游戏、教育与培训、医疗与健康领域协同作用,显著增强用户的临场感、交互感以及系统的可用性,提出一套从“理论基础-用户驱动-技术实现-应用验证”的系统化技术路径与方法论,全景式为设计师在人工智能飞速发展下应对复杂的沉浸式设计需求提供方向。结论未来研究应重点探索多模态AI交互技术与智能化交互系统整合下的实时交互设计,提升数据安全性与人工智能系统的可用性,以实现更加人性化、安全和可靠的沉浸式体验,从而进一步增强用户的空间沉浸感与情感认同。

Abstract

The work aims to uncover the substantial progress made in the field of artificial intelligence (AI) within the realm of immersive design, with the goal of refining the methodologies and enhancing the quality of AI-driven immersive environments. Based on a systematic review of the latest research in the WOS database from 2015 to 2024, the literature was categorized into four domains of theoretical foundations, technical approaches, user experience, and application scenarios. The analysis highlighted how AI technologies such as computer vision, natural language processing, machine learning, and deep learning enhanced core factors influencing immersion through multi-modal interaction, personalization and intelligence, real-time feedback, and dynamic adjustment and exerted synergistic effects across various fields, including education and training, healthcare, and entertainment, significantly boosting users' sense of presence, interactivity, and system usability. A systematic technological pathway and methodology, structured around "theoretical foundation-user motivation-technical implementation-application validation" was proposed to provide designers with a comprehensive direction for addressing complex immersive design demands amidst the rapid advancement of AI. Future research should focus on integrating multimodal AI interaction technologies with intelligent systems for real-time interactive design while strengthening data security and system explainability. Such efforts will enable more human-centric, safe, and reliable immersive experiences, further enhancing users' spatial immersion and emotional engagement.

导出引用1

刘莹莹. 基于人工智能的沉浸式设计综述[J]. 包装工程. 2025, 46(20): 1-15 https://doi.org/10.19554/j.cnki.1001-3563.2025.20.001

LIU Yingying. Review of AI-Based Immersive Design[J]. Packaging Engineering. 2025, 46(20): 1-15 https://doi.org/10.19554/j.cnki.1001-3563.2025.20.001

中图分类号： TB472

参考文献

[1] AGRAWAL S, SIMON A, BECH S, et al.Defining Immersion: Literature Review and Implications for Research on Audiovisual Experiences[J]. Journal of the Audio Engineering Society, 2020, 68(6): 404-417.
[2] MURRAY J.H. Hamlet on the Holodeck: The Future of Narrative in Cyberspace[M]. New York: The Free Press, 1997.
[3] SAFFO D, BARTOLOMEO S D, CRNOVRSANIN T, et al. Unraveling the Design Space of Immersive Analytics: A Systematic Review[J]. IEEE Transactions on Visualization and Computer Graphics, 2023, PP: PP.
[4] YE S N, CHU X T, WU Y C.A Survey on Immersive Visualization[J]. Journal of Computer-Aided Design & Computer Graphics, 2021, 33(4): 497-507.
[5] Huang J, Han D, Chen Y, et al.A Survey on Human- Computer Interaction in Mixed Reality[J]. Journal of Computer-Aided Design & Computer Graphics, 2016, 28(6): 869-880.
[6] MCCARTHY J, MINSKY M L, ROCHESTER N, SHANNON C. E. A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence[EB/OL]. (1955-08-31) [2025-04-15]. http://www-formal.stanford. edu/jmc/history/dartmouth/dartmouth.html.
[7] HUNT E B.Artificial intelligence[M]. New York: Academic Press, 2014.
[8] RUSSELL S J, NORVIG P.Artificial Intelligence: A Modern Approach[M]. Essex: Pearson Education Limited, 2016.
[9] TURING A M.The Essential Turing: Seminal Writings in Computing, Logic, Philosophy, Artificial Intelligence, and Artificial Life[M]. Oxford: Oxford University Press, 2004: 433-460.
[10] JIANG L, ZHANG Y T.Ethical Risk and Pathway of AIGC Cross-Modal Content Generation Technology[J]. International Journal of Social Sciences & Humanities (IJSSH), 2024, 9(1): 85-99.
[11] VERGARA VILLEGAS O O, NANDAYAPA M, SOSSA AZUELA J H, et al. Artificial Intelligence for Industry 4.0 in Iberoamerica[J]. Computación y Sistemas, 2021, 25(4): 761-773.
[12] SINGH M, JOSHI M, TYAGI K D, et al.Future Professions in Agriculture, Medicine, Education, Fitness, Research and Development, Transport, and Communication[J]. Topics in Artificial Intelligence Applied to Industry 4 0, 2024: 181-202.
[13] GHESH N, ALEXANDER M, DAVIS A.The Artificial Intelligence-Enabled Customer Experience in Tourism: A Systematic Literature Review[J]. Tourism Review, 2024, 79(5): 1017-1037.
[14] WANG D L, XIAO J W, LI J K, et al.Review of Stereo Image Disparity Estimation Methods Based on Depth Learning[J]. Computer Engineering and Applications, 2022, 58(20): 16-27.
[15] LEE L K, WEI X D, CHUI K T, et al.A Systematic Review of the Design of Serious Games for Innovative Learning: Augmented Reality, Virtual Reality, or Mixed Reality?[J]. Electronics, 2024, 13(5): 890.
[16] DE SMEDT Q, WANNOUS H, VANDEBORRE J P.Skeleton-Based Dynamic Hand Gesture Recognition[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Los Alamitos, CA: IEEE Computer Society, 2016: 1-9.
[17] SAHOO J P, PRAKASH A J, PŁAWIAK P, et al. Real- Time Hand Gesture Recognition Using Fine-Tuned Convolutional Neural Network[J]. Sensors, 2022, 22(3): 706.
[18] Chamberlain L. Eye Tracking Methodology; Theory and Practice[J]. Qualitative Market Research, 2007, 10(2): 217-220.
[19] CLAY V, KÖNIG P, KÖNIG S U. Eye Tracking in Virtual Reality[EB/OL]. (2019-01-01)[2025-04-17]. https:// doi.org/10.16910/jemr.12.1.3.
[20] HOU S, JIA C, ZHANG M.Review of Eye Movement-based Interaction Techniques for Virtual Reality Systems[J]. Journal of Computer Applications, 2022, 42(11): 3534.
[21] MCRLEAU-PONTY M.Phenomenologyofperception[M]. Oxfordshire: Tayor and Francis, 1982: 3-25.
[22] 邵志芳. 认知心理学: 理论、实验和应用[M]. 上海: 上海教育出版社, 2006.
SHAO Z F.Cognitive Psychology[M]. Shanghai: Shanghai Education Publishing House, 2006.
[23] 周逵. 沉浸式传播中的身体经验: 以虚拟现实游戏的玩家研究为例[J]. 国际新闻界, 2018, 40(5): 6-26.
ZHOU K.Bodily Experience in Immersive Communication: A Case Study of Virtual Reality Game Players[J]. Chinese Journal of Journalism & Communication, 2018, 40(5): 6-26.
[24] CROATTI A, RICCI A.Developing Agent-Based Pervasive Mixed Reality Systems: The MiRAgE Framework[M]// Advances in Practical Applications of Agents, Multi-Agent Systems, and Complexity: The PAAMS Collection. Cham: Springer International Publishing, 2018: 301-304.
[25] BOTEV J, DREWING K, HAMANN H, et al.ChronoPilot—Modulating Time Perception[C]// 2021 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR). Taichung, Taiwan, China. IEEE, 2021: 215-218.
[26] PARTARAKIS N, ZABULIS X.A Review of Immersive Technologies, Knowledge Representation, and AI for Human-Centered Digital Experiences[J]. Electronics, 2024, 13(2): 269.
[27] LLOBERA J, BOULIC R.A Tool to Design Interactive Characters Based on Embodied Cognition[J]. IEEE Transactions on Games, 2019, 11(4): 311-319.
[28] STUART J, AUL K, STEPHEN A, et al.The Effect of Virtual Human Rendering Style on User Perceptions of Visual Cues[J]. Frontiers in Virtual Reality, 2022, 3: 864676.
[29] KITSON A, PRPA M, RIECKE B E.Immersive Interactive Technologies for Positive Change: A Scoping Review and Design Considerations[J]. Frontiers in Psychology, 2018, 9: 1354.
[30] CUMMINGS J J, BAILENSON J N.How Immersive Is Enough? A Meta-Analysis of the Effect of Immersive Technology on User Presence[J]. Media Psychology, 2016, 19(2): 272-309.
[31] YEO A, KWOK B W J, JOSHNA A, et al. Entering the Next Dimension: A Review of 3D User Interfaces for Virtual Reality[J]. Electronics, 2024, 13(3): 600.
[32] SEMERTZIDIS N, VRANIC-PETERS M, ANDRES J, et al.Neo-Noumena: Augmenting Emotion Communication[C]// Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Honolulu: ACM, 2020: 1-13.
[33] BOWMAN C, FUJITA H, PERIN G.Towards a Knowledge Based Environment for the Cognitive Understanding and Creation of Immersive Visualization of Expressive Human Movement Data[M]// Trends in Applied Knowledge-Based Systems and Data Science. Cham: Springer International Publishing, 2016: 182-192.
[34] MURPHY C, CAREW P J, STAPLETON L.A Human- Centred Systems Manifesto for Smart Digital Immersion in Industry 5.0: A Case Study of Cultural Heritage[J]. Ai & Society, 2024, 39(5): 2401-2416.
[35] DICKINSON P, GERLING K, HICKS K, et al.Virtual Reality Crowd Simulation: Effects of Agent Density on User Experience and Behaviour[J]. Virtual Reality, 2019, 23(1): 19-32.
[36] TAUPIAC J D, RODRIGUEZ N, STRAUSS O.Immercity: A Curation Content Application in Virtual and Augmented Reality[M]// Virtual, Augmented and Mixed Reality: Applications in Health, Cultural Heritage, and Industry. Cham: Springer International Publishing, 2018: 223-234.
[37] SULTANA M, KIM I S, JUNG S K.Deep Matting for AR Based InteriorDesign[C]// Frontiers of Computer Vision. Singapore: Springer Singapore, 2020: 31-42.
[38] LOMBARD M, DITTON T.At the Heart of It All: The Concept of Presence[J]. Journal of Computer-Mediated Communication, 1997, 3(2).
[39] FANG H, GUO J J, WU H.Wearable Triboelectric Devices for Haptic Perception and VR/AR Applications[J]. Nano Energy, 2022, 96: 107112.
[40] ZHOU B, YU A, MENKE J, et al.Real-Time Hand Model Estimation from Depth Images for Wearable Augmented Reality Glasses[C]// 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). Beijing: IEEE, 2019: 269-273.
[41] LI L, JIANG S, SHULL P B, et al.SkinGest: Artificial Skin for Gesture Recognition via Filmy Stretchable Strain Sensors[J]. Advanced Robotics, 2018, 32(21): 1112-1121.
[42] WANG J, ERKOYUNCU J, ROY R.A Conceptual Design for Smell Based Augmented Reality: Case Study in Maintenance Diagnosis[J]. Procedia CIRP, 2018, 78: 109-114.
[43] CAO J, LAM K Y, LEE L H, et al.Mobile Augmented Reality: User Interfaces, Frameworks, and Intelligence[J]. ACM Computing Surveys, 2023, 55(9): 1-36.
[44] LI W T, XUE Z J, LI J Y, et al.The Interior Environment Design for Entrepreneurship Education under the Virtual Reality and Artificial Intelligence-Based Learning Environment[J]. Frontiers in Psychology, 2022, 13: 944060.
[45] HERMAN B, SAMSEL F, BARES A, et al.Printmaking, Puzzles, and Studio Closets: Using Artistic Metaphors to Reimagine the User Interface for Designing Immersive Visualizations[C]// 2020 IEEE VIS Arts Program (VISAP). Salt Lake City, UT, USA. IEEE, 2020: 19-28.
[46] KOO S, KIM J, KIM C, et al.Development of an Augmented Reality Tour Guide for a Cultural Heritage Site[J]. Journal on Computing and Cultural Heritage, 2020, 12(4): 1-24.
[47] XU X H, YU A N, JONKER T R, et al.XAIR: A Framework of Explainable AI in Augmented Reality[C]// Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. Hamburg Germany: ACM, 2023: 1-30.
[48] GU Y, WANG Q, GU W L.The Innovative Application of Visual Communication Design in Modern Art Design[J]. Electronics, 2023, 12(5): 1150.
[49] DOSOVITSKIY A.An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale[J]. arXiv preprint arXiv:2010.11929, 2020.
[50] RAMESH A., PAVLOV M., GOH G., et al.Zero-Shot Text-to-Image Generation[C]// Proceedings of the 38th International Conference on Machine Learning (ICML 2021). Cambridge, MA: PMLR, 2021: 8821-8831.
[51] LI Y, XIE Y, WANG X J, et al.A Fast Method for Large- Scale Scene Data Acquisition and 3D Reconstruction[C]// 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). Beijing: IEEE, 2019: 321-325.
[52] HAZARIKA A, RAHMATI M.Towards an Evolved Immersive Experience: Exploring 5G-and Beyond-Enabled Ultra-Low-Latency Communications for Augmented and Virtual Reality[J]. Sensors, 2023, 23(7): 3682.
[53] LIANG H, DENG S J, CHANG J, et al.Semantic Framework for Interactive Animation Generation and Its Application in Virtual Shadow Play Performance[J]. Virtual Reality, 2018, 22(2): 149-165.
[54] SAJJADI P, BAGHER M M, OLIVER J, et al.Mixed or Virtual: Does Device Type Matter in Human-ECA Interactions[C]// 2020 6th International Conference of the Immersive Learning Research Network (iLRN). San Luis Obispo, CA, USA. IEEE, 2020: 230-235.
[55] XU S Y, WEI Y, ZHENG P, et al.LLM Enabled Generative Collaborative Design in a Mixed Reality Environment[J]. Journal of Manufacturing Systems, 2024, 74: 703-715.
[56] ANDRIST S, BOHUS D, FENIELLO A, et al.Developing Mixed Reality Applications with Platform for Situated Intelligence[C]// 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). Christchurch, New Zealand. IEEE, 2022: 48-50.
[57] XIONG W, WU L, ALLEVA F, et al.The Microsoft 2017 Conversational Speech Recognition System[C]// 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Calgary, AB, Canada. IEEE, 2018: 5934-5938.
[58] ZHANG G L.Design of Virtual Reality Augmented Reality Mobile Platform and Game User Behavior Monitoring Using Deep Learning[J]. International Journal of Electrical Engineering & Education, 2023, 60(2_suppl): 205-221.
[59] KUMAR A.Gamification in Training with Next Generation AI-Virtual Reality, Animation Design and Immersive Technology[J]. Journal of Experimental & Theoretical Artificial Intelligence, 2024, 36(7): 1121-1134.
[60] XIE Z.The Symmetries in Film and Television Production Areas Based on Virtual Reality and Internet of Things Technology[J]. Symmetry, 2020, 12(8): 1377.
[61] ZHU R J, FAN D L, LIN J, et al.Machine-Learning-Assisted Soft Fiber Optic Glove System for Sign Language Recognition[J]. IEEE Robotics and Automation Letters, 2024, 9(2): 1540-1547.
[62] DU H Y.Semantic Communications and Generative Artificial Intelligence for User-Centric Metaverse Services[D]. Singap ore: Nanyang Technological University, 2024.
[63] LU Y.Beyond the Stage: The Evolution and Transformation of Scenography in the Digital Age[D]. Turin: Politecnico di Torino, 2024.
[64] HONG J Y. Innovation and Practice of Virtual Reality Interaction Design in Chinese Medicine Science Museums[EB/OL]. (2024-01-01) [2025-05-17]. https://doi. org/10.18686/aitr.v2i3.4403.
[65] NAWAZ A, AHMAD A.Conversational AI for Cognitive, Emotional, and Social Engagement of Elderly Persons: A Large Language Model-Based Framework[C]// International Joint Conference on Artificial Intelligence. Singapore: Springer Nature Singapore, 2025: 17-28.
[66] CHEN X X, GAO W Z, CHU Y N, et al.Enhancing Interaction in Virtual-Real Architectural Environments: A Comparative Analysis of Generative AI-Driven Reality Approaches[J]. Building and Environment, 2024, 266: 112113.
[67] CARROZZINO M A, GALDIERI R, MACHIDON O M, et al.Do Virtual Humans Dream of Digital Sheep?[J]. IEEE Computer Graphics and Applications, 2020, 40(4): 71-83.
[68] AUGELLO A, CAGGIANESE G, GALLO L.Human Augmentation: An Enactive Perspective[M]// Extended Reality. Cham: Springer International Publishing, 2022: 219-228.
[69] KARACA Y, DERIAS D, SARSAR G. AI-Powered Procedural Content Generation: Enhancing NPC Behaviour for an Immersive Gaming Experience[EB/OL]. (2023-01-01)[2025-05-17]. https://doi.org/10.2139/ssrn. 4663382.
[70] MARELLI D, BIANCO S, CIOCCA G.Designing an AI-Based Virtual Try-on Web Application[J]. Sensors, 2022, 22(10): 3832.
[71] CHANG C Y, CHUANG B I, HSIA C C, et al.Framework Design for Multiplayer Motion Sensing Game in Mixture Reality[M]// MultiMedia Modeling. Cham: Springer International Publishing, 2019: 703-708.
[72] OREA-GINER A, MUÑOZ-MAZÓN A, VILLACÉ- MOLINERO T, et al. Cultural Tourist and User Experience with Artificial Intelligence: A Holistic Perspective from the Industry 5.0 Approach[EB/OL]. (2022-01- 01) [2025-05-17]. https://doi.org/10.1108/JTF-04-2022-0115.
[73] IGRAS-CYBULSKA M, HEKIERT D, CYBULSKI A, et al.Towards Multimodal VR Trainer of Voice Emission and Public Speaking-Work-in-Progress[C]// 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). Shanghai: IEEE, 2023: 355-359.
[74] SATTAR M U, PALANIAPPAN S, LOKMAN A, et al.Motivating Medical Students Using Virtual Reality Based Education[J]. International Journal of Emerging Technologies in Learning (IJET), 2020, 15(2): 160.
[75] ALSHAHRANI K, JAMEEL QURESHI R.Review the Prospects and Obstacles of AIEnhanced Learning Environments: The Role ofChatGPT in Education[J]. International Journal of Modern Education and Computer Science, 2024, 16(4): 71-86.
[76] FAILLA C, CHILÀ P, VETRANO N, et al.Virtual Reality for Autism: Unlocking Learning and Growth[J]. Frontiers in Psychology, 2024, 15: 1417717.
[77] BIAN D X, LIN Z P, LU H, et al.The Application of Extended Reality Technology-Assisted Intraoperative Navigation in Orthopedic Surgery[J]. Frontiers in Surgery, 2024, 11: 1336703.
[78] RÜGER C, MOOSBURNER S, SAUER I M. Extended- Reality-Technologien Zur Unterstützung Chirurgischen Handelns[J]. Der Chirurg, 2020, 91(7): 544-552.
[79] LEE H C, HUANG C L, HO S H, et al.The Effect of a Virtual Reality Game Intervention on Balance for Patients with Stroke: A Randomized Controlled Trial[J]. Games for Health Journal, 2017, 6(5): 303-311.
[80] GRASSO J.Robot-Assisted Surgery[M]// Digital Health. Amsterdam: Elsevier, 2025: 171-186.