Review on User Experience Evaluation of Mobile Generative AI Products

ZHU Xiaoyang; WANG Wei; YANG Yijing; DU Le

doi:10.19554/j.cnki.1001-3563.2025.24.009

PDF(893 KB)

Packaging Engineering ›› 2025, Vol. 46 ›› Issue (24) : 100-114. DOI: 10.19554/j.cnki.1001-3563.2025.24.009

Industrial Design

Review on User Experience Evaluation of Mobile Generative AI Products

ZHU Xiaoyang¹, WANG Wei^1,2,3,*, YANG Yijing¹, DU Le⁴

Author information +

History +

Abstract

The work aims to review the relevant research on user experience evaluation of mobile generative AI products, so as to summarize current research state and shortcomings in this field, forecast its development trends and optimization paths, and provide references for building a systematic evaluation system. Using literature research and content analysis methods, this study organized and analyzed information from three perspectives of evaluation objects, evaluation methods, and evaluation metrics. It critically reviewed the problems in existing research, explored development directions based on industry research progress and related literature, and prospected the feasible future optimization path. Current research in this area is relatively limited, with gaps in certain application domains and modalities, and systematic and standard evaluation frameworks and method systems have not been established. With the expansion of human-computer interaction forms, the application of multi-modal transformation, and the rise of open domain dialogue, the user experience evaluation of mobile generative AI products will become more complex, requiring the comprehensive use of various classical evaluation methods and continuous exploration of automated evaluation. The evaluation indicators will also develop in the direction of greater precision, multi-level structures, and multiple scenarios. Finally, looking forward to user experience evaluation systems for AI systems in future disruptive transformations, the study points out the contradiction between the future and the present, the system layer and the application layer. It is necessary to build a flexible, scalable and adaptable evaluation system.

Key words

generative AI / user experience evaluation / mobile product / evaluation metrics

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

ZHU Xiaoyang, WANG Wei, YANG Yijing, DU Le. Review on User Experience Evaluation of Mobile Generative AI Products[J]. Packaging Engineering. 2025, 46(24): 100-114 https://doi.org/10.19554/j.cnki.1001-3563.2025.24.009

References

[1] DASBOROUGH M T.Awe-Inspiring Advancements in AI: The Impact of ChatGPT on the Field of Organizational Behavior[J]. Journal of Organizational Behavior, 2023, 44(2): 177-179.
[2] MEDIATEK. MediaTek携手生态伙伴联合发布《生成式AI手机产业白皮书》, 共同定义生成式AI手机[EB/OL]. (2024-05-07)[2024-08-11]. https://developer.mediatek.com/ai/664efa379ae7e51a90135fcb.html.
MEDIATEK. Mediatek and Eco-Partners Jointly Released the Generative Ai Mobile Phone Industry White Paper to Jointly Define Generative Ai Mobile Phones[EB/OL]. (2024-05-07)[2024-08-11]. https://developer.mediatek.com/ai/664efa379ae7e51a90135fcb.html.
[3] Technical Committee ISO/TC 159/SC 4, Ergonomics of Human-System Interaction. ISO9241-210:2019, Ergo Nomics of Human-System Interaction-Part 210: Human Centred Design for Interactive Systems: ISO 9241-210: 2019[S]. Switzerland: International Organization for Standardization (ISO), 2019.
[4] RODDEN K, HUTCHINSON H, FU X.Measuring the User Experience on a Large Scale: User-Centered Metrics for Web Applications[C]//Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Atlanta: ACM, 2010: 2395-2398.
[5] 杨焕, 鲍坤坊. 移动应用程序的用户体验度量与评价指标体系研究[J]. 装饰, 2021(7): 101-105.
YANG H, BAO K F.Research on User Experience Measurement and Evaluation Index System of Mobile Applications[J]. Zhuangshi, 2021(7): 101-105.
[6] 华为. 应用流畅体验设计[EB/OL]. (2025-04-24)[2025-06-25]. https://developer.huawei.com/consumer/cn/doc/best-practices-V5/bpta-smooth-application-design-V5#section1593016361909.
HUAWEI. Application smooth experience design[EB/OL]. (2025-04-24)[2025-06-25]. https://developer.huawei.com/consumer/cn/doc/best-practices-V5/bpta-smooth-application-design-V5#section1593016361909.
[7] 国家市场监督管理总局, 国家标准化管理委员会. 移动终端人-系统交互工效学触控界面感知流畅性: GB/T 42396—2023[S]. 北京: 中国标准出版社, 2023.
State Administration for Market Regulation, Standardization Administration of the People's Republic of China. Ergonomics of Human-System Interaction in Mobile Terminal—Perceived Fluency of Touch Interface: GB/T 42396—2023[S]. Beijing: Standards Press of China, 2023.
[8] LAUGWITZ B, HELD T, SCHREPP M.Construction and Evaluation of a User Experience Questionnaire[C]//HCI and Usability for Education and Work. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008: 63-76.
[9] BANGOR A, KORTUM P T, MILLER J T.An Empirical Evaluation of the System Usability Scale[J]. International Journal of Human-Computer Interaction, 2008, 24(6): 574-594.
[10] 罗仕鉴, 郭和睿, 沈诚仪, 等. 智能产品用户体验设计评价研究现状与进展[J]. 计算机集成制造系统, 2024, 30(6): 1919-1935.
LUO S J, GUO H R, SHEN C Y, et al.Status and Progress of Research on User Experience Design Evaluation for Intelligent Products[J]. Computer Integrated Manufacturing Systems, 2024, 30(6): 1919-1935.
[11] AMERSHI S, WELD D, VORVOREANU M, et al.Guidelines for Human-AI Interaction[C]//Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Glasgow Scotland Uk. ACM, 2019: 1-13.
[12] WEISZ J D, HE J, MULLER M, et al.Design Principles for Generative AI Applications[C]//Proceedings of the CHI Conference on Human Factors in Computing Systems. Honolulu: ACM, 2024: 1-22.
[13] GOOGLE PAIR. People+ AI Guidebook[EB/OL]. (2019-05-08)[2025-06-25]. https://pair.withgoogle.com/guidebook/.
[14] MOONFOX.2023年AIGC应用app智能化评测报告[EB/OL]. (2024-03-14)[2024-08-11]. https://www.moonfox.cn/insight/report/1386.
MOONFOX.2023 Aigc Application App Intelligent Evaluation Report[EB/OL]. (2024-03-14)[2024-08-11]. https://www.moonfox.cn/insight/report/1386.
[15] LEHMANN F, MARKERT N, DANG H, et al.Suggestion Lists Vs. Continuous Generation: Interaction Design for Writing with Generative Models on Mobile Devices Affect Text Length, Wording and Perceived Authorship[C]//Mensch und Computer 2022. Darmstadt: ACM, 2022: 192-208.
[16] WANG P J, LEE Y C, CHEN U D, et al.NotiSummary: Exploring the Potential of AI-Driven Text Summarization on Smartphone Notification Management[C]//Adjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing. Cancun: ACM, 2023: 113-117.
[17] WU R L, YU C, PAN X L, et al.MindShift: Leveraging Large Language Models for Mental-States-Based Problematic Smartphone Use Intervention[C]//Proceedings of the CHI Conference on Human Factors in Computing Systems. Honolulu: ACM, 2024: 1-24.
[18] WEBER C J, BURGKART S, ROTHE S.Wr-AI-Ter: Enhancing Ownership Perception in AI-Driven Script Writing[C]//ACM International Conference on Interactive Media Experiences. Stockholm: ACM, 2024: 145-156.
[19] LIN S S, WARNER J, ZAMFIRESCU-PEREIRA J D, et al. Rambler: Supporting Writing with Speech via LLM-Assisted Gist Manipulation[C]//Proceedings of the CHI Conference on Human Factors in Computing Systems. Honolulu: ACM, 2024: 1-19.
[20] CAI R Z, JANAKA N, CHEN Y, et al.PANDALens: Towards AI-Assisted In-Context Writing on OHMD during Travels[C]//Proceedings of the CHI Conference on Human Factors in Computing Systems. Honolulu: ACM, 2024: 1-24.
[21] ZHOU C, YAN Z H, RAM A, et al.GlassMail: Towards Personalised Wearable Assistant for On-the-Go Email Creation on Smart Glasses[C]//Designing Interactive Systems Conference. Copenhagen: ACM, 2024: 372-390.
[22] BELDA-MEDINA J, KOKOŠKOVÁ V. Integrating Chatbots in Education: Insights from the Chatbot-Human Interaction Satisfaction Model (CHISM)[J]. International Journal of Educational Technology in Higher Education, 2023, 20(1): 62.
[23] ZHANG D Y, WU J G, FU Z X.From Shy to Fly: Facilitating EFL Learners' Willingness to Communicate with an AI Chatbot and an Intelligent Tutoring System[J]. System, 2024, 127: 103501.
[24] ZHOU Q, HASHIM H, SULAIMAN N A.Supporting English Speaking Practice in Higher Education: The Impact of AI Chatbot-Integrated Mobile-Assisted Blended Learning Framework[J]. Education and Information Technologies, 2025, 30(10): 14629-14660.
[25] NIE J P, SHAO H V, FAN Y A, et al.LLM-Based Conversational AI Therapist for Daily Functioning Screening and Psychotherapeutic Intervention via Everyday Smart Devices[J]. ACM Transactions on Computing for Healthcare, 2025: 3712299.
[26] FAN X Z, WU Z H, YU C, et al.ContextCam: Bridging Context Awareness with Creative Human-AI Image Co-Creation[C]//Proceedings of the CHI Conference on Human Factors in Computing Systems. Honolulu: ACM, 2024: 1-17.
[27] LAWTON T, GRACE K, IBARROLA F J.When Is a Tool a Tool? User Perceptions of System Agency in Human-AI Co-Creative Drawing[C]//Proceedings of the 2023 ACM Designing Interactive Systems Conference. Pittsburgh: ACM, 2023: 1978-1996.
[28] GONZALEZ R, COLLINS J, AZENKOT S, et al.Investigating Use Cases of AI-Powered Scene Description Applications for Blind and Low Vision People[C]//Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '24). New York: ACM, 2024: 1-21.
[29] PARK J, LEE C, CHO E, et al.Enhancing the Podcast Browsing Experience through Topic Segmentation and Visualization with Generative AI[C]//ACM International Conference on Interactive Media Experiences. Stockholm: ACM, 2024: 117-128.
[30] MO W, SINGH A, HOLLOWAY C.From Information Seeking to Empowerment: Using Large Language Model Chatbot in Supporting Wheelchair Life in Low Resource Settings[C]//The 26th International ACM SIGACCESS Conference on Computers and Accessibility. St. John's: ACM, 2024: 21.
[31] SOHN J J, GUO N, CHUNG Y.Sustainable E-Commerce Marketplace: Reshaping Consumer Purchasing Behavior through Generative AI (Artificial Intelligence)[C]//HCI in Business, Government and Organizations. Cham: Springer Nature Switzerland, 2024: 254-269.
[32] SU X, FROEHLICH J E, KOH E, et al.SonifyAR: Context-Aware Sound Generation in Augmented Reality[C]//Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology. Pittsburgh PA USA. ACM, 2024: 128.
[33] SCHREPP M, HINDERKS A, THOMASCHEWSKI J.Design and Evaluation of a Short Version of the User Experience Questionnaire (UEQ-S)[J]. International Journal of Interactive Multimedia and Artificial Intelligence, 2017, 4(6): 103.
[34] VENKATESH, MORRIS, DAVIS, et al. User Acceptance of Information Technology: Toward a Unified View[J]. MIS Quarterly, 2003, 27(3): 425.
[35] JIN Y C, CHEN L, CAI W L, et al.CRS-Que: A User-Centric Evaluation Framework for Conversational Recommender Systems[J]. ACM Transactions on Recommender Systems, 2024, 2(1): 1-34.
[36] ALSANOUSI B, ALBESHER A S, DO H, et al.Investigating the User Experience and Evaluating Usability Issues in AI-Enabled Learning Mobile Apps: An Analysis of User Reviews[J]. International Journal of Advanced Computer Science and Applications, 2023, 14(6): 18-29.
[37] ALABDULJABBAR R.User-Centric AI: Evaluating the Usability of Generative AI Applications through User Reviews on App Stores[J]. PeerJ Computer Science, 2024, 10: e2421.
[38] RESE A, TRÄNKNER P. Perceived Conversational Ability of Task-Based Chatbots - which Conversational Elements Influence the Success of Text-Based Dialogues?[J]. International Journal of Information Management, 2024, 74: 102699.
[39] FONTANA DE VARGAS M, YU C, SHANE H C, et al. Co-Designing QuickPic: Automated Topic-Specific Communication Boards from Photographs for AAC- Based Language Instruction[C]//Proceedings of the CHI Conference on Human Factors in Computing Systems. Honolulu: ACM, 2024: 1-16.
[40] LIEB A, GOEL T.Student Interaction with NewtBot: An LLM-As-Tutor Chatbot for Secondary Physics Education[C]//Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. Honolulu: ACM, 2024: 1-8.
[41] GAO W W, DU K X, LUO Y J, et al.EasyAsk: An In-App Contextual Tutorial Search Assistant for Older Adults with Voice and Touch Inputs[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2024, 8(3): 120-145.
[42] NEPAL S, PILLAI A, CAMPBELL W, et al.MindScape Study: Integrating LLM and Behavioral Sensing for Personalized AI-Driven Journaling Experiences[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2024, 8(4): 186.
[43] RAVISHAN K, SZABÓ D, BERKEL N V, et al.From Voice to Value: Leveraging AI to Enhance Spoken Online Reviews on the Go[C]//Proceedings of the International Conference on Mobile and Ubiquitous Multimedia. New York: Association for Computing Machinery, 2024: 351-364.
[44] ZHOU J J, HU Y H.Beyond Words: Infusing Conversational Agents with Human-Like Typing Behaviors[C]//ACM Conversational User Interfaces 2024. Luxembourg Luxembourg. ACM, 2024: 24.
[45] BELZ J H, WEILKE L M, WINTER A, et al.Story- Driven: Exploring the Impact of Providing Real-Time Context Information on Automated Storytelling[C]//Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology. Pittsburgh PA:ACM, 2024: 73.
[46] ARCE-URRIZA M, CHOCARRO R, CORTIÑAS M, et al. From Familiarity to Acceptance: The Impact of Generative Artificial Intelligence on Consumer Adoption of Retail Chatbots[J]. Journal of Retailing and Consumer Services, 2025, 84: 104234.
[47] YEH P L, KUO W C, TSENG B L, et al.Does the AI-Driven Chatbot Work? Effectiveness of the Woebot App in Reducing Anxiety and Depression in Group Counseling Courses and Student Acceptance of Technological Aids[J]. Current Psychology, 2025, 44(9): 8133-8145.
[48] KHAN F, TULON T A, MASRUR N, et al.Development and Evaluation of ShasthoBondhu: MHealth App for Guiding Emergency Remote Healthcare with Wearable IoT and AI Fusion[C]//2024 Advances in Science and Engineering Technology International Conferences (ASET). Abu Dhabi: IEEE, 2024: 1-10.
[49] MUGHAL S F, AAMIR S, SAMAD A, et al.Mai: A Transformer Based Domain Specific Chatbot for Menstrual Health[J]. ACM Journal on Responsible Computing, 2025, 2(1): 1-32.
[50] YANG B F, JIANG S Y, XU L L, et al.DrHouse: An LLM-Empowered Diagnostic Reasoning System through Harnessing Outcomes from Sensor Data and Expert Knowledge[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2024, 8(4): 1-29.
[51] GRIMM V, RUBART J.Authoring Educational Hypercomics Assisted by Large Language Models[C]//Proceedings of the 35th ACM Conference on Hypertext and Social Media. Poznan: ACM, 2024: 88-97.
[52] JUNG G, CHOI S, KANG Y J, et al.MyListener: An AI-Mediated Journaling Mobile Application for Alleviating Depression and Loneliness Using Contextual Data[C]//Companion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing. Melbourne: ACM, 2024: 137-141.
[53] KJAERULFF S B, PEDERSEN S B, SIGVARDSEN T J, et al.Exploring VUI-Supported Mindfulness Techniques for Smoking Cessation[C]//ACM Conversational User Interfaces 2024. Luxembourg: ACM, 2024: 1-11.
[54] KIM D, RAVI P, WILLIAMS R, et al.App Planner: Utilizing Generative AI for Design Thinking in K-12 Mobile App Development Education[J]. Journal of Educational Technology & Society, 2024, 27(4): 315-332.
[55] MAURICIO C, DOMINGUES G, PADUA I, et al.What if HMDS could Generate Audio Descriptions of Real Scenes for Visually Impaired People?[C]//Symposium on Virtual and Augmented Reality. Manaus: ACM, 2024: 203-212.
[56] D P K, D A, J I C, et al.WRISTVIEW: Augmented Reality and Generative AI Integration for Enhanced Online Shopping Experiences[C]//2024 8th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud)(I-SMAC). Kirtipur: IEEE, 2024: 1115-1120.
[57] SUEN H Y, HUNG K E.Building Trust in Automatic Video Interviews Using Various AI Interfaces: Tangibility, Immediacy, and Transparency[J]. Computers in Human Behavior, 2023, 143: 107713.
[58] FLOHR L A, KALINKE S, KRÜGER A, et al. Chat or Tap? - Comparing Chatbots with 'Classic' Graphical User Interfaces for Mobile Interaction with Autonomous Mobility-on-Demand Systems[C]//Proceedings of the 23rd International Conference on Mobile Human- Computer Interaction. Toulouse: ACM, 2021: 1-13.
[59] KLEIN A M, KÖLLN K, DEUTSCHLÄNDER J, et al. Design and Evaluation of Voice User Interfaces: What should One Consider?[M]//Design, Operation and Evaluation of Mobile Communications. Cham: Springer Nature Switzerland, 2023: 167-190.
[60] 量子位. 中国AIGC最值得关注企业&产品榜单揭晓！首份应用全景图谱发布[EB/OL]. (2024-04-17)[2024- 08-15]. https://news.qq.com/rain/a/20240417A03UMV00.
QBITAI. China AIGC Most Noteworthy Enterprises & Products List Announced! The First Application Panorama Map Is Released[EB/OL]. (2024-04-17)[2024- 08-15]. https://news.qq.com/rain/a/20240417A03UMV00.
[61] HUANG M L, ZHU X Y, GAO J F.Challenges in Building Intelligent Open-Domain Dialog Systems[J]. ACM Transactions on Information Systems, 2020, 38(3): 1-32.
[62] CHEN S, CHENG H C, SITU J, et al.Towards Inclusive Video Commenting: Introducing Signmaku for the Deaf and Hard-of-Hearing[C]//Proceedings of the CHI Conference on Human Factors in Computing Systems. Honolulu: ACM, 2024: 1-18.
[63] LIU V, LONG T, RAW N, et al. Generative Disco: Text-to-Video Generation for Music Visualization[EB/OL]. (2023-09-28)[2025-11-28]. https://arxiv.org/abs/2304.08551.
[64] LOUIE R, COENEN A, HUANG C Z, et al.Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models[C]//Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Honolulu: ACM, 2020: 1-13.
[65] HOLMES S, MOORHEAD A, BOND R, et al.Usability Testing of a Healthcare Chatbot: Can we Use Conventional Methods to Assess Conversational User Interfaces?[C]//Proceedings of the 31st European Conference on Cognitive Ergonomics. Belfast: ACM, 2019: 207-214.
[66] PARASKEVAS T, KONSTANTA I, GERMANAKOS P, et al.Measuring the Desirability of an Intelligent Advisor for Predicting the Perceived Quality of News Articles[C]//Proceedings of the 2nd International Conference of the ACM Greek SIGCHI Chapter. Athens: ACM, 2023: 1-8.
[67] JUNG J, SEO H, JUNG S, et al.Interactive User Interface for Dialogue Summarization[C]//Proceedings of the 28th International Conference on Intelligent User Interfaces. Sydney: ACM, 2023: 934-957.
[68] MENDES SAMAGAIO Á, LOPES CARDOSO H, RIBEIRO D.A Chatbot for Recipe Recommendation and Preference Modeling[C]//Progress in Artificial Intelligence. Cham: Springer International Publishing, 2021: 389-402.
[69] FAHN V, RIENER A.Time to Get Conversational: Assessment of the Potential of Conversational User Interfaces for Mobile Banking[M]. Proceedings of Mensch und Computer 2021. 2021: 34-43.
[70] KLEIN A M, HINDERKS A, SCHREPP M, et al.Measuring User Experience Quality of Voice Assistants[C]//2020 15th Iberian Conference on Information Systems and Technologies (CISTI). Seville: IEEE, 2020: 1-4.
[71] SILVA F A, SHOJAEI A S, BARBOSA B.Chatbot- Based Services: A Study on Customers' Reuse Intention[J]. Journal of Theoretical and Applied Electronic Commerce Research, 2023, 18(1): 457-474.
[72] CHEN J S, LE T T Y, FLORENCE D. Usability and Responsiveness of Artificial Intelligence Chatbot on Online Customer Experience in E-Retailing[J]. International Journal of Retail & Distribution Management, 2021, 49(11): 1512-1531.
[73] BUBAŠ G, ČIŽMEŠIJA A, KOVAČIĆ A. Development of an Assessment Scale for Measurement of Usability and User Experience Characteristics of Bing Chat Conversational AI[J]. Future Internet, 2024, 16(1): 4.
[74] CHERRY E, LATULIPE C.Quantifying the Creativity Support of Digital Tools through the Creativity Support Index[J]. ACM Transactions on Computer-Human Interaction, 2014, 21(4): 1-25.
[75] GERO K I, LIU V, CHILTON L.Sparks: Inspiration for Science Writing Using Language Models[C]//Designing Interactive Systems Conference. Melbourne: ACM, 2022: 1002-1019.
[76] XU H R, CHEN S Y, YING Z.Magical Brush: A Symbol-Based Modern Chinese Painting System for Novices[C]//Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. Hamburg: ACM, 2023: 1-14.
[77] KISELEVA J, WILLIAMS K, HASSAN AWADALLAH A, et al.Predicting User Satisfaction with Intelligent Assistants[C]//Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. Pisa: ACM, 2016: 45-54.
[78] 葛燕, 陈亚楠, 刘艳芳, 等. 电生理测量在用户体验中的应用[J]. 心理科学进展, 2014, 22(6): 959-967.
GE Y, CHEN Y N, LIU Y F, et al.Electrophysiological Measures Applied in User Experience Studies[J]. Advances in Psychological Science, 2014, 22(6): 959-967.
[79] FEDERICI S, MELE M L, BRACALENTI M, et al.A Chatbot Solution for EGLU-Box Pro: The Usability Evaluation Platform for Italian Public Administrations[C]//Human-Computer Interaction. Theory, Methods and Tools. Cham: Springer International Publishing, 2021: 268-279.
[80] LEE S, RYU H, PARK B, et al.Using Physiological Recordings for Studying User Experience: Case of Conversational Agent-Equipped TV[J]. International Journal of Human-Computer Interaction, 2020, 36(9): 815-827.
[81] LIM G, PERRAULT S T. Rapid AIdeation: Generating Ideas With the Self and in Collaboration With Large Language Models[J]. arXiv preprint arXiv:2403.12928, 2024.
[82] LECOCQ X, WARNIER V, DEMIL B, et al.Using Artificial Intelligence (AI) Generative Technologies for Business Model Design with IDEATe Process: A Speculative Viewpoint[J]. Journal of Business Models, 2024, 12(1): 21-35.
[83] FIRDAUS M, THANGAVELU N, EKBAL A, et al.I Enjoy Writing and Playing, Do You? : A Personalized and Emotion Grounded Dialogue Agent Using Generative Adversarial Network[J]. IEEE Transactions on Affective Computing, 2023, 14(3): 2127-2138.
[84] 张妍, 赵宇翔, 吴大伟, 等. 人智交互情境中用户对生成式人工智能的心智感知及反应研究[J]. 情报理论与实践, 2024, 47(8): 140-149.
ZHANG Y, ZHAO Y X, WU D W, et al.Study of Users’ Mind Perceptions and Responses to Generative AI in Human-AI Interaction Context[J]. Information Studies (Theory & Application), 2024, 47(8): 140-149.
[85] 王清菊, 吴萍, 范洁云, 等. 智能时代用户体验特点及其测量方法[J]. 包装工程, 2022, 43(4): 142-146.
WANG Q J, WU P, FAN J Y, et al.Characteristics and Measurement Methods of User Experience in Intelligent Era[J]. Packaging Engineering, 2022, 43(4): 142-146.
[86] BUCHTA K, WÓJCIK P, PELC M, et al. NUX IVE - a Research Tool for Comparing Voice User Interface and Graphical User Interface in VR[C]//2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). Christchurch: IEEE, 2022: 982-983.
[87] HU Y, WANG X, WANG Y, et al. AUITestAgent: Automatic Requirements Oriented GUI Function Testing[EB/OL]. (2024-07-12)[2025-11-28]. https://arxiv.org/abs/2407.09018.
[88] 陈晓皎, 束云峰, 汪睿涵, 等. 大语言模型驱动的UI评估系统[J]. 图学学报, 2024, 45(6):1178-1187.
CHEN X J, SHU Y F, WANG R H, et al.Large language model-driven UI evaluation system[J]. Journal of Graphics, 2024, 45(6): 1178-1187.
[89] ALSANOUSI B, LUDI S, DO H.LIME-Aided Automated Usability Issue Detection from User Reviews: Leveraging LLMs for Enhanced User Experience Analysis[C]//ACHI 2024:The Seventeenth International Conference on Advances in Computer-Human Interactions. Barcelona: IARIA, 2024: 200-209.
[90] WILL S, CHRISTOPHER R, DAVID D, et al. Can ChatGPT Replace UX Researchers? An Empirical Analysis of Comment Classifications[EB/OL]. (2023- 05-23)[2025-35-20]. https://measuringu.com/classification-agreement-between-ux-researchers-and-chatgpt/.
[91] LIN Y C, NEVILLE J, STOKES J W, et al.Interpretable User Satisfaction Estimation for Conversational Systems with Large Language Models[C]//Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. Bangkok: Association for Computational Linguistics, 2024: 11100-11115.
[92] MOUSSAWI S, KOUFARIS M.Perceived Intelligence and Perceived Anthropomorphism of Personal Intelligent Agents: Scale Development and Validation[C]//Proceedings of the 52nd Hawaii International Conference on System Sciences. Maui: ScholarSpace, 2019: 115-124.
[93] SVIKHNUSHINA E, PU P.Key Qualities of Conversational Chatbots - the PEACE Model[C]//26th International Conference on Intelligent User Interfaces. College Station: ACM, 2021: 520-530.
[94] SHANG J, ZHENG Z, YING X, et al. AI-native Memory: A Pathway from LLMs Towards AGI[EB/OL]. (2024-08-28)[2025-06-25]. https://arxiv.org/abs/2406.18312.
[95] KNOB P R, PIZZOL N D, MUSSE S R, et al.Arthur and Bella: Multi-Purpose Empathetic AI Assistants for Daily Conversations[J]. The Visual Computer, 2024, 40(4): 2933-2948.
[96] GOYAL R, KUMAR P, SINGH V P.A Systematic Survey on Automated Text Generation Tools and Techniques: Application, Evaluation, and Challenges[J]. Multimedia Tools and Applications, 2023, 82(28): 43089-43144.
[97] ZHU W, BHAT S.GRUEN for Evaluating Linguistic Quality of Generated Text[C]//Findings of the Association for Computational Linguistics: EMNLP 2020. Online: Association for Computational Linguistics, 2020: 94-108.
[98] TORRANCE E P. Torrance Tests of Creative Thinking[EB/OL]. (1966)[2025-06-25]. https://psycnet.apa.org/doiLanding?doi=10.1037%2Ft05532-000.
[99] CHAKRABARTY T, LABAN P, AGARWAL D, et al.Art or Artifice? Large Language Models and the False Promise of Creativity[C]//Proceedings of the CHI Conference on Human Factors in Computing Systems. Honolulu: ACM, 2024: 1-34.
[100] LYU Y R, WANG X X, LIN R, et al.Communication in Human-AI Co-Creation: Perceptual Analysis of Paintings Generated by Text-to-Image System[J]. Applied Sciences, 2022, 12(22): 11312.
[101] BANDI A, ADAPA P V S R, Yudu Eswar Vinay Pratap Kumar Kuchi. The Power of Generative AI: A Review of Requirements, Models, Input-Output Formats, Evaluation Metrics, and Challenges[J]. Future Internet, 2023, 15(8): 260.
[102] 王俊, 谢青伶, 刘畅. 日常生活情境下用户与生成式人工智能交互行为分析[J]. 图书情报知识, 2025, 42(2): 60-69+93.
WANG J, XIE Q L, LIU C. User's Interaction Behavior with Generative Artificial Intelligence in Daily Life Contexts[J]. Documentation, Information & Knowledge, 2025, 42(2): 60-69+93.
[103] KIM S, YUN S, LEE H, et al.Propile: Probing Privacy Leakage in Large Language Models[C]//Advances in Neural Information Processing Systems 36. New Orleans: Curran Associates, Inc., 2023: 20750-20762.
[104] HARTVIGSEN T, GABRIEL S, PALANGI H, et al.ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Dublin: Association for Computational Linguistics, 2022: 3309-3326.
[105] FRANCESCHELLI G, MUSOLESI M.Copyright in Generative Deep Learning[J]. Data & Policy, 2022, 4: e17.
[106] CHEN C, FU J, LYU L.A Pathway Towards Responsible Ai Generated Content[C]//Proceedings of the Thirty- Second International Joint Conference on Artificial Intelligence. Macao: International Joint Conferences on Artificial Intelligence Organization, 2023: 7033-7038.
[107] WEIDINGER L, RAUH M, MARCHAL N, et al. Sociotechnical Safety Evaluation of Generative Ai Systems[EB/OL]. (2023-10-31)[2025-11-28]. https://arxiv.org/abs/2310.11986.
[108] 洪杰文, 常静宜. 用户关切视角下的“生成式AI” 伦理框架建构[J]. 新闻与写作, 2023(12): 44-55.
HONG J W, CHANG J Y.The Construction of "Generative AI" Ethical Framework from the Perspective of User Concern[J]. News and Writing, 2023(12): 44-55.
[109] 钉钉. 重磅！IDC、钉钉联合发布2024 AIGC 应用层十大趋势[EB/OL]. (2024-01-03)[2025-04-12]. https://ow.dingtalk.com/article/SWA00DRAYduQ6zTwhWRI9Shjps7CCj5LMWgiSS.
DINGTALK. Blockbuster! Idc, Dingtalk Jointly Release2024 Aigc Application Layer Top 10 Trends[EB/OL]. (2024-01-03)[2025-04-12]. https://ow.dingtalk.com/article/SWA00DRAYduQ6zTwhWRI9Shjps7CCj5LMWgiSS.
[110] HERNÁNDEZ-ORALLO J. Evaluation in Artificial Intelligence: From Task-Oriented to Ability-Oriented Measurement[J]. Artificial Intelligence Review, 2017, 48(3): 397-447.
[111] WANG X, JIANG L, HERNANDEZ-ORALLO J, et al. Evaluating General-Purpose AI with Psychometrics[EB/OL]. (2023-12-29)[2025-11-28]. https://arxiv.org/abs/2310.16379.
[112] YAO J, YI X, WANG X, et al. From Instructions to Intrinsic Human Values--A Survey of Alignment Goals for Big Models[EB/OL]. (2023-09-04)[2025-11-28]. https://arxiv.org/abs/2308.12014.
[113] WANG X, DUAN S, YI X, et al.On the Essence and Prospect: An Investigation of Alignment Approaches for Big Models[C]//Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence. Jeju: International Joint Conferences on Artificial Intelligence Organization, 2024: 8308-8316.