Multimodal Interaction Design of Intelligent Cars for the Elderly Based on FTA-CREAM

CAI Yixin; LI Yongfeng; ZHU Liping

doi:10.19554/j.cnki.1001-3563.2026.06.003

PDF(1963 KB)

Packaging Engineering ›› 2026, Vol. 47 ›› Issue (6) : 18-30. DOI: 10.19554/j.cnki.1001-3563.2026.06.003

Industrial Design

Multimodal Interaction Design of Intelligent Cars for the Elderly Based on FTA-CREAM

CAI Yixin, LI Yongfeng^*, ZHU Liping

Author information +

History +

Abstract

The work aims to reduce risks and improve user experience for elderly drivers during multimodal interaction with intelligent cars. A multimodal interaction design method based on FTA (Fault Tree Analysis) and CREAM (Cognitive Reliability and Failure Analysis Method) was proposed for intelligent cars for the elderly. Firstly, a contextual analysis of the multimodal interaction process of the elderly was conducted. Then, a hierarchical task analysis was carried out to explore the generalized performance conditions and contextual influence indices of CREAM to determine the control modes of the subtasks, and obtain the cognitive failure probabilities of the subtasks. Next, a fault tree of multimodal interaction of the elderly was established to derive the minimum cut-off set through qualitative analysis, and the probability of the cognitive failure of the subtasks was taken as the basic event of the fault tree probability to calculate the top event probability of the fault tree, and rank the importance of the basic events. Finally, based on the ranking results, the improvement measures were formulated from three dimensions of cognitive factors, physiological conditions, and technology acceptance to carry out design improvement, and the results were verified. The driving activity load of the elderly was effectively reduced, and the multimodal interaction risk was reduced. The proposed method can effectively assess risks in multimodal interaction between elderly users and intelligent cars and reasonably analyze the causes of erroneous behaviors, thereby improving interaction design and enabling elderly users to achieve a better interaction experience.

Key words

multimodal interaction design / Cognitive Reliability and Error Analysis Method (CREAM) / Fault Tree Analysis (FTA) / intelligent car / the elderly

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

CAI Yixin, LI Yongfeng, ZHU Liping. Multimodal Interaction Design of Intelligent Cars for the Elderly Based on FTA-CREAM[J]. Packaging Engineering. 2026, 47(6): 18-30 https://doi.org/10.19554/j.cnki.1001-3563.2026.06.003

References

[1] 黄建中, 马煜箫, 胡刚钰. 基于健康视角的老年人出行行为研究进展[J]. 科技导报, 2020, 38(7):69-75.
HUANG J Z, MA Y X, HU G Y.A Review of Research on Travel Behaviors of the Elderly from a Health Perspective[J]. Science & Technology Review, 2020, 38(7):69-75.
[2] PADEN B, ČÁP M, YONG S Z, et al. A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles[J]. IEEE Transactions on Intelligent Vehicles, 2016, 1(1):33-55.
[3] 王敏, 孙玉灵. 人机交互领域中面向老年群体的对话智能体研究现状综述[J]. 计算机应用与软件, 2024, 41(3):1-8.
WANG M, SUN Y L.A Review of Current Research on Conversational Intelligences for the Elderly in the Field of human-Computer Interaction[J]. Computer Applications and Software, 2024, 41(3):1-8.
[4] 魏婷. 多模态交互情境下的智能车载产品用户体验设计方法研究[D]. 西安:陕西科技大学, 2023.
WEI T.Research on the Design Method of User Experience of Intelligent Vehicle Products under Multi-Modal Interaction Context[D]. Xi’an:Shaanxi University of Science & Technology, 2023.
[5] KIM M, SEONG E, JWA Y, et al.A Cascaded Multimodal Natural User Interface to Reduce Driver Distraction[J]. IEEE Access, 2020, 8:112969-112984.
[6] ROIDER F, RÜMELIN S, PFLEGING B, et al. Investigating the Effects of Modality Switches on Driver Distraction and Interaction Efficiency in the Car[J]. Journal on Multimodal User Interfaces, 2019, 13(2):89-97.
[7] 王畅, 张岩, 赵晨, 等. 新型全触屏人机交互模式对驾驶风险的影响特性[J]. 科学技术与工程, 2023, 23(22):9673-9679.
WANG C, ZHANG Y, ZHAO C, et al.Influence Characteristics of New Full Touch Screen Human-Machine Interaction Mode on Driving Risk[J]. Science Technology and Engineering, 2023, 23(22):9673-9679.
[8] 蔡萌亚, 王文丽. 汽车智能座舱交互设计研究综述[J]. 包装工程, 2023, 44(6):430-440.
CAI M Y, WANG W L.Review of Interaction Design in Automotive Intelligent Cockpit[J]. Packaging Engineering, 2023, 44(6):430-440.
[9] 耿秀丽, 潘亚虹. 考虑用户体验的产品服务系统模块重要度判定方法[J]. 计算机集成制造系统, 2020, 26(5):1295-1303.
GENG X L, PAN Y H.Importance Degree Determination Approach for Product Service System Modules Based on User Experience[J]. Computer Integrated Manufacturing Systems, 2020, 26(5):1295-1303.
[10] YAZDI M, MOHAMMADPOUR J, LI H, et al.Fault Tree Analysis Improvements:A Bibliometric Analysis and Literature Review[J]. Quality and Reliability Engineering International, 2023, 39(5):1639-1659.
[11] SOKUKCU M, SAKAR C.Risk Analysis of Collision Accidents during Underway STS Berthing Maneuver through Integrating Fault Tree Analysis (FTA) into Bayesian Network (BN)[J]. Applied Ocean Research, 2022, 126:103290.
[12] XIAO Q, LI Y P, LUO F, et al.Analysis and Assessment of Risks to Public Safety from Unmanned Aerial Vehicles Using Fault Tree Analysis and Bayesian Network[J]. Technology in Society, 2023, 73:102229.
[13] CHEN M Q, QU R, FANG W G.Case-Based Reasoning System for Fault Diagnosis of Aero-Engines[J]. Expert Systems with Applications, 2022, 202:117350.
[14] 赵传洋, 刘红兵, 蒋哲,等. 基于故障树的浮式风机系泊系统失效风险分析[J]. 哈尔滨工程大学学报, 2023, 44(08):1263-1269.
ZHAO C Y, LIU H B, JIANG Z, et al.Failure Risk Analysis of Floating Offshore Wind Turbine Mooring Systems Based on the Fault Tree Method[J]. Journal of Harbin Engineering University, 2023, 44(8):1263-1269.
[15] BORYCZKO K, SZPAK D, ŻYWIEC J, et al.The Use of a Fault Tree Analysis (FTA) in the Operator Reliability Assessment of the Critical Infrastructure on the Example of Water Supply System[J]. Energies, 2022, 15(12):4416.
[16] 陈嘉慧, 米洁, 张胜伦, 等. 基于故障树和贝叶斯网络的履带车辆行星变速机构可靠性分析[J]. 机械强度, 2024, 46(1):129-135.
CHEN J H, MI J, ZHANG S L, et al.Reliability Analysis of Tracked Vehicle Planetary Transmission Mechanism Based on Fault Tree and Bayesian Network[J]. Journal of Mechanical Strength, 2024, 46(1):129-135.
[17] LIU J F, LI Y L, MA X M, et al.FaultTreeAnalysisUsingBayesian Optimization:AReliableandEffective Fault Diagnosis Approaches[J]. Journal of Failure Analysis and Prevention, 2021, 21(2):619-630.
[18] YIN H L, LIU C H, WU W, et al.Safety Assessment of Natural Gas Storage Tank Using Similarity Aggregation Method Based Fuzzy Fault Tree Analysis (SAM-FFTA) Approach[J]. Journal of Loss Prevention in the Process Industries, 2020, 66:104159.
[19] 张力, 胡鸿. 大型复杂人-机-环境系统中的人因可靠性[M]. 北京:国防工业出版社, 2024.
ZHANG L, HU H.Human Reliability in Large Complex Man-Machine-Environment Systems[M]. Beijing:National Defense Industry Press, 2024.
[20] 施金斌, 刘晓佳, 马国旺, 等. 基于改进认知可靠性与失误分析方法的隧道驾驶人因可靠性分析模型[J]. 科学技术与工程, 2023, 23(25):10983-10989.
SHI J B, LIU X J, MA G W, et al.Human Reliability Analysis Model for Tunnel Driving Based on Improved CREAM Method[J]. Science Technology and Engineering, 2023, 23(25):10983-10989.
[21] UNG S T.Evaluation of Human Error Contribution to Oil Tanker Collision Using Fault Tree Analysis and Modified Fuzzy Bayesian Network Based CREAM[J]. Ocean Engineering, 2019, 179:159-172.
[22] PEI H N, GONG H, BAI Z H, et al.A Human Factor Reliability Analysis Method for Maritime Transport Based on an Improved CREAM Model and Group Decision-Making[J]. Ocean Engineering, 2024, 293:116664.
[23] SHI H, WANG J H, ZHANG L, et al.New Improved CREAM Model for Human Reliability Analysis Using a Linguistic D Number-Based Hybrid Decision Making Approach[J]. Engineering Applications of Artificial Intelligence, 2023, 120:105896.
[24] LIU X, LIU Z, LIN K C, et al.Evaluation of the Operation Process of Medical Equipment to Enhance Ergonomic Reliability Based on FRAM-Moran’s I and CREAM[J]. Applied Sciences, 2022, 12(1):200.
[25] UNG S T.Human Error Assessment of Oil Tanker Grounding[J]. Safety Science, 2018, 104:16-28.
[26] 许娜, 卢佳烨. 基于安全和情感化的汽车辅助驾驶系统多模态交互设计研究综述[J]. 汽车工程学报, 2024, 14(3):336-353.
XU N, LU J Y.A Review of Multi-Modal Interaction Design for Assisted Driving Systems Based on Safety and Affectivity[J]. Chinese Journal of Automotive Engineering, 2024, 14(3):336-353.
[27] AZOFEIFA J D, NOGUEZ J, RUIZ S, et al.Systematic Review of Multimodal Human-Computer Interaction[J]. Informatics, 2022, 9(1):13.
[28] 王伟伟, 顾炎辉, 余隋怀, 等. 智能座舱人机交互设计关键技术研究进展[J]. 机械设计, 2024, 41(8):30-36.
WANG W W, GU Y H, YU S H, et al.Research Progress on Key Technologies of Intelligent Cockpit Human-Computer Interaction Design[J]. Journal of Machine Design, 2024, 41(8):30-36.
[29] GHADHAB M, JUNGES S, KATOEN J P, et al.Safety Analysis for Vehicle Guidance Systems with Dynamic Fault Trees[J]. Reliability Engineering & System Safety, 2019, 186:37-50.
[30] 何欣宇, 代姗姗, 肖萌, 等. 煤矿预警信息发布影响因素重要度分析[J]. 安全与环境工程, 2024, 31(3):32-46.
HE X Y, DAI S S, XIAO M, et al.Importance Analysis of Factors of Coal Mine Early Warning Information Dissemination[J]. Safety and Environmental Engineering, 2024, 31(3):32-46.
[31] 董玉颜, 李永锋. 基于故障树分析与逼近理想解排序法的重卡车载信息系统界面设计[J]. 机械设计, 2023, 40(1):141-147.
DONG Y Y, LI Y F.Interface Design of Heavy Lorry Vehicle Information System Based on FTA and TOPSIS[J]. Journal of Machine Design, 2023, 40(1):141-147.
[32] 张爱琳, 刘晓佳. 改进CREAM模型的船舶引航员人因可靠性预测[J]. 中国航海, 2021, 44(1):32-37.
ZHANG A L, LIU X J.Improved CREAM Model for Prediction of Human Reliability Concerning Harbor Pilots[J]. Navigation of China, 2021, 44(1):32-37.
[33] 孙延浩, 张涛, 刘宁馨, 等. 基于证据推理的改进加权CREAM模型[J]. 安全与环境工程, 2022, 29(2):16-22.
SUN Y H, ZHANG T, LIU N X, et al.Improved Weighted CREAM Model Based on Evidential Reasoning[J]. Safety and Environmental Engineering, 2022, 29(2):16-22.
[34] KURT Y B, AKYUZ E, ARSLAN O.A Quantitative HAZOP Risk Analysis under Extended CREAM Approach for Maritime Autonomous Surface Ship (MASS) Operation[J]. Marine Technology Society Journal, 2022, 56(4):59-73.
[35] TAHERI M R, MORTAZAVI S B, ASILIAN H, et al.Investigating Human Error in Iran's Copper Mines Using the CREAM Based on Human Cognitive Reliability Analysis[J]. International Journal of Occupational Safety and Ergonomics, 2023, 29(4):1423-1428.
[36] WANG D D, YANG G L, HAN J X, et al.Quantitative Assessment of Human Error of Emergency Behavior for Hazardous Chemical Spills in Chemical Parks[J]. Process Safety and Environmental Protection, 2024, 189:930-949.
[37] HOLLNAGEL E.Cognitive Reliability and Error Analysis Method[M]. Oxford:Elsevier, 1998.
[38] HE X H, WANG Y, SHEN Z P, et al.A Simplified CREAM Prospective Quantification Process and Its Application[J]. Reliability Engineering & System Safety, 2008, 93(2):298-306.
[39] 于泳, 张帅, 吴通. 基于认知补偿的ROV操控系统人因失误预测方法[J]. 机械设计与研究, 2020, 36(6):154-158.
YU Y, ZHANG S, WU T.Human Error Prediction Method of ROV Control System Based on Cognitive Compensation[J]. Machine Design & Research, 2020, 36(6):154-158.
[40] 华明, 刘晓青. 基于FTA的苏州河河口水闸风险源识别[J]. 水利水电技术(中英文), 2022, 53(S1):366-371.
HUA M, LIU X Q.Risk Source Identification of Suzhou River Estuary Sluice Based on FTA[J]. Water Resources and Hydropower Engineering, 2022, 53(S1):366-371.
[41] ELIDOLU G, AKYUZ E, ARSLAN O, et al.Quantitative Failure Analysis for Static Electricity-Related Explosion and Fire Accidents on Tanker Vessels under Fuzzy Bow-Tie CREAM Approach[J]. Engineering Failure Analysis, 2022, 131:105917.
[42] MASALEGOOYAN Z, PIADEH F, BEHZADIAN K.A Comprehensive Framework for Risk Probability Assessment of Landfill Fire Incidents Using Fuzzy Fault Tree Analysis[J]. Process Safety and Environmental Protection, 2022, 163:679-693.
[43] SCHIAVO G, MICH O, FERRON M, et al.Trade-Offs in the Design of Multimodal Interaction for Older Adults[J]. Behaviour & Information Technology, 2022, 41(5):1035-1051.
[44] PAUZIEÉ A. A Method to Assess the Driver Mental Workload:The Driving Activity Load Index (DALI)[J]. IET Intelligent Transport Systems, 2008, 2(4):315-322.
[45] 詹蓓灵, 陆菁. 用户体验情感测量与情感识别研究综述[J]. 包装工程, 2024, 45(12):12-21.
ZHAN B L, LU J.Review of User Experience Emotion Measurement and Emotion Recognition[J]. Packaging Engineering, 2024, 45(12):12-21.
[46] KOSCH T, KAROLUS J, ZAGERMANN J, et al.A Survey on Measuring Cognitive Workload in Human-Computer Interaction[J]. ACM Computing Surveys, 2023, 55(13s):1-39.