Effect of System Transparency and Situational Certainty on Pilots' Approach Decision-making in Human-Machine Collaboration

PENG Xing; DONG Hao; LI Jiahao; ZHU Qi; JIANG Hao; YANG Jiazhong

doi:10.19554/j.cnki.1001-3563.2026.14.007

PDF(1043 KB)

Packaging Engineering ›› 2026, Vol. 47 ›› Issue (14) : 73-82. DOI: 10.19554/j.cnki.1001-3563.2026.14.007

Special subject: Innovation in Human Factors Engineering andIndustrial Design for Aviation Equipment

Effect of System Transparency and Situational Certainty on Pilots' Approach Decision-making in Human-Machine Collaboration

PENG Xing, DONG Hao, LI Jiahao, ZHU Qi, JIANG Hao, YANG Jiazhong^*

Author information +

History +

Abstract

The work aims to investigate the effects of system transparency on pilots' human-machine collaborative approach decision-making under different levels of situational certainty. Twenty-eight male pilots holding commercial pilot licenses were recruited. An aviation approach decision-making task was combined with the Wizard of Oz paradigm, and a 2 (System Transparency: Low, High) × 2 (Situational Certainty: Certain, Ambiguous) within-subjects design was adopted. Decision accuracy, accuracy gain, response time, compliance rate, and subjective trust were measured. The results showed significant interactions between system transparency and situational certainty in final decision accuracy, accuracy gain, compliance rate, and subjective trust. In certain situations, the high-transparency system significantly improved final decision accuracy, accuracy gain, compliance rate, and subjective trust. However, in ambiguous situations, no significant differences were found between the high- and low-transparency systems. In addition, the high-transparency system significantly prolonged decision response times in both situations. These findings indicate that the effects of system transparency on pilots' human-machine collaborative approach decision-making have situational boundaries. The advantages of high transparency are mainly reflected in certain situations with clear decision cues and are accompanied by a certain time cost. Therefore, the design of aviation intelligent assistance systems for the approach phase should consider task situational certainty and optimize the presentation of transparency information to improve effective collaborative decision-making between pilots and intelligent agents.

Key words

pilots / human-machine collaboration / aviation decision-making / system transparency / situational certainty / human-machine trust

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

PENG Xing, DONG Hao, LI Jiahao, ZHU Qi, JIANG Hao, YANG Jiazhong. Effect of System Transparency and Situational Certainty on Pilots' Approach Decision-making in Human-Machine Collaboration[J]. Packaging Engineering. 2026, 47(14): 73-82 https://doi.org/10.19554/j.cnki.1001-3563.2026.14.007

References

[1] 王梓宇, 游旭群. 航空决策模型及其影响因素分析[J]. 心理科学进展, 2017, 25(11): 2002-2010.
WANG Z Y, YOU X Q.Models and influential factors of aeronautical decision making[J]. Advances in Psychological Science, 2017, 25(11): 2002-2010.
[2] KELLY D, EFTHYMIOU M.An analysis of human factors in fifty controlled flight into terrain aviation accidents from 2007 to 2017[J]. Journal of Safety Research, 2019, 69: 155-165.
[3] YıLMAZ A A. Critical Connections: Enhancing Aviation Safety Through Network Analysis[J]. Akıllı Ulaşım Sistemleri ve Uygulamaları Dergisi, 2024, 7(2): 263-285.
[4] VEMPATI L, WOODS S, SOLANO R C.Qualitative analysis of general aviation pilots' aviation safety reporting system incident narratives using the human factors analysis and classification system[J]. The International Journal of Aerospace Psychology, 2023, 33(3): 182-196.
[5] 彭姓, 王翔, 仲家瑞, 等. 奖惩机制在不同风险水平下对航空决策的影响[J]. 安全与环境学报, 2026, 26(01): 21-30.
PENG X, WANG X, ZHONG J R, et al.Influence of reward and punishment mechanisms on aviation decision-making among pilots under risk[J]. Journal of Safety and Environment, 2026, 26(1): 21-30.
[6] ERJAVAC A J, IAMMARTINO R, FOSSACECA J M.Evaluation of preconditions affecting symptomatic human error in general aviation and air carrier aviation accidents[J]. Reliability Engineering & System Safety, 2018, 178: 156-163.
[7] ZHANG C, LIU C, LIU H, et al.Incorporation of pilot factors into risk analysis of civil aviation accidents from 2008 to 2020: A data-driven Bayesian network approach[J]. Aerospace, 2022, 10(1): 9.
[8] PUTRA I, TJ H W, WAHYOEDI S.Pilot decisions are the result of management determining flight safety[J]. International Journal of Educational Review, Law and Social Sciences, 2023, 3(4): 1394-1401.
[9] COMERFORD D, BRANDT S L, LACHTER J B, et al. NASA's single-pilot operations technical interchange meeting: proceedings and findings[C]// Technical Interchange Meeting.2013 (ARC-E-DAA-TN8313).
[10] SUTTHITHATIP S, PERINPANAYAGAM S, ASLAM S, et al.Explainable AI in aerospace for enhanced system performance[C]//2021 IEEE/AIAA 40th Digital Avionics Systems Conference (DASC). IEEE, 2021: 1-7.
[11] CHEN J Y, BARNES M J.Human-agent teaming for multirobot control: A review of human factors issues[J]. IEEE Transactions on Human-Machine Systems, 2014, 44(1): 13-29.
[12] BHASKARA A, SKINNER M, LOFT S.Agent transparency: A review of current theory and evidence[J]. IEEE Transactions on Human-Machine Systems, 2020, 50(3): 215-224.
[13] LIU B.In AI we trust? Effects of agency locus and transparency on uncertainty reduction in human-AI interaction[J]. Journal of Computer-Mediated Communication, 2021, 26(6): 384-402.
[14] TATASCIORE M, LOFT S.Can increased automation transparency mitigate the effects of time pressure on automation use?[J]. Applied Ergonomics, 2024, 114: 104142.
[15] LYONS J B, SADLER G G, KOLTAI K, et al.Shaping trust through transparent design: theoretical and experimental guidelines[C]//Advances in Human Factors in Robots and Unmanned Systems: Proceedings of the AHFE 2016 International Conference on Human Factors in Robots and Unmanned Systems, July 27-31, 2016, Walt Disney World®, Florida, USA. Cham: Springer International Publishing, 2016: 127-136.
[16] LEICHTMANN B, HUMER C, HINTERREITER A, et al.Effects of explainable artificial intelligence on trust and human behavior in a high-risk decision task[J]. Computers in Human Behavior, 2023, 139: 107539.
[17] SADLER G, BATTISTE H, HO N, et al.Effects of transparency on pilot trust and agreement in the autonomous constrained flight planner[C]//2016 IEEE/ AIAA 35th Digital Avionics Systems Conference (DASC). IEEE, 2016: 1-9.
[18] 李奕洁, 张玲, 黄琪璋, 等. 系统透明度信息对人机信任和协同决策的影响[J]. 包装工程, 2023, 44(20): 25-33.
LI Y J, ZHANG L, HUANG Q Z, et al.Impact of System Transparency Information on Human-Machine Trust and Collaborative Decision Making[J]. Packaging Engineering, 2023, 44(20): 25-33.
[19] 王栩金凤, 杨奕诚, 李伟涛, 等. AI透明度和可靠性对人-AI协同决策绩效与信任的影响[J]. 人类工效学, 2024, 30(2): 53-60.
WANG X J F, YANG Y C, LI W T, et al. The Impact of AI Transparency and Reliability on Human-AI Collaborative Decision-Making Performance and Trust[J]. Chinese Journal of Ergonomics, 2024, 30(2): 53-60.
[20] SIMON L, RAUFFET P, GUERIN C.Effects of agent transparency and situation criticality upon human- autonomy trust and risk perception in decision-making[J]. Cognition, Technology & Work, 2025, 27(1): 21-38.
[21] CAUSSE M, PéRAN P, DEHAIS F, et al. Affective decision making under uncertainty during a plausible aviation task: An fMRI study[J]. NeuroImage, 2013, 71: 19-29.
[22] ERLEI A, DAS R, MEUB L, et al.For what it’s worth: Humans overwrite their economic self-interest to avoid bargaining with AI systems[C]//Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 2022: 1-18.
[23] KANG H.Sample size determination and power analysis using the G* Power software[J]. Journal of educational evaluation for health professions, 2021, 18: 17.
[24] FABRE E F, MATTON N, BELTRAN F, et al.Hierarchy in the cockpit: How captains influence the decision- making of young and inexperienced first officers[J]. Safety Science, 2022, 146: 105536.
[25] CHONG L, ZHANG G, GOUCHER-LAMBERT K, et al.Human confidence in artificial intelligence and in themselves: The evolution and impact of confidence on adoption of AI advice[J]. Computers in Human Behavior, 2022, 127: 107018.
[26] KELLEY J F.An empirical methodology for writing user-friendly natural language computer applications[C]//Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1983: 193-196.
[27] ZHANG Y, LIAO Q V, BELLAMY R K.Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making[C]//Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 2020: 295-305.
[28] BANSAL G, WU T, ZHOU J, et al.Does the whole exceed its parts? the effect of ai explanations on complementary team performance[C]//Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 2021: 1-16.
[29] NETA M, SCHLAGGAR B L, PETERSEN S E.Separable responses to error, ambiguity, and reaction time in cingulo-opercular task control regions[J]. Neuroimage, 2014, 99: 59-68.
[30] STOWERS K, KASDAGLIS N, NEWTON O, et al.Intelligent agent transparency: The design and evaluation of an interface to facilitate human and intelligent agent collaboration[C]//Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Sage CA: Los Angeles, CA: SAGE Publications, 2016, 60(1): 1706-1710.
[31] MERCADO J E, RUPP M A, CHEN J Y, et al.Intelligent agent transparency in human-agent teaming for Multi-UxV management[J]. Human Factors, 2016, 58(3): 401-415.
[32] BHASKARA A, DUONG L, BROOKS J, et al.Effect of automation transparency in the management of multiple unmanned vehicles[J]. Applied Ergonomics, 2021, 90: 103243.
[33] SALIMZADEH S, HE G, GADIRAJU U.Dealing with uncertainty: Understanding the impact of prognostic versus diagnostic tasks on trust and reliance in human-AI decision making[C]//Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 2024: 1-17.
[34] TATASCIORE M, LOFT S.Calibrating reliance on automated advice: transparency and trust calibration feedback[J]. International Journal of Human-Computer Interaction, 2025, 41(23): 14723-14733.
[35] CHEN J Y, PROCCI K, BOYCE M, et al.Situation Awareness-Based Agent Transparency[R]. 2014.
[36] Ngo V M.Balancing AI transparency: trust, certainty, and adoption[J]. Information Development, 2025: 02666669251346124.
[37] PAPENMEIER A, KERN D, ENGLEBIENNE G, et al.It's complicated: The relationship between user trust, model accuracy and explanations in AI[J]. ACM Transactions on Computer-Human Interaction (TOCHI), 2022, 29(4): 1-33.
[38] LAMMERT O, RICHTER B, SCHüTZE C, et al. Humans in XAI: increased reliance in decision-making under uncertainty by using explanation strategies[J]. Frontiers in Behavioral Economics, 2024, 3: 1377075.
[39] KIRLIK A.Modeling strategic behavior in human- automation interaction: Why an" aid" can (and should) go unused[J]. Human Factors, 1993, 35(2): 221-242.
[40] KöRBER M. Theoretical considerations and development of a questionnaire to measure trust in automation [C]//Congress of the International Ergonomics Association. Cham: Springer International Publishing, 2018: 13-30.
[41] MILLER D, JOHNS M, MOK B, et al.Behavioral measurement of trust in automation: the trust fall[C]//Proceedings of the human factors and ergonomics society annual meeting. Sage CA: Los Angeles, CA: SAGE Publications, 2016, 60(1): 1849-1853.
[42] CHEN N, ZHANG X.When misunderstanding meets artificial intelligence: the critical role of trust in human- AI and human-human team communication and performance[J]. Frontiers in Psychology, 2025, 16: 1637339.