Information Interface Design of Robot Swarm from the Perspective of Visual Affordance

LIU Yushun; GONG Xiaodong; WANG Yun; LI Hong; GONG Qian; LI Xinyan

doi:10.19554/j.cnki.1001-3563.2025.12.005

PDF(4002 KB)

Packaging Engineering ›› 2025, Vol. 46 ›› Issue (12) : 60-72. DOI: 10.19554/j.cnki.1001-3563.2025.12.005

Special Subject: Intelligence-Infused Design, Synergistic Empo-werment

Information Interface Design of Robot Swarm from the Perspective of Visual Affordance

LIU Yushun¹, GONG Xiaodong¹, WANG Yun¹, LI Hong¹, GONG Qian¹, LI Xinyan^2*

Author information +

History +

Abstract

The work aims to propose information design strategies for typical functions in robot swarm information interfaces, aiming to establish a match between "bottom-up" interface information and "top-down" user needs, thereby enhancing interface visual affordance and improving user decision-making performance in swarm control scenarios. Based on the user decision-making path in the target recognition module of the interface, "top-down" information needs were extracted. The affordance theory and the five elements of interaction design were combined to construct "bottom-up" interface information design strategies. In the design practice phase, an interactive prototype for target recognition tasks was developed based on these strategies, and a comparative experiment was conducted to validate their effectiveness. The optimized solution incorporating affordance design strategies demonstrated superior performance in decision time, cognitive load levels, pupil diameter, and gaze revisitation frequency compared with the solution without these strategies. The design strategies proposed based on the affordance theory are conducive to improving the decision-making performance of users and providing design methods and ideas for the design of complex information interfaces.

Key words

information interface / visual affordance / information design / robot swarm

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

LIU Yushun, GONG Xiaodong, WANG Yun, LI Hong, GONG Qian, LI Xinyan. Information Interface Design of Robot Swarm from the Perspective of Visual Affordance[J]. Packaging Engineering. 2025, 46(12): 60-72 https://doi.org/10.19554/j.cnki.1001-3563.2025.12.005

References

[1] 王伟嘉, 郑雅婷, 林国政, 等. 集群机器人研究综述[J]. 机器人, 2020, 42(2): 232-256.
WANG J W, ZHENG Y T, LIN G Z, et al.Swarm Robotics: A Review[J]. Robot, 2020, 42(2): 232-256.
[2] 武文亮, 周兴社, 沈博, 等. 集群机器人系统特性评价研究综述[J]. 自动化学报, 2022, 48(5): 1153-1172.
WU W L, ZHOU X S, SHEN B, et al.A Review of Swarm Robotic Systems Property Evaluation Research[J]. Acta Automatica Sinica, 2022, 48(5): 1153-1172.
[3] 史殿习, 洪臣, 康颖, 等. 面向多无人机协同飞行控制的云系统架构[J]. 计算机学报, 2020, 43(12): 2352-2371.
SHI D X, HONG C, KANG Y, et al.Cloud-based Control System Architecture for Multi-UAVs Cooperative Flight[J]. Chinese journal of computers, 2020, 43(12): 2352-2371.
[4] 陈甜甜, 赵洋. 国外领先安防巡逻机器人发展现状概述[J]. 中国安防, 2021(11): 109-116.
CHEN T T, ZHAO Y.Overview of the Development Status of Leading Foreign Security Patrol Robots[J]. China Security & Protection, 2021(11): 109-116.
[5] 于洋, 孙思卿, 张立川, 等. 自主水下航行器集群组网技术发展与展望[J]. 水下无人系统学报, 2024, 32(2): 194-207.
YU Y, SUN S Q, ZHANG L C, et al.Development and Prospects of Networking Technologies for Autonomous Undersea Vehicles[J]. Journal of Unmanned Undersea System, 2024, 32(2): 194-207.
[6] NIROUI F, LIU Y, BICHAY R, et al.A Graphical User Interface for Multi-Robot Control in Urban Search and Rescue Applications[C]//Proceedings of the 2016 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS), Tokyo: IEEE, 2016: 217-222.
[7] ANDOLINA S, FORLIZZI J.The Design of Interfaces for Multi-Robot Path Planning and Control[C]//Proceedings of the 2014 IEEE International Workshop on Advanced Robotics and Its Social Impacts. Evanston: IEEE, 2014: 7-13.
[8] CHIEN T L, ZOU J H, GUO H, et al.Multiple Robot Based Intelligent Security System[C]//Proceedings of the 27th Chinese Control Conference (CHICC). Kunming: CHICC, 2008: 647-651.
[9] DIVBAND SOORATI M, CLARK J, GHOFRANI J, et al.Designing a User-Centered Interaction Interface for Human-Swarm Teaming[J]. Drones, 2021, 5(4): 131.
[10] FURUKAWA H.Usage of Different Levels of Functional Information in Multiple Robot Operation[C]//Proceedings of the 4th International Conference on Autonomous Robots and Agents (ICARA), Wellington: ICARA, 2009: 74-78.
[11] WAGNER A R, ENDO Y, ULAM P, et al.Multi-robot User Interface Modeling[C]//Proceedings of the 8th International Symposium on Distributed Autonomous Robotic Systems (DARS), Tokyo: DARS, 2006: 237-248.
[12] WANG A.Intelligent Warehouse Multi-Robot Scheduling System Based on Improved A* Algorithm[C]//Proceedings of the 3rd Conference on Fully Actuated System Theory and Applications (FASTA). Shenzhen: FASTA, 2024: 1305-1310.
[13] LONČAR I, SANTOS P, OMERDIC E. Advancing Offshore Underwater Applications Using a Robotic System with Robust Navigation and Virtual Supervision Capabilities[C]//Proceedings of the 2023 IEEE 9th World Forum on Internet of Things (WF-IoT). Aveiro: IEEE, 2023: 1-7.
[14] QIN W, LIU S, YANG Y, DUAN J.Design of Multi-Robot Communication and Human-Computer Interaction Interface[C]//Proceedings of the 36th Chinese Control and Decision Conference (CCDC). Xi'an: IEEE, 2024: 5524-5529.
[15] SEO S H, YOUNG J E, IRANI P.How Are Your Robot Friends Doing? A Design Exploration of Graphical Techniques Supporting Awareness of Robot Team Members in Teleoperation[J]. International Journal of Social Robotics, 2021(13): 725-749.
[16] MUKHOPADHYAY S C, GUPTA G S.Autonomous Robots and Agents[M]. Berlin: Springer, 2007.
[17] BAY J S, BORRELLI L E, CHAPMAN K L, et al.User Interface and Display Management Design for Multiple-Robot Command and Control[C]//Proceedings of SPIE 4195, Mobile Robots XV and Telemanipulator and Telepresence Technologies VII. Boston: SPIE, 2001: 1-12.
[18] GREGORY J, FINK J, STUMP E, et al.Application of Multi-Robot Systems to Disaster-Relief Scenarios with Limited Communication[C]//Proceedings of the 10th Conference on Field and Service Robotics (FSR). Toronto: FSR, 2015: 42-56.
[19] SEO S H, YOUNG J E.What Happened While I Was Away? Leveraging Visual Transition Techniques to Convey Robot States in Multi-robot Teleoperation[C]//Proceedings of the International Conference on Social Robotics (ICSR 2021). Singapore: ICSR, 2021: 749-756.
[20] TROUVAIN B, SCHLICK C, MEVERT M.Comparison of a Map- vs. Camera-Based User Interface in a Multi- Robot Navigation Task[C]//Proceedings of the 2003 IEEE International Conference on Systems, Man and Cybernetics (SMC). Washington: SMC, 2003: 3224-3231.
[21] RULE A, FORLIZZI J.Designing Interfaces for Multi-User, Multi-Robot Systems[C]//Proceedings of the ACM/IEEE International Conference on Human- Robot Interaction (HRI). New York: HRI, 2012: 97-104.
[22] CHEN S.Multi-modal User Interface for Multi-robot Control in Underground Environments[C]//Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Kyoto: IROS, 2022: 9995-10002.
[23] KÖTTER D, WIEDON G, MEIERKORD D, TRINH M, PETROVIC O, BRECHER C. Development of an Augmented Reality User Interface for Collaborative Robotics in Quality Inspection for Manufacturing[C]//Proceedings of the 5th International Conference on Control and Robotics (ICCR). Tokyo: ICCR, 2023: 107-112.
[24] HATANAKA T, MOCHIZUKI T, MAESTRE TORREBLANCA J M, et al. Impact of VR Technology on a Human in Semi-autonomous Multi-robot Navigation: Control-Theoretic Perspective[J]. IFAC-PapersOnLine, 2022, 55(41): 38-43.
[25] GIBSON J J.The Theory of Affordances[M] Hillsdale: Erlbaum Associates, 1977.
[26] 薛绍华. 知觉即行动: 从哲学概念到机器实现[M]. 北京: 中国科学技术出版社, 2020.
XUE S H.Perception is Action: From Philosophical Concept to Machine Simulation[M]. Beiing: China Science and Technology Press, 2020.
[27] NORMAN D A.The Design of Everyday Things, Currency[J]. Design of Everyday Things,1998,7(4), 245-246.
[28] 陈辉, 胡伟峰. 基于可供性的智能厨电交互界面设计研究[J]. 包装工程, 2024, 45(12): 425-434.
CHEN H, HU W F.Design of Intelligent Kitchen Appliance Interaction Interface Based on Affordance[J]. Packagaing Engineering, 2024, 45(12): 425-434.
[29] 都蒙蒙. 奖赏反馈对刺激驱动注意的调节作用[D]. 杭州: 浙江大学, 2019.
DU M M.Reward-Modulating Effect on Stimulus- Driven Attention[D]. Hangzhou: Zhejiang University, 2019.
[30] 张德香. 注意引导对视觉工作记忆表征的影响[D]. 济南: 山东师范大学, 2019.
ZHANG D X.The Effect of Attentional Guidance on Visual Working Memory Representations[D]. Jinan: Shandong Normal University, 2019.
[31] ZHENG W, SUN Y, WU H, et al.The Interaction of Top-down and Bottom-up Attention in Visual Working Memory[J]. Scientific Reports, 2024(14): 17397.
[32] 叶东辉. 指挥与控制对机器人集群作战至关重要[J]. 防务视点, 2015(1): 10-12.
YE D H.Command and Control are Crucial for Robotic Swarm Operations[J]. Defense Point, 2015(1): 10-12.
[33] GABER W W. (1991). Technology Affordances Conference on Human Factors in Computing Systems[C]//Proceedings of the CHI 1991. New Orleans: CHI, 1991.
[34] Falahatpisheh Z, Khajeheian D.Affordances and IT Design: A Typology for Social Media and Platform Affordances[C]//Proceedings of the 13th CMI Conference on Cybersecurity and Privacy (CMI)-Digital Transformation-Potentials and Challenges. Copenhagen: IEEE, 2020: 97-103
[35] 杨璇. 基于分层思维的信息视觉化动态图形设计[J]. 创意与设计, 2022(2): 29-35.
YANG X.Information Visualization Dynamic Graphic Design Based on Hierarchical Thinking[J]. Creation and Design, 2022(2): 29-35.
[36] 徐芳棋, 李昕. 交互设计中的用户体验五要素综述[J]. 流行色, 2020(2): 132-133.
XU F Q, LI X.An Overview of the Five Elements of User Experience in Interaction Design[J]. Fashion Color, 2020(2): 132-133.
[37] 袁梅, 白刚, 陈炅. 虚拟多功能显示系统设计[J]. 系统仿真学报, 2006(6): 1578-1581.
YUAN M, BAI G, CHEN J.Design of Virtual Multi-Function Display System[J]. Journal of System Simulation, 2006(6): 1578-1581.
[38] 魏子晗, 李兴珊. 决策过程的追踪:基于眼动的证据[J]. 心理科学进展, 2015, 23(12):13.
WEI Z H, LI X S.Decision Process Tracing: Evidence from Eye-movement Data[J]. Advances in Psychological Science, 2015, 23(12): 13.
[39] 杨玚, 邓赐平. NASA-TLX 量表作为电脑作业主观疲劳感评估工具的信度、效度研究[J]. 心理研究, 2010, 3(3): 36-41.
YANG Y, DENG C P.A study on the Reliability and Validity of NASA-TLX as A Measurement of Subjective Fatigue after Computer Operation[J]. Psychological Research, 2010, 3(3): 36-41.
[40] 余雯, 闫巩固, 黄志华. 决策中的过程追踪技术:介绍与展望[J]. 心理科学进展, 2013, 21(4): 606-614.
YU W, YAN G G, HUANG Z H.Process-tracing Techniques in Decision-making: Introduction and Prospects[J]. Advances in Psychological Science, 2013, 21(4): 606-614.
[41] KAHNEMAN D, BEATTY J.Pupil Diameter and Load on Memory[J]. Science, 1966, 154(3756): 1583-1595.
[42] 汪海波, 薛澄岐, 黄剑伟, 等. 基于认知负荷的人机交互数字界面设计和评价[J]. 电子机械工程, 2013(5): 4.
WANG H B, XUE C Q, HUANG J W, et al.Design and Evaluation of Human-computer Digital Interface Based on Cognitive Load[J]. Electro-Mechanical Engineering, 2013(5):4.
[43] 陈晓皎, 薛澄岐, 陈默, 等. 基于眼动追踪实验的数字界面质量评估模型[J]. 东南大学学报(自然科学版), 2017, 47(1): 38-42.
CHEN X J, XUE C Q, CHEN M, et al.Quality Assessment Model of Digital Interface Based on Eye-tracking Experiments[J]. Journal of Southeast University (Natural Science Edition), 2017, 47(1): 38-42.
[44] HART S G, STAVELAND L E.Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research[J]. Advances in Psychology, 2017, 52(6): 139-183.
[45] 宫晓东, 龚迁, 刘毓舜, 等. 基于认知负荷理论的雷达信息视图交互设计策略研究[J]. 包装工程, 2021, 42(20): 66-76.
GONG X D, GONG Q, LIU Y S, et al.Interactive Design Strategy Research on Radar Information Views Based on Cognitive Load Theory[J]. Packagaing Engineering, 2021, 42(20): 66-76.
[46] HEALEY C, ENNS J.Attention and Visual Memory in Visualization and Computer Graphics[J]. IEEE Transactions on Visualization and Computer Graphics, 2012, 18(7): 1170-1188.
[47] SWELLER J.Cognitive Load During Problem Solving: Effects on Learning[J]. Cognitive Science, 2010, 12(2): 257-285.
[48] JUST M A, CARPENTER P A.Eye Fixations and Cognitive Processes[J]. Cognitive Psychology, 1976, 8(4): 441-480.