To address the prevalent problems of missing, inaccurate, and inconsistent metadata in cultural relic digital resources under the background of museum digital resource management, the work aims to establish an operable metadata quality assessment system as well as an automated quality assessment tool for cultural relic digital resources, and to verify their practicability and usability. Firstly, relevant researches were systematically reviewed through the National Public Service Platform for Standards Information, CNKI, and Google Scholar, from which preliminary quality dimensions and indicators were distilled. Subsequently, two rounds of Delphi consultation were conducted to screen and refine the indicator system, and the Analytic Hierarchy Process (AHP) was employed to determine the weights of each indicator, resulting in a metadata quality assessment framework for cultural relic digital resources. On this basis, an automated quality assessment tool driven by large language models (LLMs) was designed and developed. A comparative experiment on museum datasets was then carried out to examine the differences between purely manual assessment and LLM-assisted assessment, and a five-point Likert-scale questionnaire was used to collect experts' subjective assessment of the practical value of the framework and the effectiveness of the tool. A metadata quality assessment framework for cultural relic digital resources was constructed, comprising seven primary and eleven secondary indicators. Experimental results demonstrated that the constructed assessment framework achieved high scores in dimensions including "completeness, clarity and operability". Comparative experiments revealed that, in contrast to fully manual assessment, experts generally recognized that the automated metadata quality assessment tool presented significant advantages in "effectiveness, efficiency and satisfaction". The constructed quality assessment framework and automated quality assessment tool can provide theoretical and methodological references for metadata quality assessment of cultural relic digital resources, and possess practical value for advancing the high-quality development of cultural relic digitization.
Key words
cultural relic digital resources /
metadata quality /
assessment framework /
automation tools /
large language model
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
References
[1] 翟进, 王瑞津. 数据时代的文化江河——北京易华录在文物数据要素体系化建设中的实践与探索[J]. 文化数字化, 2024, (2): 62-71.
ZHAI J, WANG R J.Cultural Rivers in the Data Era: Practice and Exploration of Beijing E-Hualu in Systematic Construction of Cultural Relic Data Elements[J]. Culture Digitalization, 2024, (2): 62-71.
[2] 杜怿平. 人工智能技术在我国智慧博物馆的应用及深化路径[J]. 科技管理研究, 2024, 44(7): 162-169.
DU Y P.Application and Deepening Path of Artificial Intelligence Technology in Smart Museums in China[J]. Science and Technology Management Research, 2024, 44(7): 162-169.
[3] 刘祺, 黄君义, 冯耕中, 等. 数字经济发展根基: 数据质量研究述评与未来展望[J]. 系统工程理论与实践, 2025, 45(7): 2101-2123.
LIU Q, HUANG J Y, FENG G Z, et al.Data Quality as the Foundation of Digital Economy Development: A Review and Future Prospects[J]. Systems Engineering Theory and Practice, 2025, 45(7): 2101-2123.
[4] WANG R Y, STRONG D M.Beyond Accuracy: What Data Quality Means to Data Consumers[J]. Journal of Management Information Systems, 1996, 12(4): 5-33.
[5] ISO/IEC. Software Engineering—Software Product Quality Requirements and Evaluation (SQuaRE)—Data Quality Model: ISO/IEC 25012:2008[S]. Geneva: International Organization for Standardization, 2008.
[6] BRUCE T R, HILLMANN D I.The Continuum of Metadata Quality: Defining, Expressing, Exploiting[C]// HILLMANN D I, WESTBROOKS E L (eds.). Metadata in Practice. Chicago, IL: American Library Association, 2004: 238-256.
[7] 石油信息与计算机应用专业标准化委员会. 数据质量控制与评估原则: SY/T 7005-2014[S]. 北京: 石油工业出版社, 2014.
Petroleum Information and Computer Application Professional Standardization Committee. Principles for Data Quality Control and Evaluation: SY/T 7005- 2014[S]. Beijing: Petroleum Industry Press, 2014.
[8] 全国质量管理和质量保证标准化技术委员会. 质量管理体系要求: GB/T 19001-2016[S]. 北京: 中国标准出版社, 2016.
National Technical Committee on Quality Management and Quality Assurance Standardization. Quality Management Systems Requirements: GB/T 19001-2016[S]. Beijing: Standards Press of China, 2016.
[9] 全国信息技术标准化技术委员会. 信息技术数据质量评价指标: GB/T 36344-2018[S]. 北京: 中国标准出版社, 2018.
National Technical Committee for Information Technology Standardization. Information Technology Data Quality Evaluation Indicators: GB/T 36344-2018[S]. Beijing: Standards Press of China, 2018.
[10] 全国信息技术标准化技术委员会. 物联网数据质量评价方法: GB/T 44811-2024[S]. 北京: 中国标准出版社, 2024.
National Technical Committee for Information Technology Standardization of China. Internet of Things—Data Quality Evaluation Method: GB/T 44811-2024[S]. Beijing: Standards Press of China, 2024.
[11] LIM S, LIEW C L.Metadata quality and interoperability of GLAM digital images[J]. Aslib Proceedings, 2011, 63(5): 484-498.
[12] KIRÁLY P, BÜCHLER M. Measuring completeness as metadata quality metric in Europeana[C]// 2018 IEEE International Conference on Big Data (Big Data). Seattle, WA, USA: IEEE, 2018: 2711-2720.
[13] PHILLIPS M E, ZAVALINA O L, TARVER H.Using metadata record graphs to understand digital library metadata[C]// Proceedings of the International Conference on Dublin Core and Metadata Applications. Dublin Core Metadata Initiative, 2020.
[14] 方丽瑜. 博物馆在线藏品数据库的关键质量框架设计——以故宫博物院数字文物库为例[J]. 故宫博物院院刊, 2024, (5): 13-23, 139-140.
FANG L Y. Design of a Key Quality Framework for Online Museum Collection Databases: A Case Study of the Palace Museum Digital Relics Repository[J]. Palace Museum Journal, 2024, (5): 13-23, 139-140.
[15] 李华飙. 馆藏文物三维模型的数据质量评价方法研究[J]. 博物馆管理, 2020, (4): 5-10.
LI H B.Study on Data Quality Evaluation Methods for 3D Models of Museum Collections[J]. Museum Management, 2020, (4): 5-10.
[16] 蒋振, 蒋超, 铁铮, 等. 革命文物三维模型展示设计评价指标体系构建与应用研究[J]. 包装工程, 2024, 45(8): 121-130.
JIANG Z, JIANG C, TIE Z, et al.Construction and Application of an Evaluation Index System for the Display Design of 3D Models of Revolutionary Cultural Relics[J]. Packaging Engineering, 2024, 45(8): 121-130.
[17] CANDELA G.An Automatic Data Quality Approach to Assess Semantic Data from Cultural Heritage Institutions[J]. Journal of the Association for Information Science and Technology, 2023, 74(7): 866-878.
[18] LORENZINI M, ROSPOCHER M, TONELLI S.Automatically Evaluating the Quality of Textual Descriptions in Cultural Heritage Records[J]. International Journal on Digital Libraries, 2021, 22(2): 217-231.
[19] 郑子聪, 谢董颖, 黄阳梅, 等. 基于德尔菲法和层次分析法的机构健康传播工作评价指标体系构建[J]. 中国公共卫生, 2025, 41(11): 1379-1385.
ZHENG Z C, XIE D Y, HUANG Y M, et al.Construction of an Evaluation Index System for Institutional Health Communication Based on the Delphi Method and Analytic Hierarchy Process[J]. Chinese Journal of Public Health, 2025, 41(11): 1379-1385.
[20] 李冰艳. 健康科普类短视频质量评价指标体系的构建[D]. 合肥: 安徽医科大学, 2024.
LI B Y.Construction of a Quality Evaluation Index System for Health Science Popularization Short Videos[D]. Hefei: Anhui Medical University, 2024.
[21] BATINI C, SCANNAPIECO M.Data Quality: Concepts, Methodologies and Techniques[M]. Berlin Heidelberg: Springer, 2006.
[22] FARAJ G, MICSIK A.Representing and Validating Cultural Heritage Knowledge Graphs in CIDOC-CRM Ontology[J]. Future Internet, 2021, 13(11): 277.
[23] FLOTTORP S A, OXMAN A D, KRAUSE J, et al.A Checklist for Identifying Determinants of Practice: A Systematic Review and Synthesis of Frameworks and Taxonomies of Factors That Prevent or Enable Improvements in Healthcare Professional Practice[J]. Implementation Science, 2013, 8(1): 35.
[24] International Organization for Standardization. Ergonomics of Human-System Interaction-Part 11: Usability: Definitions and Concepts: ISO 9241-11:2018[S]. Geneva: International Organization for Standardization, 2018: 1-29.
[25] BROOKE J.SUS—A Quick and Dirty Usability Scale[J]. Usability Evaluation in Industry, 1996, 189(194): 4-7.
PDF(1786 KB)
渝公网安备 50010702501717号