计算机科学

计算机科学 ›› 2026, Vol. 53 ›› Issue (6A): 250600204-10.doi: 10.11896/jsjkx.250600204

• 大数据&数据科学 • 上一篇    下一篇

基于元数据的数据质量测量方法

陈连勇1, 宋金玉2, 李治侠1, 司长哲1, 杨文凯1, 王晶1   

  1. 1 西昌卫星发射中心 四川 西昌 615000
    2 陆军工程大学指挥控制工程学院 南京 210000
  • 出版日期:2026-06-16 发布日期:2026-06-12
  • 通讯作者: 宋金玉(sjyhello@163.com)
  • 作者简介:(nature68c@163.com)
  • 基金资助:
    国家自然科学基金(62207031)

Data Quality Measurement Method Based on Metadata

CHEN Lianyong1, SONG Jinyu2, LI Zhixia1, SI Changzhe1, YANG Wenkai1, WANG Jing1   

  1. 1 Xichang Satellite Launch Center,Xichang,Sichuan 615000,China
    2 College of Command and Control Engineering,Army Engineering University of PLA,Nanjing 210000,China
  • Published:2026-06-16 Online:2026-06-12
  • About author:CHEN Lianyong,born in 1999,master,assistant engineer.His main research interests include communication engineering and data engineering.
    SONG Jinyu,born in 1967,master,professor,is a member of CCF(No.99004M).Her main research interests include data engineering and data quality.
  • Supported by:
    National Natural Science Foundation of China(62207031).

PDF (PC)

44

摘要： 数据质量是激活数据要素潜能与保障数据价值实现的现实需要。为发现问题数据以提升数据质量,研究了基于元数据的数据质量测量方法,基于元数据构建了可实现数据一致性、完整性和有效性的数据质量测量指标集,制定了测量规则的生成要素和形式化构成形式,研制了相应的数据质量测量工具。结合某大学的教学评价数据集,检验了测量方法和测量工具的可行性和有效性。

关键词: 数据质量, 元数据, 数据质量测量

Abstract: Data quality is a realistic need to activate the potential of data elements and ensure the realization of data value.In order to find problematic data and improve data quality,this paper studies the data quality measurement methods based on metadata,constructs the data quality measurement index set based on metadata,and formulates the generation elements and formal composition forms of measurement rules,so as to achieve data quality measurement with consistency,integrity and validity.And the corre-sponding data quality measurement tool is developed.Combining with the teaching evaluation dataset of a university,the feasibility and effectiveness of the proposed measurement method and measurement tool are verified.

Key words: Data quality, Metadata, Data quality measurement

中图分类号: 

  • TP 274+.3
[1] DING X O,LI Y Z,WANG C,et al.Time series data quality Rules discovery with both row and column dependencies[J].Journal of Software,2023,34(3):1065-1086.
[2] LI J Z,WANG H Z,GAO H.State-of-the-Art of research on big data usability[J].Journal of Software,2016,27(7):1605-1625.
[3] SONG J Y,CHEN L Y,CHEN G.Data quality measurement framework research and field measurement framework construction[J].Computer Science,2024,51(4):19-27.
[4] KUANG J Q,ZHAO C,YANG L,et al.An Outlier CleaningAlgorithm Based on Deep Learning[J].Journal of Electronics & Information Technology,2022,44(2):507-513.
[5] SONG J Y,CHEN S,GUO D P,et al.Data Quality and Data Cleaning Methods[J].Command Information Sy stem and Technology,2013,4(5):63-70.
[6] LI H F,GUO R T,HU H Q,et al.An evaluation method of NOTAM data quality based on entropy weight method[J].Journal of Civil Aviation,2022(4):1-5.
[7] ZHANG W,WANG D,TAN X Y.Robust class-specific autoencoder for data cleaning and classification in the presence of label noise[J].Neural Process Lett,2019,50(2):1845-1860
[8] DAI C F,LI P,WANG W Q.Application of maximum depen-dency set in inconsistent data detection[J].Computer Engineering and Applications,2019,55(15):89-95.
[9] GE J,LIANG Y D,HUANG J P,et al.A large scale data qualityanomaly detection method based on data characteristics:CN113569006A[P].2021-10-29.
[10] LIU Z F,WANG J M,LI Q.A survey of research on metadata quality evaluation[J].Information studies:Theory & Application,2022,45(7):42-48.
[11] HUANG G,YUAN M,WU X Y,et al.Data quality assessment architecture research based on metadata-driven[J].Computer Engineering and Applications,2013,49(8):114-119.
[12] BATINI C,SCANNAPIECO M.Data and information qualitydimensions,principles and techniques[M].Beijing:National Defense Industry Press,2022:17-43.
[13] CHEN L Y,SONG J Y.Abnormal data detection method based on ant colony clustering[C]//Proceedings of the 6th International Conference on Electronic Information Technology and Computer Engineering.2022:66-72.
[14] LIU F,LI M,REN H M,et al.Data quality evaluation method based on rule base[J].Computer Systems & Appli-cations,2017,26(11):165-169.
[15] WANG W H.Research on key technologies of business rule-based expert system[D].Xi'an:Northwest University,2011.
[16] LIU B,GENG Y R.Mining method for data quality detectionrules[J].Pattern Recognition and Artificial Intelligence,2012,25(5):835-844.
[17] CARUCCIO L,DEUFEMIA V,NAUMANNE F,et al.Discovering relaxed functional dependencies based on multi-Attribute dominance [C]//Proceedings of the 37th InternationalConfe-rence on Data Engineering,2021:2354-2355.
[18] WEN F,HUANG H L,LI T D,et al.Research on library data fast mining algorithm based on FP-growth association rules[J].Journal of Chongqing University of Technology(Natural Science),2020,34(6):189-194.
[19] WANG D Y,LI L,ZHANG L,et al.Research and application of FP-Tree algorithm rule mining[J].Journal of Northeastern University(Natural Science Edition),2021,53(2):67-72.
[20] YAO P F.Research on operational data quality inspectionmethod based on association rules[J].Electronic Design Enginee-ring,2022,30(3):126-130.
[21] QIU X Q,HU Y M,ZHU A X,et al.Research on associatedrule-based error checking method on assessment index database of cultivated land quality:a case Study on Guangzhou city[J].China Land Science,2020,34(3):75-83.
[22] YU M,ZHAO X N,XU Z.Survey on using dependencies to improve data consistency[J].Journal of Computer Applications,2018,38(S2):72-76.
[23] DU Y F,SHEN D R,NIE T Z,et al.A cleaning method for consistency and currency in related data[J].Chinese Journal of Computers,2017,40(1):15.
[24] ZHU Y Q,ZHOU L W,CHEN G.Research of relationship between functional dependency and association rules in data-base theory teaching[J].Application Research of Computers,2014,31(7):2085-2087.
[25] CARPINETO C,ROMANO G,D'ADAMO P.Inferring dependencies from relations:a conceptual clustering approach[J].Computational Intelligence,1999,15(4):415-441.
[26] AN L X,ZHANG H Y.The mathematical mechanism of functional dependency and its deformation in the view of data mining[J].Fuzzy Systems and Mathematics,2020,34(6):1-11.
[27] ZAHNG C S,TU Y,WENG H,et al.Dependency discovery and data recovery of conditional functions based on association rules[J].Application Research of Computers,2016,33(2):384-387.
[28] XIONG Z M,WANG B,TAO R,et al.An association rule mining reduction algorithm based on determining prime attributes[J].Computer Engineering and Science,2021,43(4):738-745.
[29] ZHOU J L,DIAO X C,CAO J J.Mining of constant conditional functional dependencies based on pruning free itemsets[J].Journal of Tsinghua University(Science and Technology),2016,56(3):253-261.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市两江新区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发