Computer Science

Computer Science ›› 2023, Vol. 50 ›› Issue (11A): 230300073-6.doi: 10.11896/jsjkx.230300073

• Big Data & Data Science • Previous Articles     Next Articles

Soil Moisture Data Reconstruction Based on Low Rank Matrix Completion Method

WANG Shan, LIU Lu   

  1. School of Information Engineering,East China Jiaotong University,Nanchang 330006,China
  • Published:2023-11-09
  • About author:WANG Shan,born in 1981,Ph.D,professor,master supervisor.His main research interests include image proces-sing and artificial intelligence.
    LIU Lu,born in 1998,postgraduate.Her main research interests include image processing and matrix completion.
  • Supported by:
    National Natural Science Foundation of China(41965007).

PDF (PC)

422

Abstract: Soil moisture plays an important role in meteorology,climatology and other disciplines.However,the current observational soil moisture data lacks of high precision and high spatial resolution,and its applicability is greatly limited.Matrix completion(MC) is the application of compressed sensing on matrix.It aims at partial missing,contaminated and damaged large-scale data,and aims to recover all the data of the matrix from a low-rank incomplete matrix by using the correlation between its elements.It is applicable to data with high spatial and temporal correlation but many missing values,such as soil moisture.However,the matrix rank is required to be correlated or approximately correlated,while the rank of soil moisture is unstable.Therefore,we presuppose the rank of the matrix and introduce principal component analysis(PCA) to reduce the matrix dimension while retaining most of the information.On this basis,matrix filling of soil moisture data with missing values is carried out.The experiment selects ERA-Interim 2022 satellite soil moisture data in some areas,and the results show that,compared with traditional MC algorithms,the error of experiment results of low rank matrix completion(PCA-MC) using principal component analysis is reduced by 28.6%.The root mean square error is reduced by 5.78%,the maximum error is reduced by 14.8%,and the reconstruction time is shortened at the same time,which indicates that the PCA-MC method can effectively reconstruct the large-scale matrix with missing values compared with the MC method.

Key words: Soil moisture, Satellite data, Matrix completion, Principal component analysis, Data reconstruction

CLC Number: 

  • TP751
[1]JACKSON T J,SCHMUGGE J,ENGMAN E T.Remote Sen-sing Applications to Hydrology[J].Hydrological Sciences Journal/Journal des Sciences Hydrologiques,1996,41(4):517-530.
[2]THOMAS J,JACKSONIII.Measuring surface soil moistureusing passive microwave remote sensing[J].Hydrological Processes,1993,7(2):139-152.
[3]KOSTER R D,SUAREZ M J,HEISER M.Variance and predictability of precipitation at seasonal-to-interannual timescales [J].Hydrometeor,2000,1(1):26-46.
[4]YEH T,WETHERALD R T,MANABE S.The effect of soilmoisture on the short-term climate and hydrology change－A numerical experiment[J].Monthly Weather Review,1984,112(3):474-490.
[5]LI J Y,XI Y,ZHAO J F.Effects of soil moisture on soil methane uptake in tropical forests in southern China[J].Acta Ecologica Sinica,42(12):4978-4987.
[6]ENTIN J K,ROBOCK A,VINNIKOV K Y,et al.Evaluation of global soil wetness project soil moisture simulations[J].Journal of the Meteorological Society of Japan,2009,77(1):183-198.
[7]FENG Z M,YANG Y Z,DING X Q,et al.Optimization of spatial interpolation method for meteorological elements[J].Geographical Studies,2004(3):357-364.
[8]CANDES E,RECHT B.Exact matrix completion via convex optimization[J].Communications of the ACM,2012,55(6):111-119.
[9]KESHAVAN R,MONTANARI A,OH S.Matrix completionfrom noisy entries[J].Advances in Neural Information Proces-sing Systems,2010(69),22.
[10]BECKER S,CEVHER V,KYRILLIDIS A.Randomized low-memory singular value projection[J].arXiv:1303,0167,2013.
[11]ZHAO Y J,ZHENG B Y,CHEN S N.Compressed sensing and matrix filling and their applications in image processing[J].Journal of Nanjing University of Posts and Telecommunications(Natural Science Edition),2015,35(3):18-26.
[12]JOHNSON C R.Matrix Completion Problems:A Survey[M].Matrix Completion Problems:A Survey,1990.
[13]COVER T M,HART P E.Nearest neighbor pattern classification[J].IEEE Trans. Inf. Theory,1953,13(1):21-27.
[14]KONG L,JIANG D,WU M Y.Optimizing the spatio-temporal distribution of cyber-physical systems for environment abstraction[C] //2010 IEEE 30th International Conference on Distri-buted Computing Systems.2010:179-188.
[15]ZHU H,ZHU Y,LI M,et al.SEER:Metropolitan-scale traffic perception based on lossy sensory data[C]//IEEE INFOCOM 2009.2009:217-225.
[16]WANG S,HU J,SHAN H,et al.Temperature Field Data Reconstruction Using the Sparse Low-Rank Matrix Completion Method[J].Advances in Meteorology,2019,(Pt.2):3676182-1-3676182-10.
[17]SUN G L,LIU X C,LI Z Z,et al.A multi-time slot data acquisition method for ambient temperature and humidity based on matrix filling:China,CN 112036000 A[P].2020-12-04.https://www.zhuanlichaxun.net/pdf/10219422.html.
[18]LI C Y,ZHU L,BAO W Z,et al.Convex local sensitive low rank matrix approximation[C] //2017 International Joint Conference on Neural Networks(IJCNN).2017:256-261.
[19]TRIPATHI R,MOHAN B,RAJAWAT K.Adaptive low-rankmatrix completion[J].IEEE Transactions on Signal Processing,2017,65(14):3603-3616.
[20]CAI J F,CANDÈS E J,SHEN Z.A singular value thresholding algorithm for matrix completion[J].2010,20(4):1956-1982.
[21]CANDÉS E J,LI X,MA Y,et al.Robust principal componentanalysis?[J].Journal of the ACM (JACM),2011,58(3):1-37.
[22]CANDÈS E J,TAO T.The power of convex relaxation:Near-optimal matrix completion[J].IEEE Transactions on Information Theory,2010,56(5):2053-2080.
[23]CANDES E J,PLAN Y.Matrix completion with noise[J].Proceedings of the IEEE,2010,98(6):925-936.
[24]FAZEL M.Matrix rank minimization with applications[D].Stanford University,2002.
[25]JAIN P,MEKA R,DHILLON I.Guaranteed rank minimization via singular value projection[J].arXiv:0909.5457,2009.
[26]LIN Z,CHEN M,MA Y.The augmented lagrange multipliermethod for exact recovery of corrupted low-rank matrices[J].arXiv:1009.5055,2010.
[27]PEARSON K J T L,EDINBURGH,MAGAZINE D P,et al.On lines and planes of closest fit to systems of points in space[J].The London,Edinburgh,and Dublin Philosophical Magazine and Journal of Science,1901,2(11):559-572.
[28]HOTELLING H.Analysis of a complex of statistical variables into principal components[J].Journal of Educational Psycho-logy,1933,24(6):417.
[29]LIANG S J,ZHANG Z H,CUI L L,et al.Dimension reduction method based on principal component analysis and nuclear independent component analysis[J].Systems Engineering and electronics,2011,33(9):2144-2148.
[30]WEN Z,YIN W,ZHANG Y.Solving a low-rank factorizationmodel for matrix completion by a nonlinear successive over-relaxation algorithm[J].Mathematical Programming Computation,2012,4(4):333-361.
[31]SHI C X,XIE Z H,TIAN X J,et al.Soil moisture assimilation by satellite remote sensing based on EnKF algorithm[C] //Sa-tellite Remote sensing Application Technology and Processing Methods.2008 Annual Meeting of the Chinese Meteorological Society,2008:372.
[1] CAI Qiquan, LU Juhong, YU Zhiyong, HUANG Fangwan. Data Completion of Air Quality Index Based on Multi-dimensional Sparse Representation [J]. Computer Science, 2023, 50(8): 52-57.
[2] LI Kejia, HU Xuexian, CHEN Yue, YANG Hongjian, XU Yang, LIU Yang. Differential Privacy Linear Regression Algorithm Based on Principal Component Analysis andFunctional Mechanism [J]. Computer Science, 2023, 50(8): 342-351.
[3] LI Qi-ye, XING Hong-jie. KPCA Based Novelty Detection Method Using Maximum Correntropy Criterion [J]. Computer Science, 2022, 49(8): 267-272.
[4] QI Xiu-xiu, WANG Jia-hao, LI Wen-xiong, ZHOU Fan. Fusion Algorithm for Matrix Completion Prediction Based on Probabilistic Meta-learning [J]. Computer Science, 2022, 49(7): 18-24.
[5] QUE Hua-kun, FENG Xiao-feng, LIU Pan-long, GUO Wen-chong, LI Jian, ZENG Wei-liang, FAN Jing-min. Application of Grassberger Entropy Random Forest to Power-stealing Behavior Detection [J]. Computer Science, 2022, 49(6A): 790-794.
[6] GAO Ji-hang, ZHANG Yan. Fault Diagnosis of Shipboard Zonal Distribution Power System Based on FWA-PSO-MSVM [J]. Computer Science, 2022, 49(11A): 210800209-5.
[7] LIU Fang-zheng, MA Bo-wen, LYU Bo-feng, HUANG Ji-wei. UAV Base Station Deployment Method for Mobile Edge Computing [J]. Computer Science, 2022, 49(11A): 220200089-7.
[8] WU Shan-jie, WANG Xin. Prediction of Tectonic Coal Thickness Based on AGA-DBSCAN Optimized RBF Neural Networks [J]. Computer Science, 2021, 48(7): 308-315.
[9] HU Xin-tong, SHA Chao-feng, LIU Yan-jun. Post-processing Network Embedding Algorithm with Random Projection and Principal Component Analysis [J]. Computer Science, 2021, 48(5): 124-129.
[10] WANG Yi-hao, DING Hong-wei, LI Bo, BAO Li-yong, ZHANG Ying-jie. Prediction of Protein Subcellular Localization Based on Clustering and Feature Fusion [J]. Computer Science, 2021, 48(3): 206-213.
[11] FENG An-ran, WANG Xu-ren, WANG Qiu-yun, XIONG Meng-bo. Database Anomaly Access Detection Based on Principal Component Analysis and Random Tree [J]. Computer Science, 2020, 47(9): 94-98.
[12] ZHANG Ning, SHI Hong-wei, ZHENG Lang, SHAN Zi-hao, WU Hao-xiang. PCANet-based Multi-factor Stock Selection Model for Value Growth [J]. Computer Science, 2020, 47(11A): 64-67.
[13] HAN Xu, CHEN Hai-yun, WANG Yi, XU Jin. Face Recognition Using SPCA and HOG with Single Training Image Per Person [J]. Computer Science, 2019, 46(6A): 274-278.
[14] ZHANG Ming-yue, WANG Jing. Interactive Likelihood Target Tracking Algorithm Based on Deep Learning [J]. Computer Science, 2019, 46(2): 279-285.
[15] GAO Zhong-shi, SU Yang , LIU Yu-dong. Study on Intrusion Detection Based on PCA-LSTM [J]. Computer Science, 2019, 46(11A): 473-476.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.