基于全局属性注意力神经过程模型的数据补全研究

doi:10.11896/jsjkx.210800038

Computer Science ›› 2022, Vol. 49 ›› Issue (10): 111-117.doi: 10.11896/jsjkx.210800038

• Database & Big Data & Data Science • Previous Articles Next Articles

Study on Data Filling Based on Global-attributes Attention Neural Process Model

CHEN Kai¹, LIU Man^1,2, WANG Zhi-teng¹, MAO Shao-chen¹, SHEN Qiu-hui¹, ZHANG Hong-jun¹

1 Institute of Command and Control Engineering,Army Engineering University of PLA,Nanjing 210007,China
2 73131 Unit of PLA,Zhangzhou,Fujian 363000,China

Received:2021-08-03 Revised:2022-01-13 Online:2022-10-15 Published:2022-10-13
About author:CHENG Kai,born in 1983,Ph.D,asso-ciate professor.His main research in-terests include data analysis and data mining,artificial intelligence and planning techniques.
ZHANG Hong-jun,born in 1963,Ph.D,professor,Ph.D supervisor.His main research interests include data & knowledge engineering and computer simulation theory.
Supported by:
National Natural Science Foundation of China(61806221).

Abstract

Abstract: The attention neural process(ANP) model which adopts the method of generative model,takes any number context points of the sample as input,and outputs the distribution function of the entire sample,so as to approximate the function of Gaussian process regression(GPR) to realize the data fullfilling task.In reality,many scenes or datasets containe the attributes or labels data which are critical for generating the missing data.However,the ANP ignores full use of them.Inspired by CVAE model which control sample generation with lable as condition,this paper proposes global attribute attentional neural process(GANP),which embeds sample attributes or labels into ANP network to make the model generate samples more accurately,especially when the number of input context points are scarce.In detail,the sample attributes are embedded into the encoder network,so that the latent variables contain the sample attribute information.At the same time,the sample attributes are added as features in the decoder network to help generate more accurate samples.Finally,experimental results prove the superiority of GANP in both qualitative and quantitative,and it also reveals that GANP expands the application of NP families which can solve the Gaus-sian process regression problem more flexibly,quickly and accurately.

Key words: Neural process, Cross attention, Variational inference, Gaussian process, Global attribute

CLC Number:

TP391.4

CHEN Kai, LIU Man, WANG Zhi-teng, MAO Shao-chen, SHEN Qiu-hui, ZHANG Hong-jun. Study on Data Filling Based on Global-attributes Attention Neural Process Model[J].Computer Science, 2022, 49(10): 111-117.

References

[1]RASMUSSEN C E.Gaussian processes in machine learning[C]//Summer School on Machine Learning.Berlin:Springer,2003:63-71.
[2]QUINONERO-CANDELA J,RASMUSSEN C E.A unifyingview of sparse approximate Gaussian process regression[J].The Journal of Machine Learning Research,2005,6(65):1939-1959.
[3]GARNELO M,SCHWARZ J,ROSENBAUM D,et al.Neuralprocesses[J].arXiv:1807.01622,2018.
[4]WANG J X,KURTH-NELSON Z,TIRUMALA D,et al.Lear-ning to reinforcement learn[J].arXiv:1611.05763,2016.
[5]FINN C,ABBEEL P,LEVINE S.Model-agnostic meta-learning for fast adaptation of deep networks[C]//International Confe-rence on Machine Learning.PMLR,2017:1126-1135.
[6]KIM H,MNIH A,SCHWARZ J,et al.Attentive neural processes[J].arXiv:1901.05761,2019.
[7]GORDON J,BRUINSMA W P,FOONG A Y K,et al.Convolutional conditional neural processes[J].arXiv:1910.13556,2019.
[8]SINGH G,YOON J,SON Y,et al.Sequential neural processes[J].arXiv:1906.10264,2019.
[9]KUMAR S.Spatiotemporal Modeling using Recurrent NeuralProcesses[D].Pittsburgh,PA:Carnegie Mellon University,2019.
[10]ZHU J,QIN S,WANG W,et al.Probabilistic trajectory prediction for autonomous vehicles with attentive recurrent neural process[J].arXiv:1910.08102,2019.
[11]KUMAR A,ESLAMI S M,REZENDE D J,et al.Consistentgenerative query networks[J].arXiv:1807.02033,2018.
[12]MA Y,WANG S Q,MAO Y X.Path Planning of Mobile Robot Based on Neural Process-particle Swarm Optimization[J].Journal of Hubei University of Technology,2020,35(1):17-20.
[13]SUN X L,GUO Y,LI N,et al.Missing data imputing algorithm based on modified neural process[J].Journal of University of Chinese Academy of Sciences,2021,38(2):280-287.
[14]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-based lear-ning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[15]LIU Z,LUO P,WANG X,et al.Deep learning face attributes in the wild[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:3730-3738.
[16]BLUNDELL C,CORNEBISE J,KAVUKCUOGLU K,et al.Weight uncertainty in neural network[C]//International Conference on Machine Learning.PMLR,2015:1613-1622.
[17]GARNELO M,ROSENBAUM D,MADDISON C,et al.Conditional neural processes[C]//International Conference on Machine Learning.PMLR,2018:1704-1713.
[18]KINGMA D P,WELLING M.Auto-encoding variational bayes[J].arXiv:1312.6114,2013.
[19]DOERSCH C.Tutorial on variational autoencoders[J].arXiv:1606.05908,2016.
[20]SOHN K,LEE H,YAN X.Learning structured output repre-sentation using deep conditional generative models[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems.2015:3483-3491.
[21]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Advances in Neural Information Processing Systems.2017:5998-6008.
[22]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Study on Data Filling Based on Global-attributes Attention Neural Process Model

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 9

Metrics

Comments

Recommended 0

[1]	YANG Yue, FENG Tao, LIANG Hong, YANG Yang. Image Arbitrary Style Transfer via Criss-cross Attention [J]. Computer Science, 2022, 49(6A): 345-352.
[2]	ZHANG Mo-hua, PENG Jian-hua. Integration of Internal and External Priors Based on Dirichlet Process Mixture Model [J]. Computer Science, 2020, 47(5): 172-180.
[3]	LOU Hao-feng, ZHANG Duan. Gaussian Process Assisted CMA-ES Application in Medical Image Registration [J]. Computer Science, 2018, 45(11A): 234-237.
[4]	LIU Fu-yong, GAO Xian-qiang and ZHANG Zhu. Improved Bayesian Probabilistic Model Based Recommender System [J]. Computer Science, 2017, 44(5): 285-289.
[5]	HUI Jing-li, PAN Wei, WU Kang-kang and ZHOU Xiao-ying. Attribute Reduction Based on Asymmetric Variable Neighborhood Rough Set [J]. Computer Science, 2015, 42(6): 282-287.
[6]	LIU Jian-wei, CUI Li-peng, LI Hai-en and LUO Xiong-lin. Research and Development on Inference Technique in Probabilistic Graphical Models [J]. Computer Science, 2015, 42(4): 1-18.
[7]	. Baseline Algorithm Based on Adaptive Gaussian Process Machine Learning [J]. Computer Science, 2012, 39(11): 79-82.
[8]	LIU Chang-hong,YANG Yang,CHEN Yong. Incrementally Learning Human Pose Mapping Model [J]. Computer Science, 2010, 37(3): 268-270.
[9]	WANG Xiu-mei ,GAO Xin-bo （ School of Electronic Engineering, Xidian University, Xi＇an 710071, China ）. [J]. Computer Science, 2009, 36(3): 189-192.