Computer Science ›› 2015, Vol. 42 ›› Issue (9): 56-60.doi: 10.11896/j.issn.1002-137X.2015.09.012

Previous Articles     Next Articles

Representation Ability Research of Auto-encoders in Deep Learning

WANG Ya-si, YAO Hong-xun, SUN Xiao-shuai, XU Peng-fei and ZHAO Si-cheng   

  • Online:2018-11-14 Published:2018-11-14

Abstract: Deep learning frameworks and unsupervised learning methods have become increasingly popular and attracted the attention of many researchers in machine learning and artificial intelligence fields.This paper started from the “building-blocks” of deep learning methods and focused on the representation ability research of auto-encoders,especially on auto-encoders’ ability to reduce dimensionality and the stability of their representation ability.We expected that starting from the basis of deep learning can help us understand it better.Firstly,auto-encoder and restricted Boltzmann machine are two "building-blocks" of deep learning methods,both of which can be used to transform representation and be seen as relatively new nonlinear dimensionality reduction methods.Secondly,we investigated whether auto-encoder is a good representation transformation method under the context of understanding visual features,including eva-luating single-layer auto-encoder’s representation ability compared with classic methodology principal component analysis.Experiments based on original pixels and local descriptors demonstrate auto-encoder’s ability to reduce dimensiona-lity,the stability of its representation ability and the effectiveness of proposed AE-based transformation strategy.Fina-lly,future research direction was discussed.

Key words: Deep learning,Representation transformation,Dimensionality reduction,Single-layer auto-encoder

[1] 谭璐.高维数据的降维理论及应用[D].长沙:国防科学技术大学,2005Tan Lu.The theory and application of the dimension reduction on the high-dimensional data set [D].Changsha:National university of defense technology,2005
[2] 蒋宗礼.人工神经网络导论[M].北京:高等教育出版社,2001 Jiang Zong-li.Introduction to artificial neural networks [M].Bei Jing:Higher Education Press,2001
[3] Hinton G,Salakhutdinov R.Reducing the dimensionality of data with neural networks [J].Science,2006,313(5786):504-507
[4] Le Q V,Ranzato M A,Monga R.Building high-level scale unsupervised learning [C]∥International Conference on Acoustics,Speech and Signal Processing.Vancouver,Canada,2013:8595-8598
[5] Krizhevsky A,Sutskever I,Hinton G.Imagenet classificationwith deep convolutional neural networks [C]∥Advances in Neural Information Processing Systems.California,America,2012:1106-1114
[6] Zeiler M D,Fergus R.Visualizing and Understanding Convolutional Neural Networks [J].arXiv:1311.2901,2013
[7] 朱明,武妍.基于深度网络的图像处理研究[J].电子技术与软件工程,2014(5):101-102 Zhu Ming,Wu Yan.Image processing research based on deep networks [J].Electronic Technology and Software Engineering,2014(5):101-102
[8] Bengio Y.Learning deep architectures for AI [J].Foundations and Trends in Machine Learning,2009,2(1):1-12

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!