Computer Science ›› 2015, Vol. 42 ›› Issue (9): 56-60, 65.doi: 10.11896/j.issn.1002-137X.2015.09.012

Previous Articles     Next Articles

Representation Ability Research of Auto-encoders in Deep Learning

WANG Ya-si, YAO Hong-xun, SUN Xiao-shuai, XU Peng-fei and ZHAO Si-cheng   

  • Online:2018-11-14 Published:2018-11-14

Abstract: Deep learning frameworks and unsupervised learning methods have become increasingly popular and attracted the attention of many researchers in machine learning and artificial intelligence fields.This paper started from the “building-blocks” of deep learning methods and focused on the representation ability research of auto-encoders,especially on auto-encoders’ ability to reduce dimensionality and the stability of their representation ability.We expected that starting from the basis of deep learning can help us understand it better.Firstly,auto-encoder and restricted Boltzmann machine are two "building-blocks" of deep learning methods,both of which can be used to transform representation and be seen as relatively new nonlinear dimensionality reduction methods.Secondly,we investigated whether auto-encoder is a good representation transformation method under the context of understanding visual features,including eva-luating single-layer auto-encoder’s representation ability compared with classic methodology principal component analysis.Experiments based on original pixels and local descriptors demonstrate auto-encoder’s ability to reduce dimensiona-lity,the stability of its representation ability and the effectiveness of proposed AE-based transformation strategy.Fina-lly,future research direction was discussed.

Key words: Deep learning,Representation transformation,Dimensionality reduction,Single-layer auto-encoder

[1] 谭璐.高维数据的降维理论及应用[D].长沙:国防科学技术大学,2005Tan Lu.The theory and application of the dimension reduction on the high-dimensional data set [D].Changsha:National university of defense technology,2005
[2] 蒋宗礼.人工神经网络导论[M].北京:高等教育出版社,2001 Jiang Zong-li.Introduction to artificial neural networks [M].Bei Jing:Higher Education Press,2001
[3] Hinton G,Salakhutdinov R.Reducing the dimensionality of data with neural networks [J].Science,2006,313(5786):504-507
[4] Le Q V,Ranzato M A,Monga R.Building high-level scale unsupervised learning [C]∥International Conference on Acoustics,Speech and Signal Processing.Vancouver,Canada,2013:8595-8598
[5] Krizhevsky A,Sutskever I,Hinton G.Imagenet classificationwith deep convolutional neural networks [C]∥Advances in Neural Information Processing Systems.California,America,2012:1106-1114
[6] Zeiler M D,Fergus R.Visualizing and Understanding Convolutional Neural Networks [J].arXiv:1311.2901,2013
[7] 朱明,武妍.基于深度网络的图像处理研究[J].电子技术与软件工程,2014(5):101-102 Zhu Ming,Wu Yan.Image processing research based on deep networks [J].Electronic Technology and Software Engineering,2014(5):101-102
[8] Bengio Y.Learning deep architectures for AI [J].Foundations and Trends in Machine Learning,2009,2(1):1-12

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75, 88 .
[2] XIA Qing-xun and ZHUANG Yi. Remote Attestation Mechanism Based on Locality Principle[J]. Computer Science, 2018, 45(4): 148 -151, 162 .
[3] LI Bai-shen, LI Ling-zhi, SUN Yong and ZHU Yan-qin. Intranet Defense Algorithm Based on Pseudo Boosting Decision Tree[J]. Computer Science, 2018, 45(4): 157 -162 .
[4] WANG Huan, ZHANG Yun-feng and ZHANG Yan. Rapid Decision Method for Repairing Sequence Based on CFDs[J]. Computer Science, 2018, 45(3): 311 -316 .
[5] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[6] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[7] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[8] LIU Qin. Study on Data Quality Based on Constraint in Computer Forensics[J]. Computer Science, 2018, 45(4): 169 -172 .
[9] ZHONG Fei and YANG Bin. License Plate Detection Based on Principal Component Analysis Network[J]. Computer Science, 2018, 45(3): 268 -273 .
[10] SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99, 116 .