计算机科学 ›› 2019, Vol. 46 ›› Issue (6A): 215-221.

• 模式识别与图像处理 • 上一篇    下一篇

结合卷积神经网络多层特征融合和K-Means聚类的服装图像检索方法

侯媛媛, 何儒汉, 李敏, 陈佳   

  1. 武汉纺织大学湖北省服装信息化工程技术研究中心 武汉430200
  • 出版日期:2019-06-14 发布日期:2019-07-02
  • 通讯作者: 何儒汉(1974-),男,博士,教授,主要研究方向为机器学习、计算机视觉、多媒体检索、图像和视频处理,E-mail:heruhan@wtu.edu.cn
  • 作者简介:侯媛媛(1995-),女,硕士生,主要研究方向为计算机视觉、深度学习、图像分析与处理;李 敏(1978-),女,博士,副教授,主要研究方向为计算机视觉、模式识别等;陈 佳(1982-),女,博士,副教授,主要研究方向为数据库、数据挖掘、图像处理等。
  • 基金资助:
    本文受国家自然科学基金面上项目(61170093)资助。

Clothing Image Retrieval Method Combining Convolutional Neural Network Multi-layerFeature Fusion and K-Means Clustering

HOU Yuan-yuan, HE Ru-han, LI Min, CHEN Jia   

  1. Engineering Research Center of Hubei Province for Clothing Information,Wuhan Textile University,Wuhan 430200,China
  • Online:2019-06-14 Published:2019-07-02

摘要: 随着服装电子商务的蓬勃发展,海量的服装图像数据被累积,对服装图像“以图搜图”成为了当前的一个热点研究方向。服装图像有着丰富的整体语义信息和大量细节信息,要对其实现精准检索是一项挑战性难题。传统的基于人工语义标注的服装图像方法和以人工设计的颜色与纹理等内容特征进行服装图像检索的方法均存在较大局限性。文中利用卷积神经网络多层特征融合提取特征,然后使用K-Means聚类加快服装图像的检索,充分利用深度卷积神经网络在图像特征提取上的有效性和层次性,融合不同卷积层次特征的细节信息和抽象语义信息以提升检索的准确度,并利用K-Means加快检索速度。所提方法首先对服装图像数据集进行统一的尺寸处理,然后利用卷积神经网络进行训练和特征提取,抽取出服装图像从低到高的多层次特征,进而将多种层次的特征进行融合,最终使用K-Means聚类方法对提取的图像库特征进行有效检索。在DeepFashion子类数据集Category and Attribute Prediction Benchmark和In-shop Clothes Retrieval Benchmark上的实验结果表明,所提方法能有效增强服装图像的特征表达能力,提高了检索准确率和检索速度,优于其他主流方法。

关键词: K-Means聚类, 服装图像检索, 卷积神经网络, 特征融合

Abstract: The booming of clothing e-commerce has accumulated a large amount of clothing image data,and the “image search” of clothing images has become a hot research direction.Apparel images have rich overall semantic information and a large amount of detailed information,and achieving accurate retrieval is a challenging problem.Traditional me-thods of clothing image based on artificial semantic annotation and methods of image retrieval based on artificially designed content features such as color and texture have significant limitations.This paper proposed a clothing image retrieval method based on multi-layer feature fusion and K-Means clustering of convolutional neural networks,which makes full use of the effectiveness and hierarchy of deep convolutional neural network in image feature extraction,fuses the detailed information and abstract semantic information of different convolutional hierarchical featuresto improve retrieval accuracy,and uses K-Means to improve the retrieval speed.The proposed method firstly performs uniform size processing on the clothing image data set,then uses the convolutional neural network for training and feature extraction,extracts multi-level features of the clothing image from low to high,and then fuses various levels of features.Finally,the K-Means clustering method is used to efficiently retrieve large-scale image data.The experimental results on the DeepFashion sub-category data set Category and Attribute Prediction Benchmark and In-shop Clothes Retrieval Benchmark show that the proposed method can effectively enhance the feature expression ability of clothing images,and improve its retrieval accuracy and retrieval speed.The proposed method is supprior to other mainstream methods.

Key words: Clothing image retrieval, Convolution neural network, Feature fusion, K-Means clustering

中图分类号: 

  • TP183
[1]ALBIOL A,MONZO D,MARTIN A,et al.Face recognition using HOG-EBGM[J].Pattern Recognition Letters,2008,29(10):1537-1543.
[2]LO T W R,SIEBERT J P.Local feature extraction and matching on range images:2.5D SIFT[J].Computer Vision & Image Understanding,2009,113(12):1235-1250.
[3]ZHOU L,GENG Z,ZHANG J,et al.ORB feature based web pornographic image recognition[J].Neurocomputing,2016,173(P3):511-517.
[4]贾巧丽,王娟,孔兵.基于形状特征和颜色的服装图像检索[J].现代计算机(专业版),2011(7):30-32.
[5]薛培培,邬延辉.基于图像内容和支持向量机的服装图像检索方法研究[J].移动通信,2016(2):79-82.
[6]胡玉平,肖行,罗东俊.基于GrabCut改进算法的服装图像检索方法[J].计算机科学,2016,43(S2):242-246.
[7]HINTON G,OSINDERO S.A fast learning algorithm for deep belief nets [J].Neural Computation,2006,18(7):1527-1554.
[8]CIRESAN D,MEIER U,SCHMIDHUBER J.Multi-column Deep Neural Networks for Image Classification [C]∥Procee-dings of IEEE Conference on Computer Vision and Pattern Re-cognition.Washington D C,USA:IEEE Press,2012:3642-3649.
[9]CIRSHICK R,DONAHUE J,DARRELL T,et al.Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation [C]∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D C,USA:IEEE Press,2014:580-587.
[10]LIN K,YANG H F,LIU K H,et al.Rapid clothing retrieval via deep learning of binary codes and hierarchical search[C]∥Proceedings of the 5th ACM on International Conference on Multimedia Retrieval.ACM,2015:498-502.
[11]KRIZHECSY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks[C]∥Advances in Neural Information Processing Systems,2012,25(2):1097-1105.
[12]KIAPOU M H,HAN X,LAZEBNIK S,et al.Where to Buy It:Matching Street Clothing Photos in Online Shopes[C]∥2015 IEEE International Conference on Computer Vision(ICCV).Santiago:IEEE,2015:3343-3351.
[13]FUKUSHIMA K.Neocognitron:a self-organizing neural net-work model for a mechanism of pattern recognition unaffected by shift in position [J].Biological Cybernetics,1980,36(4):193-202.
[14]王利华,邹俊忠,张见,等.基于深度卷积神经网络的快速图像分类算法[J].计算机工程与应用,2017,53(13):181-188.
[15]刘海龙,李宝安,吕学强,等.基于深度卷积神经网络的图像检索算法研究[J].计算机应用研究,2017,34(12):3816-3819.
[16]YIM J,JU J,JUNG H,et al.Image Classification Using Convolutional Neural Networks With Multi-stage Feature [M]∥Robot Intelligence Technology and Applications 3.Springer International Publishing,2015.
[17]SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]∥2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Boston:IEEE,2015.
[18]SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[J].Computer Science,2014,1(2):3.
[19]HE K,ZHANG X,REN S,et al.Deep Residual Learning for Ima-ge Recognition [C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE Computer Society,2016:770-778.
[20]SCHROFF F,KALENICHENKO D,PHILBIN J.FaceNet:A unified embedding for face recognition and clustering[C]∥2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Boston:IEEE,2015:815-823.
[21]SURAL S,QIAN G,PRAMANIK S.Segmentation and histogram generation using the HSV color space for image retrieval[C]∥International Conference on Image Processing.IEEE,2002:589-592.
[22]DALAL N,TRIGGS B.Histograms of oriented gradients for human detection[C]∥IEEE Computer Society Conference on Computer Vision and Pattern Recognition.IEEE,2005:886-893.
[23]CHANG C C,LIN C J.LIBSVM:A library for support vector machines[J].ACM Transactions on Intelligent Systems and Technology,2011,2(3):1-27.
[1] 周乐员, 张剑华, 袁甜甜, 陈胜勇.
多层注意力机制融合的序列到序列中国连续手语识别和翻译
Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion
计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[2] 李宗民, 张玉鹏, 刘玉杰, 李华.
基于可变形图卷积的点云表征学习
Deformable Graph Convolutional Networks Based Point Cloud Representation Learning
计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[3] 陈泳全, 姜瑛.
基于卷积神经网络的APP用户行为分析方法
Analysis Method of APP User Behavior Based on Convolutional Neural Network
计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121
[4] 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥.
基于注意力机制的医学影像深度哈希检索算法
Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism
计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[5] 檀莹莹, 王俊丽, 张超波.
基于图卷积神经网络的文本分类方法研究综述
Review of Text Classification Methods Based on Graph Convolutional Network
计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064
[6] 金方焱, 王秀利.
融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取
Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM
计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
[7] 张颖涛, 张杰, 张睿, 张文强.
全局信息引导的真实图像风格迁移
Photorealistic Style Transfer Guided by Global Information
计算机科学, 2022, 49(7): 100-105. https://doi.org/10.11896/jsjkx.210600036
[8] 戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮.
基于DNGAN的磁共振图像超分辨率重建算法
Super-resolution Reconstruction of MRI Based on DNGAN
计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105
[9] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[10] 刘月红, 牛少华, 神显豪.
基于卷积神经网络的虚拟现实视频帧内预测编码
Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network
计算机科学, 2022, 49(7): 127-131. https://doi.org/10.11896/jsjkx.211100179
[11] 徐鸣珂, 张帆.
Head Fusion:一种提高语音情绪识别的准确性和鲁棒性的方法
Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition
计算机科学, 2022, 49(7): 132-141. https://doi.org/10.11896/jsjkx.210100085
[12] 孙福权, 崔志清, 邹彭, 张琨.
基于多尺度特征的脑肿瘤分割算法
Brain Tumor Segmentation Algorithm Based on Multi-scale Features
计算机科学, 2022, 49(6A): 12-16. https://doi.org/10.11896/jsjkx.210700217
[13] 吴子斌, 闫巧.
基于动量的映射式梯度下降算法
Projected Gradient Descent Algorithm with Momentum
计算机科学, 2022, 49(6A): 178-183. https://doi.org/10.11896/jsjkx.210500039
[14] 杨涵, 万游, 蔡洁萱, 方铭宇, 吴卓超, 金扬, 钱伟行.
基于步态分类辅助的虚拟IMU的行人导航方法
Pedestrian Navigation Method Based on Virtual Inertial Measurement Unit Assisted by GaitClassification
计算机科学, 2022, 49(6A): 759-763. https://doi.org/10.11896/jsjkx.211200148
[15] 郁舒昊, 周辉, 叶春杨, 王太正.
SDFA:基于多特征融合的船舶轨迹聚类方法研究
SDFA:Study on Ship Trajectory Clustering Method Based on Multi-feature Fusion
计算机科学, 2022, 49(6A): 256-260. https://doi.org/10.11896/jsjkx.211100253
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!