Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (1): 198-206.doi: 10.11896/jsjkx.230500232

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Seal Removal Based on Generative Adversarial Gated Convolutional Network

WU Guibin1, YANG Zongyuan1, XIONG Yongping1, ZHANG Xing2, WANG Wei2   

  1. 1 School of Computer Science(National Pilot Software Engineering School),Beijing University of Posts and Telecommunications,Beijing 100876,China
    2 Artificial Intelligence Lab of China Resources Digital Technology,Shenzhen,Guangdong 518049,China
  • Received:2023-05-31 Revised:2023-09-15 Online:2024-01-15 Published:2024-01-12
  • About author:WU Guibin,born in 1982,Ph.D.His main research interests include natural language processing,image processing and deep learning.
    XIONG Yongping,born in 1982,Ph.D,associate professor.His main research interests include document intelligence and OCR,visual IoT and machine vision.
  • Supported by:
    Research on Key Technology of Multi-spectral Optical Imaging inside GIS based on Fiber Bundle Image Transmission(5500-202216134A-1-1-ZN).

PDF (PC)

1544

Abstract: Seals on invoices and documents seriously affect the accuracy of text recognition,so seal elimination techniques play an important role in the pre-processing of document analysis,and document enhancement.However,threshold-based methods and deep learning-based methods suffer from incomplete seal elimination and modification of background pixels.Thus,this paper proposes a two-stage seal elimination network,SealErase.The first stage is a U-shaped segmentation network for generating bina-rized masks with seal position,and the second stage is an inpainting network for refined seal elimination.Due to the lack of available public paired datasets for seal elimination,existing methods cannot design pixel-level evaluation metrics to measure the quality of the generated images.Moreover,training the neural network using paired training sets can effectively improve the performance of the network.To this end,this paper constructs a high-simulated seal elimination dataset containing 8 000 samples,taking into account the generalisation to real scenes and the robustness to noise.The seals are divided into two types:seals in real document images and synthetic seals.In order to objectively evaluate the performance of SealErase,it devises a comprehensive evaluation metric based on the image generation quality and the recognition accuracy of characters obscured by seals to evaluate the elimination performance of the SealErase network.The existing seal elimination methods are compared on the seal elimination dataset,and the experimental results show that the SealErase network improve the peak signal to noise ratio by 26.79% and the mean structural similarity by 4.48% in the evaluation metric of image generation quality compared to the state-of-the-art methods.After seal elimination by SealErase network,the accuracy of recognition of characters obscured by seals is improved by 38.86%.Experimental results show that SealErase is equally effective in eliminating seals and preserving the obscured characters in real scenes.

Key words: Seal removal, Image inpainting, Seal synthesis, Generative adversarial networks, Gated Convolutions, SealErase

CLC Number: 

  • TP391
[1]SINGH A,BACCHUWAR K,BHASIN A.A survey of OCRapplications[J].International Journal of Machine Learning and Computing,2012,2(3):314-319.
[2]ZHAO Y T,LI Z M,WANG H J,et al.Image preprocessed study of the seal imprint verification[J].Chinese Journal of Scientific Instrument,2004,25(z3):401-403,410.
[3]JI J J,LOU Z.Filtering of color seal on bank notes based on re-segmentation[J].Modern Electronics Technique,2014,37(22):5-9.
[4]LI X L,ZOU C M,YANG G T,et al.SealGAN:Research on the seal elimination based on generative adversarial network[J].Acta Automatica Sinica,2021,47(11):2614-2622.
[5]ZHU J Y,PARK T,ISOLA P,et al.Unpaired image-to-imagetranslation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2017:2223-2232.
[6]WANG J,MIAO J,QING L Y,et al.Seal removal based on Pix2Pix network[J].Journal of Beijing Information Science & Technology University,2021,36(4):39-43.
[7]ISOLA P,ZHU J Y,ZHOU T,et al.Image-to-image translation with conditional adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2017:1125-1134.
[8]WANG Z,BOVIK A C,SHEIKH H R,et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004,13(4):600-612.
[9]ANVARI Z,ATHITSOS V.A Survey on Deep learning based Document Image Enhancement[EB/OL].https://arxiv.org/pdf/2112.02719.pdf.
[10]HRADIS M,KOTERA J,ZEMCIK P,et al.Convolutional neural networks for direct text deblurring[C]//Proceedings of BMVC.2015.
[11]GANGEH M J,TIYYAGURA S R,DASARATHA S V,et al.Document enhancement system using auto-encoders[C]//Workshop on Document Intelligence at NeurIPS 2019.2019.
[12]MAO X,SHEN C,YANG Y B.Image restoration using verydeep convolutional encoder-decoder networks with symmetric skip connections[J].arXiv:1603.09056,2016.
[13]SOUIBGUI M A,KESSENTINI Y.DE-GAN:a conditional ge-nerative adversarial network for document enhancement[J].ar-Xiv:2010.08764,2020.
[14]MIRZA M,OSINDERO S.Conditional generative adversarialnets[EB/OL].https://arxiv.org/pdf/1411.1784.pdf.
[15]JEMNI S K,SOUIBGUI M A,KESSENTINI Y,et al.Enhance to read better:A Multi-Task Adversarial Network for Handwritten Document Image Enhancement[J].Pattern Recognition,2022,123:108370-108383.
[16]ZHAO L L,SHEN L,HONG R C.Survey on image inpainting research progress[J].Computer Science,2021,48(3):14-26.
[17]ELHARROUSS O,ALMAADEED N,AL-MAADEED S,et al.Image inpainting:A review[J].Neural Processing Letters,2020,51(2):2007-2028.
[18]LIU G,REDA F A,SHIH K J,et al.Image inpainting for irre-gular holes using partial convolutions[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:85-100.
[19]YU J,LIN Z,YANG J,et al.Free-form image inpainting with gated convolution[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.IEEE,2019:4471-4480.
[20]RADFORD A,METZ L,CHINTALA S.Unsupervised repre-sentation learning with deep convolutional generative adversarial networks[EB/OL].https://arxiv.org/pdf/1511.06434.pdf.
[21]PATHAK D,KRAHENBUHL P,DONAHUE J,et al.Context encoders:Feature learning by inpainting[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2016:2536-2544.
[22]YU J,LIN Z,YANG J,et al.Generative image inpainting with contextual attention[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2018:5505-5514.
[23]YU Y,ZHAN F,LU S,et al.WaveFill:A Wavelet-based Gene-ration Network for Image Inpainting[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.IEEE,2021:14114-14123.
[24]NAKAMURA T,ZHU A,YANAI K,et al.Scene text eraser[C]//2017 14th IAPR International Conference on Document Analysis and Recognition(ICDAR).IEEE,2017:832-837.
[25]ZHANG S,LIU Y,JIN L,et al.2019.Ensnet:Ensconce text in the wild[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:801-808.
[26]LIU C,LIU Y,JIN L,et al.EraseNet:End-to-end text removal in the wild[J].IEEE Transactions on Image Processing,2020,29:8760-8775.
[27]MILLETARI F,NAVAB N,AHMADI S A.V-net:Fully convolutional neural networks for volumetric medical image segmentation[C]//2016 fourth International Conference on 3D vision(3DV).IEEE,2016:565-571.
[28]MIYATO T,KATAOKA T,KOYAMA M,et al.Spectral normalization for generative adversarial networks[EB/OL].https://arxiv.org/pdf/1802.05957.pdf.
[1] YAN Yan, SUI Yi, SI Jianwei. Remote Sensing Image Pan-sharpening Method Based on Generative Adversarial Network [J]. Computer Science, 2023, 50(8): 133-141.
[2] WANG Jinwei, ZENG Kehui, ZHANG Jiawei, LUO Xiangyang, MA Bin. GAN-generated Face Detection Based on Space-Frequency Convolutional Neural Network [J]. Computer Science, 2023, 50(6): 216-224.
[3] XU Guo-ning, CHEN Yi-peng, CHEN Yi-ming, CHEN Jin-yin, WEN Hao. Data Debiasing Method Based on Constrained Optimized Generative Adversarial Networks [J]. Computer Science, 2022, 49(6A): 184-190.
[4] XU Hui, KANG Jin-meng, ZHANG Jia-wan. Digital Mural Inpainting Method Based on Feature Perception [J]. Computer Science, 2022, 49(6): 217-223.
[5] DOU Zhi, WANG Ning, WANG Shi-jie, WANG Zhi-hui, LI Hao-jie. Sketch Colorization Method with Drawing Prior [J]. Computer Science, 2022, 49(4): 195-202.
[6] GAO Zhi-yu, WANG Tian-jing, WANG Yue, SHEN Hang, BAI Guang-wei. Traffic Prediction Method for 5G Network Based on Generative Adversarial Network [J]. Computer Science, 2022, 49(4): 321-328.
[7] LI Si-quan, WAN Yong-jing, JIANG Cui-ling. Multiple Fundamental Frequency Estimation Algorithm Based on Generative Adversarial Networks for Image Removal [J]. Computer Science, 2022, 49(3): 179-184.
[8] TAN Xin-yue, HE Xiao-hai, WANG Zheng-yong, LUO Xiao-dong, QING Lin-bo. Text-to-Image Generation Technology Based on Transformer Cross Attention [J]. Computer Science, 2022, 49(2): 107-115.
[9] ZHOU Shi-jin, XING Hong-jieHebei. Memory-augmented GAN-based Anomaly Detection [J]. Computer Science, 2022, 49(11A): 211000202-9.
[10] ZHANG Wei-qi, TANG Yi-feng, LI Lin-yan, HU Fu-yuan. Image Stream From Paragraph Method Based on Scene Graph [J]. Computer Science, 2022, 49(1): 233-240.
[11] XU Tao, TIAN Chong-yang, LIU Cai-hua. Deep Learning for Abnormal Crowd Behavior Detection:A Review [J]. Computer Science, 2021, 48(9): 125-134.
[12] LIN Zhen-xian, ZHANG Meng-kai, WU Cheng-mao, ZHENG Xing-ning. Face Image Inpainting with Generative Adversarial Network [J]. Computer Science, 2021, 48(9): 174-180.
[13] PAN Xiao-qin, LU Tian-liang, DU Yan-hui, TONG Xin. Overview of Speech Synthesis and Voice Conversion Technology Based on Deep Learning [J]. Computer Science, 2021, 48(8): 200-208.
[14] WANG Jian-ming, LI Xiang-feng, YE Lei, ZUO Dun-wen, ZHANG Li-ping. Medical Image Deblur Using Generative Adversarial Networks with Channel Attention [J]. Computer Science, 2021, 48(6A): 101-106.
[15] YE Hong-liang, ZHU Wan-ning, HONG Lei. Music Style Transfer Method with Human Voice Based on CQT and Mel-spectrum [J]. Computer Science, 2021, 48(6A): 326-330.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.