Computer Science ›› 2024, Vol. 51 ›› Issue (5): 92-99.doi: 10.11896/jsjkx.231100067

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Multi-stage Intelligent Color Restoration Algorithm for Black-and-White Movies

SONG Jianfeng, ZHANG Wenying, HAN Lu, HU Guozheng, MIAO Qiguang   

  1. School of Computer Science and Technology,Xidian University,Xi'an 710071,China
  • Received:2023-11-10 Revised:2024-03-25 Online:2024-05-15 Published:2024-05-08
  • About author:SONG Jianfeng,born in 1978,associate professor.His main research interests include computer vision and deep lear-ning.
  • Supported by:
    Continuing Education Teaching Reform Research Program of Xidian University(JA2301) and National Natural Science Foundation of China(62272364).

Abstract: In the process of colorization for black and white movies,the existing automatic colorization models often produce singular result,and the reference-based colorization methods require users to specify reference images,posing a significant challenge in meeting the high requirement of reference images and consuming substantial human efforts.To address this issue,this paper proposes a multi-stage intelligent color restoration algorithm for black-and-white movies (MSICRA).Firstly,the movie is splitted into multiple scene segments by the VGG19 network. Secondly,each scene segment is cut frame by frame,and the edge intensity and grayscale difference of each frame image are used as criteria to assess image clarity,selecting images with clarity ranging from 0.95 to 1 in each scene.Subsequently,we select the first frame that meets the clarity criteria from the filtered images and apply different render factor values to colorize the selected image.We assess the colorization effects using saturation and choose the appropriate render factor for the colorization.Finally,we use the mean squared error between the pre-colorized and post-colorized images to select the best quality colorized image as the reference for the scene segment.Experimental results demonstrate that the proposed algorithm improves the PSNR by 1.32% for the black and white films Lei Feng and 2.15% for The Eternal Wave,and the SSIM by 1.84% and 1.04% respectively.The algorithm not only enables fully automatic colorization but also produces realistic colors that align with human perception.

Key words: Deep learning, Auto-colorization, Scene segmentation, Clarity

CLC Number: 

  • TP315
[1]LEI C Y,CHEN Q F.Fully Automatic Video Colorization with Self-Regularization and Diversity[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach,USA,2019:3753-3761.
[2]LIU Y,ZHAO H,CHAN K C K,et al.Temporally consistent video colorization with deep feature propagation and self-regularization learning[J].arXiv:2110.04562,2021.
[3]ZHAO Y,PO L M,YU W Y,et al.VCGAN:video colorization with hybrid generative adversarial network[J].IEEE Transactions on Multimedia,2022,25:3017-3032.
[4]ZHAO Y,PO L M,CHEUNG K W,et al.SCGAN:Saliency Map-guided Colorization with Generative Adversarial Network [J].IEEE Transactions on Circuits and Systems for Video Technology,2020,31(8):3062-3077.
[5]AKIMOTO N,HAYAKAWA A,SHIN A,et al.Reference-based video colorization with spatiotemporal correspondence[J].arXiv:2011.12528,2020.
[6]ZHANG B,HE M M,LIAO J,et al.Deep Exemplar-based VideoColorization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach,USA,2019:8052-8061.
[7]SHI M,ZHANG J Q,CHEN S Y,et al.Reference-based deepline art video colorization[J].IEEE Transactions on Visualization & Computer Graphics,2023,29(6):2965-2979.
[8]HE M M,CHEN D D,LIAO J,et al.Deep exemplar-based colo-rization[J].ACM Transac tions on Graphics (TOG),2018,37(4):1-16.
[9]WAN Z Y,ZHANG B,CHEN D D,et al.Bringing Old FilmsBack to Life[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans,USA,2022:17694-17703.
[10]JI X,JIANG B,LUO D,et al.ColorForm er:Image colorization via color memory assist ed hybrid-attention transformer[C]//European Conference on Computer Vision.Tel-Aviv,Israel,2022:20-36.
[11]ANTIC J.DeOldify [EB/OL].(2018-10-31) [2021-12-10].https://github.com/jantic/DeOld ify.
[12]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv:1409.1556,2014.
[13]WANG X M,ZHANG S Y,ZHANG J,et al.Contour Recon-struction Method for Noisy Image Based on Depth Residual Learning[J].Journal of Xidian University,2020,47(3):66-71.
[14]VASWANI A,SHAZEER N,PARMAR N,et al.Attention Is All You Need[C]//Proceedings of the Thirty-first Conference on Neural Information Processing Systems.Long Beach,USA,2017:1-11.
[15]RONNEBERGER O,FISCHER P,BROX T.U-net:Convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention(MICCAI 2015).Munich,Germany,2015:234-241.
[16]HE K M,ZHANG X Y,REN S Q,et al.Deep Residual Learning for Image Recogni tion[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition.Las Vegas,USA,2016:770-778.
[17]JIANG G Y,HUANG D J,WANG X,et al.Overview on Image Quality Assessment Methods[J].Journal of Electronics & Information Technology,2010,32(1):219-226.
[1] BAO Kainan, ZHANG Junbo, SONG Li, LI Tianrui. ST-WaveMLP:Spatio-Temporal Global-aware Network for Traffic Flow Prediction [J]. Computer Science, 2024, 51(5): 27-34.
[2] ZHANG Jianliang, LI Yang, ZHU Qingshan, XUE Hongling, MA Junwei, ZHANG Lixia, BI Sheng. Substation Equipment Malfunction Alarm Algorithm Based on Dual-domain Sparse Transformer [J]. Computer Science, 2024, 51(5): 62-69.
[3] HE Shiyang, WANG Zhaohui, GONG Shengrong, ZHONG Shan. Cross-modal Information Filtering-based Networks for Visual Question Answering [J]. Computer Science, 2024, 51(5): 85-91.
[4] HE Xiaohui, ZHOU Tao, LI Panle, CHANG Jing, LI Jiamian. Study on Building Extraction from Remote Sensing Image Based on Multi-scale Attention [J]. Computer Science, 2024, 51(5): 134-142.
[5] XU Xuejie, WANG Baohui. Multi-label Patent Classification Based on Text and Historical Data [J]. Computer Science, 2024, 51(5): 172-178.
[6] LI Zichen, YI Xiuwen, CHEN Shun, ZHANG Junbo, LI Tianrui. Government Event Dispatch Approach Based on Deep Multi-view Network [J]. Computer Science, 2024, 51(5): 216-222.
[7] HONG Tijing, LIU Dengfeng, LIU Yian. Radar Active Jamming Recognition Based on Multiscale Fully Convolutional Neural Network and GRU [J]. Computer Science, 2024, 51(5): 306-312.
[8] SUN Jing, WANG Xiaoxia. Convolutional Neural Network Model Compression Method Based on Cloud Edge Collaborative Subclass Distillation [J]. Computer Science, 2024, 51(5): 313-320.
[9] CHEN Runhuan, DAI Hua, ZHENG Guineng, LI Hui , YANG Geng. Urban Electricity Load Forecasting Method Based on Discrepancy Compensation and Short-termSampling Contrastive Loss [J]. Computer Science, 2024, 51(4): 158-164.
[10] LIN Binwei, YU Zhiyong, HUANG Fangwan, GUO Xianwei. Data Completion and Prediction of Street Parking Spaces Based on Transformer [J]. Computer Science, 2024, 51(4): 165-173.
[11] SONG Hao, MAO Kuanmin, ZHU Zhou. Algorithm of Stereo Matching Based on GAANET [J]. Computer Science, 2024, 51(4): 229-235.
[12] XUE Jinqiang, WU Qin. Progressive Multi-stage Image Denoising Algorithm Combining Convolutional Neural Network and
Multi-layer Perceptron
[J]. Computer Science, 2024, 51(4): 243-253.
[13] HUANG Kun, SUN Weiwei. Traffic Speed Forecasting Algorithm Based on Missing Data [J]. Computer Science, 2024, 51(3): 72-80.
[14] ZHENG Cheng, SHI Jingwei, WEI Suhua, CHENG Jiaming. Dual Feature Adaptive Fusion Network Based on Dependency Type Pruning for Aspect-basedSentiment Analysis [J]. Computer Science, 2024, 51(3): 205-213.
[15] CHEN Jinyin, LI Xiao, JIN Haibo, CHEN Ruoxi, ZHENG Haibin, LI Hu. CheatKD:Knowledge Distillation Backdoor Attack Method Based on Poisoned Neuronal Assimilation [J]. Computer Science, 2024, 51(3): 351-359.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!