Computer Science ›› 2026, Vol. 53 ›› Issue (7): 34-44.doi: 10.11896/jsjkx.250400046

• Computer Graphics & Multimedia • Previous Articles     Next Articles

FFiT:Faster Frame Interpolation Transformer Based on FasterViT

CHENG Zhirong, XU Yang   

  1. College of Big Data and Information Engineering,Guizhou University,Guiyang 550025,China
  • Received:2025-04-10 Revised:2025-07-14 Online:2026-07-15 Published:2026-07-10
  • About author:CHENG Zhirong,born in 2000,postgraduate.His main research interests include CNN+Transformer and motion video deblur.
    XU Yang,born in 1980,associate professor,is a member of CCF(No.D7127M).His main research interests include deep learning and action gesture recognition.
  • Supported by:
    Guizhou Province Science and Technology Plan(Qiankehe Achievements(2024)Major 004).

Abstract: In the realm of dynamic scene image acquisition,consumer-grade imaging devices commonly suffer from a compound degradation problem characterized by coexisting limited frame rates and motion blur.These issues primarily stem from inherent hardware architecture deficiencies and suboptimal exposure parameter optimization.Prevailing methodologies aimed at mitigating these degradations encounter significant technical bottlenecks in real-world blurry scenarios,including insufficient reconstruction accuracy,restricted model generalization capabilities,and low computational efficiency.To address these limitations,this paper introduces FFiT(Faster Frame Interpolation Transformer),a novel joint frame interpolation and motion decoupling optimization framework based on the FasterViT architecture.FFiT is designed to achieve efficient spatio-temporal joint modeling of dynamic blurry sequences.The framework integrates a CNN+Transformer hybrid encoding architecture,incorporating several key mo-dules:1)An improved multi-scale residual transformer block(MRTB),which leverages spatio-temporal self-attention mechanisms to enhance reconstruction accuracy and explicitly model event correlations between blurry frames;2)A high-quality feature transmitter(HQFT) module,employing a cross-scale feature distillation mechanism to bolster semantic consistency during the blurry-to-sharp domain conversion,thereby addressing the challenge of model generalization;3)A lightweight dynamic upsampling and rendering module(DYRM) that utilizes differentiable dynamic convolution to decouple resolution reconstruction from computational complexity,thus tackling computational inefficiency and enabling flexible resolution recovery.Experimental evaluations on the Adobe240 and RBI datasets demonstrate that FFiT exhibits exceptional performance in addressing the intricate spatio-temporal joint modeling of dynamic blurry sequences.Notably,FFiT reduces model parameters by 70% to accommodate computational resource constraints while achieving a 2.73 dB improvement in PSNR(Peak Signal-to-Noise Ratio) compared to baseline models.This research offers a valuable reference for resolving such compound degradation problems by effectively balancing image quality enhancement with computational efficiency.

Key words: Motion blur, Joint interpolation deblurring, CNN+Transfomer, Lightweight

CLC Number: 

  • TP751
[1]ZHONG Z,CAO M,JI X,et al.Blur interpolation transformer for real-world motion from blur[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:5713-5723.
[2]KIM T,LEE J,WANG L,et al.Event-guided deblurring of unknown exposure time videos[C]//European Conference on Computer Vision.Cham:Springer Nature Switzerland,2022:519-538.
[3]JIN M,HU Z,FAVARO P.Learning to extract flawless slow motion from blurry videos[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:8112-8121.
[4]SHEN W,BAO W,ZHAI G,et al.Video frame interpolation and enhancement via pyramid recurrent framework[J].IEEE Transactions on Image Processing,2020,30:277-292.
[5]OH J,KIM M.Demfi:deep joint deblurring and multi-frame interpolation with flow-guided attentive correlation and recursive boosting[C]//European Conference on Computer Vision.Cham:Springer Nature Switzerland,2022:198-215.
[6]ROZUMNYI D,OSWALD M R,FERRARI V,et al.Motion-from-blur:3d shape and motion estimation of motion-blurred objects in videos[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:15990-15999.
[7]GUPTA A,AICH A,ROY-CHOWDHURY A K.Alanet:Adaptive latent attention network for joint video deblurring and interpolation[C]//Proceedings of the 28th ACM International Conference on Multimedia.2020:256-264.
[8]杨雪松,何亮田.基于即插即用的盲二值图像去模糊算法[J].重庆工商大学学报(自然科学版),2025,42(4):36-43.
[9]SHANG W,REN D,YANG Y,et al.Joint Video Multi-Frame Interpolation and Deblurring under Unknown Exposure Time[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:13935-13944.
[10]CHO S J,JI S W,HONG J P,et al.Rethinking coarse-to-fine approach in single image deblurring[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:4641-4650.
[11]PARK D,KANG D U,KIM J,et al.Multi-temporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training[C]//European Conference on Computer Vision.Cham:Springer International Publishing,2020:327-343.
[12]REN D,ZHANG K,WANG Q,et al.Neural blind deconvolution using deep priors[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:3341-3350.
[13]ZHONG Z,GAO Y,ZHENG Y,et al.Efficient spatio-temporal recurrent neural network for video deblurring[C]//Computer Vision-ECCV 2020:16th European Conference.Springer International Publishing,2020:191-207.
[14]PAN J,BAI H,TANG J.Cascaded deep video deblurring using temporal sharpness prior[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:3043-3051.
[15]SHANG W,REN D,ZOU D,et al.Bringing events into video deblurring with non-consecutively blurry frames[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:4531-4540.
[16]WANG X,CHAN K C K,YU K,et al.Edvr:Video restoration with enhanced deformable convolutional networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.2019:1954-1963.
[17]ZHAO Q,ZHOU D M,YANG H,et al.Image deblurring based on residual attention and multi-feature fusion[J].Computer Science,2023,50(1):147-155.
[18]ZAMIR S W,ARORA A,KHAN S,et al.Multi-stage progressive image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:14821-14831.
[19]ZAMIR S W,ARORA A,KHAN S,et al.Restormer:Efficient transformer for high-resolution image restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:5728-5739.
[20]LIANG J,FAN Y,XIANG X,et al.Recurrent video restoration transformer with guided deformable attention[J].Advances in Neural Information Processing Systems,2022,35:378-393.
[21]CAO M,FAN Y,ZHANG Y,et al.VDTR:Video deblurring with transformer[J].IEEE Transactions on Circuits and Systems for Video Technology,2022,33(1):160-171.
[22]TOUVRON H,CORD M,DOUZE M,et al.Training data-efficient image transformers & distillation through attention[C]//International Conference on Machine Learning.PMLR,2021:10347-10357.
[23]GRAHAM B,EL-NOUBY A,TOUVRON H,et al.Levit:a vision transformer in convnet's clothing for faster inference[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:12259-12269.
[24]WANG W,XIE E,LI X,et al.Pyramid vision transformer:A versatile backbone for dense prediction without convolutions[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:568-578.
[25]YUAN L,CHEN Y,WANG T,et al.Tokens-to-token vit:Training vision transformers from scratch on imagenet[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:558-567.
[26]HATAMIZADEH A,YIN H,HEINRICH G,et al.Global context vision transformers[C]//International Conference on Machine Learning.PMLR,2023:12633-12646.
[27]ZHANG P,DAI X,YANG J,et al.Multi-scale vision longformer:A new vision transformer for high-resolution image encoding[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:2998-3008.
[28]YUAN L,HOU Q,JIANG Z,et al.Volo:Vision outlooker for visual recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,45(5):6575-6586.
[29]LIU Z,HU H,LIN Y,et al.Swin transformer v2:Scaling up capacity and resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:12009-12019.
[30]CHEN Z,XIE L,NIU J,et al.Visformer:The vision-friendly transformer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:589-598.
[31]CHEN C F R,FAN Q,PANDA R.Crossvit:Cross-attention multi-scale vision transformer for image classification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2021:357-366.
[32]LIU X F,JIANG M R,HUANG Y Q,et al.Using image stabilization and VLBE algorithm to extract foreground target from jitter video[J].Computer Science,2022,49(S2):404-411.
[33]BAO W,LAI W S,MA C,et al.Depth-aware video frame interpolation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:3703-3712.
[34]LEE H,KIM T,CHUNG T,et al.Adacof:Adaptive collaboration of flows for video frame interpolation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:5316-5325.
[35]XU X,SIYAO L,SUN W,et al.Quadratic video interpolation[C]//NeurIPS 2019.2020:1636-1645.
[36]CHI Z,MOHAMMADI NASIRI R,LIU Z,et al.All at once:Temporally adaptive multi-frame interpolation with advanced motion modeling[C]//Computer Vision-ECCV 2020:16th European Conference.2020:107-123.
[37]PUROHIT K,SHAH A,RAJAGOPALAN A N.Bringing alive blurred moments[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:6830-6839.
[38]ARGAW D M,KIM J,RAMEAU F,et al.Motion-blurred video interpolation and extrapolation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021:901-910.
[39]HATAMIZADEH A,HEINRICH G,YIN H,et al.Fastervit:Fast vision transformers with hierarchical attention[J].arXiv:2306.06189,2023.
[40]HAN Q,FAN Z,DAI Q,et al.On the connection between local attention and dynamic depth-wise convolution[J].arXiv:2106.04263,2021.
[41]SU S,DELBRACIO M,WANG J,et al.Deep video deblurring for hand-held cameras[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:1279-1288.
[1] ZHANG Shouyi, SHEN Qiang, GUO Yiran, WANG Hanyu. Rain and Fog Weather Object Detection Algorithm Based on Improved YOLOv8 Model [J]. Computer Science, 2026, 53(6A): 250300090-7.
[2] GU Qinghua, MA Xiang, LI Xuexian. Multi-scale Feature Screening-integrated Lightweight Algorithm for Blast Heap Ore Image Segmentation in Open-pit Mines [J]. Computer Science, 2026, 53(6A): 250500016-8.
[3] LI Siyu, QIAN Wenhua. HCKD:Lightweight Skin Lesion Classification Method Based on Dermoscopic Images [J]. Computer Science, 2026, 53(6A): 250600143-9.
[4] FU Shiqi, ZHU Jinxia, XU Qichen, DU Zeyu. Dynamic Sparsity and Heterogeneous Knowledge Distillation for Top-k Recommendation [J]. Computer Science, 2026, 53(6A): 250700121-9.
[5] WANG Pan, WANG Ji, ZHONG Zhengyi, BAO Weidong, ZHANG Yaohong. Lightweight Federated Continual Learning Method Based on Double Anti-forgetting Mechanism [J]. Computer Science, 2026, 53(4): 424-434.
[6] WANG Xinyu, GAO Donghuai, NING Yuwen, XU Hao, QI Haonan. Student Behavior Detection Method Based on Improved YOLO Algorithm [J]. Computer Science, 2026, 53(3): 246-256.
[7] ZHAI Jie, LI Yanhao, CHEN Lexuan, GUO Weibin. Dynamic Recommendation of Personalized Hands-on Learning Materials Based on LightweightEducational LLMs [J]. Computer Science, 2026, 53(2): 48-56.
[8] ZHUANG Jianjun, WAN Li. SCF U2-Net:Lightweight U2-Net Improved Method for Breast Ultrasound Lesion SegmentationCombined with Fuzzy Logic [J]. Computer Science, 2025, 52(7): 161-169.
[9] LI Jiangxu, CHEN Zemao, ZHANG Liqiang. Lightweight Authentication and Key Agreement Protocol for Cloud-assisted Smart Home Communication [J]. Computer Science, 2025, 52(7): 342-352.
[10] GAO Junyi, ZHANG Wei, LI Zelin. YOLO-BFEPS:Efficient Attention-enhanced Cross-scale YOLOv10 Fire Detection Model [J]. Computer Science, 2025, 52(6A): 240800134-9.
[11] LI Xiaolan, MA Yong. Study on Lightweight Flame Detection Algorithm with Progressive Adaptive Feature Fusion [J]. Computer Science, 2025, 52(4): 64-73.
[12] HU Huijuan, QIN Yifeng, XU Heand LI Peng. An Improved YOLOv8 Object Detection Algorithm for UAV Aerial Images [J]. Computer Science, 2025, 52(4): 202-211.
[13] QIU Dandan. Lightweight Deep Learning Network Algorithm Optimized Based on Pruning Algorithm [J]. Computer Science, 2025, 52(11A): 241000134-7.
[14] LIU Xingpeng, XUE Yiming, LIN Yuyang, LI Yan, PENG Wanli. Lightweight Image Super-resolution Reconstruction Based on Feature Similarity Analysis [J]. Computer Science, 2025, 52(11A): 250100057-8.
[15] DOU Zhuolun, YU Chunzhan, ZHANG Jialin, LI Yulong. Obstacle Recognition Method for Grassland Inspection Robot Based on Improved YOLOv8 [J]. Computer Science, 2025, 52(11A): 241100065-6.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!