计算机科学 ›› 2025, Vol. 52 ›› Issue (6): 316-323.doi: 10.11896/jsjkx.240300019

• 人工智能 • 上一篇    下一篇

一种基于产品复用模型的高效遥感共性产品生产算法

左宪禹, 周小虎, 周黎明, 谢毅, 刘成   

  1. 河南省大数据分析与处理重点实验室(河南大学) 河南 开封 475000
    河南大学计算机与信息工程学院 河南 开封 475000
  • 收稿日期:2024-03-03 修回日期:2024-08-17 出版日期:2025-06-15 发布日期:2025-06-11
  • 通讯作者: 刘成(liucheng@henu.edu.cn)
  • 作者简介:(xianyu_zuo@henu.edu.cn)
  • 基金资助:
    国家重点研发计划国际合作专项(2019YFE0126600);河南省科技重大专项(201400210300);河南省高校科技创新团队(24IRTSTHN021);河南省科技厅科技攻关项目(232102210009);河南省科技攻关(242102240021);2024河南省研究生联合培养基地项目(YJS2024JD30)

Efficient Remote Sensing Common Product Production Algorithm Based on Product Reuse Model

ZUO Xianyu, ZHOU Xiaohu, ZHOU Liming, XIE Yi, LIU Cheng   

  1. Henan Key Laboratory of Big Data Analysis and Processing,Henan University,Kaifeng,Henan 475000,China
    School of Computer and Information Engineering,Henan University,Kaifeng,Henan 475000,China
  • Received:2024-03-03 Revised:2024-08-17 Online:2025-06-15 Published:2025-06-11
  • About author:ZUO Xianyu,born in 1979,Ph.D,professor,Ph.D supervisor,is a member of CCF(No.G4801M).His main research interests include parallel computing and remote sensing big data processing.
    LIU Cheng,born in 1989,Ph.D,lectu-rer,is a member of CCF(No.l7262M).His main research interests include pattern recognition and image segmentation.
  • Supported by:
    National Key Research and Development Program for International Cooperation(2019YFE0126600),Henan Province Major Science and Technology Project(201400210300),Henan University Science and Technology Innovation Team(24IRTSTHN021),Henan Provincial Department of Science and Technology Research Project(232102210009),Henan Province Science and Technology Research(242102240021) and Postgraduate Education Reform and Quality Improvement Project of Henan Province(YJS2024JD30).

摘要: 随着各行业对遥感共性产品需求的不断增加,高性能遥感产品生产系统的应用范围不断扩大。优秀的任务调度算法作为该系统的关键部件,能显著提高生产效率。然而,在遥感共性产品的生产过程中面临特有的挑战,如果大量的工作流在短时间内被提交生产,这些工作流在处理中存在重复计算和数据处理的问题,且生成共性产品所需的数据量往往较大,流程处理时间长,很容易导致资源浪费和生产效率下降。为了解决这一问题,提出一种基于产品复用模型的任务划分策略。该策略着眼于优化工作流处理,首先将用户提交的工作流按照任务重复度打包成流程包,把带有重复任务的流程分配到同一个计算节点,旨在减少节点间的数据传输时间;然后引入一种产品复用模型,允许不同的处理流程复用已获得的产品结果,减少重复性计算和数据处理,从而提高生产效率,满足共性产品生产的高效化需求。为了验证所提算法的有效性,将所提算法和传统算法FCFS,SJF分别在CloudSim仿真模拟器中进行模拟实验。结果表明,所提调度算法任务的总完成时间和任务的平均响应时间均显著低于对比算法,展现出了更为优秀的性能。

关键词: 高性能计算, 共性遥感产品, 产品复用, 任务划分策略, CloudSim

Abstract: With the increasing demand for remote sensing common products in various industries,the application of high-perfor-mance remote sensing product production system is increasing.As a key component of the system,excellent task scheduling algorithm can significantly improve its production efficiency.However,there are unique challenges in the production process of remote sensing generic products.If a large number of workflows are submitted for production in a short time,there are problems of dou-ble calculation and data processing in the processing of these workflows,and the amount of data required to generate generic pro-ducts is often large,and the process processing time is long,which easily leads to resource waste and production efficiency decline.In order to solve this problem,this paper proposes a task division strategy based on product reuse model,which focuses on optimizing workflow processing.Firstly,workflow submitted by users is packaged into a process package according to task repetition,and processes with repetitive tasks are assigned to the same computing node to reduce the data transmission time between nodes.Then,a product reuse model is introduced to allow different processing processes to reuse the obtained product results,reduce repetitive calculation and data processing,so as to improve production efficiency and meet the high efficiency needs of common product production.In order to verify the effectiveness of the proposed algorithm,the proposed algorithm and other traditional algorithms FCFS and SJF are simulated in the CloudSim simulation simulator respectively.The results show that the proposed scheduling algorithm has significantly lower total task completion time and average task response time than the other two algorithms,showing better performance.

Key words: High performance computing, Generic Remote sensing products, Product reuse, Task divison strategy, CloudSim

中图分类号: 

  • TP302
[1]CHI M,PLAZA A,BENEDIKTSSON J A,et al.Big data for remote sensing:Challenges and opportunities[C]//Proceedings of the IEEE.2016:2207-2219.
[2]TONG X D.China's high resolution earth observation systemconstruction of major projects progress [J].Journal of Remote Sensing,2016,29(6):927-933.
[3]ZHOU B,LI J G,WU G F,et al.A Visual Dataflow Model for Production of Remote Sensing Products[J].Journal of Henan University(Natural Science Edition),2013,43(1):74-78.
[4]FAN Y,LI B,FAVORITE D,et al.Dras:Deep reinforcementlearning for cluster scheduling in high performance computing[J].IEEE Transactions on Parallel and Distributed Systems,2022,33(12):4903-4917.
[5]WANG X,LI N,GONG G,et al.Load-balancing scheduling of simulation tasks based on a static-dynamic hybrid algorithm[J].Journal of Simulation,2022,16(2):182-193.
[6]HOFRI M.Disk scheduling:FCFS vs.SSTF revisited[J].Communications of the ACM,1980,23(11):645-653.
[7]ALWORAFI M A,DHARI A,AL-HASHMI A A,et al.An improved SJF scheduling algorithm in cloud computing environment[C]//2016 International Conference on Electrical,Electronics,Communication,Computer and Optimization Techniques(ICEECCOT).IEEE,2016:208-212.
[8]QIU X C,ZANG L,YANG D,et al.Multilevel feedback queue Scheduling Algorithm based on Process execution time [J].Science Technology and Engineering,2015,15(1):78-83.
[9]ANDERSSON B,BARUAH S,JONSSON J.Static-priorityscheduling on multiprocessors[C]//Proceedings 22nd IEEE Real-Time Systems Symposium(RTSS 2001).IEEE,2001:193-202.
[10]MIREGURI K,GU Y J.Simulation of LoadBalancing Time Slice Rotation Algorithms in Embedded Operating System[J].Computer Simulation,2019,36(11):247-250.
[11]SHI Y L,SHEN W M,XIONG W C et al.Research on job schedule and managementsystem for remote sensing data processing with cluster[J].Computer Engineering and Applications,2012,48(25):77-82.
[12]WU H H.Research and application of task scheduling model in massive remote sensing image common product generation [D].Zhengzhou:Henan University,2023.
[13]ZHENG F B,ZHANG Z,YU T,et al.Architecture of Remote Sensing Producing Line for Supporting[J].Computer Science,2012,39(S3):181-184,190.
[14]RAJAVEL R,MALA T.Achieving service level agreement in cloud environment using job prioritization in hierarchical sche-duling[C]//Proceedings of the International Conference on Information Systems Design and Intelligent Applications 2012(INDIA 2012) held in Visakhapatnam,India,January 2012.Springer Berlin Heidelberg,2012:547-554.
[15]QIU Y,JIANG C,WANG Y,et al.Energy aware virtual machine scheduling in data centers[J].Energies,2019,12(4):646.
[16]LIU Q H,WEN J G,ZHOU X,et al.High resolution remote sensing common product generation and authenticity test technology system[J].Journal of Remote Sensing,2023,27(3):544-562.
[17]ZHAO J P,CHEN D H,LI H,et al.A Dynamic IntegrationFramework of GIS System for Remote Sensing Algorithms[J].Computer Measurement and Control,2018,26(7):186-189,194.
[18]TSUR D.Faster deterministic algorithms for Co-path Packing and Co-path/cycle Packing[J].Journal of Combinatorial Optimization,2022,44(5):3701-3710.
[19]HEILIG L,RAJKUMAR B,STEFAN V.Location-aware bro-kering for consumers in multi-cloud computing environments[J].Journal of Network and Computer Applications,2017,95(10):79-93.
[20]LIU P.Cloud Computing.2nd Edition [M].Publishing House of Electronics Industry,2011.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!