计算机科学

计算机科学 ›› 2022, Vol. 49 ›› Issue (5): 235-243.doi: 10.11896/jsjkx.210300303

• 计算机网络 • 上一篇    下一篇

混合云工作流调度综述

柳鹏1, 刘波1, 周娜琴2, 彭心怡3, 林伟伟4   

  1. 1 华南师范大学计算机学院 广州510631
    2 广州大学网络空间先进技术研究院 广州510006
    3 华南师范大学数学科学学院 广州510631
    4 华南理工大学计算机科学与工程学院 广州510640
  • 收稿日期:2021-03-31 修回日期:2021-06-06 出版日期:2022-05-15 发布日期:2022-05-06
  • 通讯作者: 周娜琴(439657699@qq.com)
  • 作者简介:(858299238@qq.com)
  • 基金资助:
    国家自然科学基金(62002078,62072187,61872084);广东省基础与应用基础研究重大项目(2019B030302002);广州市科技计划项目(202007040002,201902010040)

Survey of Hybrid Cloud Workflow Scheduling

LIU Peng1, LIU Bo1, ZHOU Na-qin2, PENG Xin-yi3, LIN Wei-wei4   

  1. 1 School of Computer Science,South China Normal University,Guangzhou 510631,China
    2 Institute of Advanced Technology in Cyberspace,Guangzhou University,Guangzhou 510006,China
    3 School of Mathematical Sciences,South China Normal University,Guangzhou 510631,China
    4 School of Computer Science and Engineering,South China University of Technology,Guangzhou 510640,China
  • Received:2021-03-31 Revised:2021-06-06 Online:2022-05-15 Published:2022-05-06
  • About author:LIU Peng,born in 1997,postgraduate.His main research interests include cloud computing and resource scheduling.
    ZHOU Na-qin,born in 1982,Ph.D,lecturer,is a member of China Computer Federation.Her main research interests include distributed computing and resource management scheduling.
  • Supported by:
    National Natural Science Foundation of China(62002078,62072187,61872084),Guangdong Major Project of Basic and Applied Basic Research(2019B030302002) and Guangzhou Science and Technology Plan Project(202007040002,201902010040).

PDF (PC)

552

摘要： 在数据爆发的大背景下,传统的云计算存在本地云资源不足和扩展成本高的窘境,而最新兴起的混合云结合了资源丰富的公有云与数据敏感的私有云,成为当下研究和应用的热点方向。而工作流作为一种有吸引力的范式,其数据规模和计算规模一直都在增长,因此工作流调度是混合云研究方向中的关键问题。为此,文中首先对混合云环境下的工作流调度技术做了深入的调查和分析,然后将混合云环境下的工作流调度进行分类与比较,重点阐述面向截止期限、成本、节约能耗和多目标约束的混合云工作流调度。在此基础上分析和总结了混合云环境下工作流调度的未来研究方向,如Serverless平台应用工作流调度、利用边缘服务器网络协同的工作流调度、融合Argo的云原生工作流调度和融合雾计算的工作流调度。

关键词: 工作流, 混合云, 资源调度

Abstract: In the context of data explosion,traditional cloud computing is faced with the dilemma of insufficient local cloud resources and high expansion cost.However,the newly emerging hybrid cloud combining resource-rich public cloud and data-sensitive private cloud has become a research hotspot and application direction at present.As an attractive paradigm,workflow has been increasing in data scale and computing scale.Therefore,workflow scheduling is a key issue in the direction of hybrid cloud research.For this reason,this paper first makes an in-depth investigation and analysis of workflow scheduling technology in hybrid cloud environment,and then classifies and compares workflow scheduling in hybrid cloud environment:for deadline,for cost,for energy-efficient and for multi-objective constraints.On this basis,the future research directions of workflow scheduling in hybrid cloud environment are analyzed and summarized:workflow scheduling based on Serverless platform,workflow scheduling based on edge server network collaboration,cloud native workflow scheduling based on Argo integration,and workflow scheduling based on fog computing fusion.

Key words: Hybrid cloud, Scheduling of resources, Workflow

中图分类号: 

  • TP393
[1]PARTOVI B,BAGHERI A,KAZARJI M H,et al.Virtual Machine Placement for Edge and Cloud Computing[C]//European Conference on Service-Oriented and Cloud Computing.Cham:Springer,2020:53-64.
[2]MEI J,LI K,OUYANG A,et al.A profit maximization schemewith guaranteed quality of service in cloud computing[J].IEEE Transactions on Computers,2015,64(11):3064-3078.
[3]What is hybrid cloud？[EB/OL].(2019-10-16)[2021-02-01].https://www.ibm.com/cloud/learn/hybrid-cloud.
[4]VAISHNNAVE M,DEVI K S,SRINIVASANP.A survey oncloud computing and hybrid cloud[J].Interence J. Appl. Eng. Res.,2019,14(2):429-434.
[5]Global Cloud Computing Market Size & Share Will Reach USD 1025.9 Billion by 2026:Facts & Factors[EB/OL].(2021-01-22)[2021-02-28].https://www.globenewswire.com/news-release/2021/01/22/2162789/0/en/Global-Cloud-Computing-Market-Size-Share-Will-Reach-USD-1025-9-Billion-by-2026-Facts-Fac-tors.html.
[6]ZHOU J,WANG T,CONG P,et al.Cost and makespan-aware workflow scheduling in hybrid clouds[J].Journal of Systems Architecture,2019,100:101631.
[7]WANG B,WANG C,SONG Y,et al.A survey and taxonomy on workload scheduling and resource provisioning in hybrid clouds[J].Cluster Computing,2020,23(4):2809-2834.
[8]KORNIICHUK M,KARPOV K,FEDOTOVA I,et al.Impactof Xen and Virtual Box virtualization environments on timing precision under stressful conditions[C]//MATEC Web of Conferences.EDP Sciences,2018:02006.
[9]ALGARNI S A,IKBAL M R,ALROOBAEA R,et al.Perfor-mance evaluation of Xen,KVM,and proxmox hypervisors[J].International Journal of Open Source Software and Processes (IJOSSP),2018,9(2):39-54.
[10]Docker:build,manage and secure your apps anywhere.[EB/OL].(2020-11-21)[2021-02-26].http://www.docker.com/.
[11]Alibaba Cloud:an integrated suite of cloud products,services and solutions.[EB/OL].(2020-10-22)[2021-02-26].https://www.alibabacloud.com/.
[12]Amazon Elastic Compute Cloud (Amazon EC2)[EB/OL].(2020-11-21)[2021-02-26].http://aws amazon.com/ec2/.
[13]BITTENCOUR T, FERNANDO L,MADEIR A,et al.HCOC:a cost optimization algorithm for workflow scheduling in hybrid clouds[J].Journal of Internet Services and Applications,2011,2.3:207-227.
[14]YUAN H,BI J,ZHOU M.Temporal task scheduling of multiple delay-constrained applications in green hybrid cloud[J].IEEE Transactions on Services Computing,2021,14(5):1558-1570.
[15]VENKATESAKUMAR V,YASOTHA R,SUBASHINI A.Abrief survey on hybrid cloud storage and its applications[J].World Scientific News,2016,46:219-232.
[16]LIN B,GUO W Z,LIN X Y.Online optimization scheduling for scientific workflows with deadline constraint on hybrid clouds[J].Concurrency and Computation:Practice and Experience,2016,28.11:3079-3095.
[17]CASAS I,TAHERI J,RANJAN R,et al.PSO-DS:a scheduling engine for scientific workflow managers[J].The Journal of Super-computing,2017,73(9):3924-3947.
[18]PASDAR A,LEE Y C,ALMI’ANI K.Hybrid scheduling for scientific workflows on hybrid clouds[J].Computer Networks,2020,181:107438.
[19]BALAGONI Y,RAO R R.A cost-effective SLA-aware scheduling for hybrid cloud environment[C]//2016 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC).IEEE,2016:1-7.
[20]XU L,DUAN J W,HUANG M.Research on hybrid cloud particle swarm optimization for multi-objective flexible job shop scheduling problem[C]//2017 6th International Conference on Computer Science and Network Technology(ICCSNT).IEEE,2017:274-278.
[21]DAS A,LEAF A,VARELA C A,et al.Skedulix:Hybrid Cloud Scheduling for Cost-Efficient Execution of Serverless Applications[C]//2020 IEEE 13th International Conference on Cloud Computing(CLOUD).IEEE,2020:609-618.
[22]ZUO L Y,SHU L,DONG S B,et al.A multi-objective hybrid cloud resource scheduling method based on deadline and cost constraints[J].IEEE Access,2016,5:22067-22080.
[23]FAN Y,LIANG Q,CHEN Y,et al.Executing time and cost-aware task scheduling in hybrid cloud using a modified DE algorithm[J].International Journal of Computational Science and Engineering,2019,18(3):217-226.
[24]KRISHNAN P,ARAVIDHAR D J.Self-adaptive PSO memetic algorithm for multi objective workflow scheduling in hybrid cloud[J].Interence Arab J.Inf.Technol.,2019,16(5):928-935.
[25]HAMED A A,AHMED R A.Availability evaluation of differentplanning and scheduling algorithms in hybrid cloud system[J].Iraqi Journal of Information & Communications Technology,2020,3(4):47-59.
[26]MS S,PM J P,ALAPPATT V.Profit maximization based task scheduling in hybrid clouds using whale optimization technique[J].Information Security Journal:A Global Perspective,2020,29(4):155-168.
[27]SHARIF S,TAHERI J,ZOMAYA A Y,et al Online multiple workflow scheduling under privacy and deadline in hybrid cloud environment[C]//2014 IEEE 6th International Conference on Cloud Computing Technology and Science.IEEE,2014:455-462.
[28]HU W,LI X,LI X.Hybrid Cloud Workflow Scheduling Method With Privacy Data[J].IEEE Access,2020,8:211540-211552.
[29]LI C L,TANG J H,LUO Y L.Hybrid cloud adaptive scheduling strategy for heterogeneous workloads[J].Journal of Grid Computing,2019,17(3):419-446.
[30]PANDA S K,JANA P K.Uncertainty-based QoS min-min algorithm for heterogeneous multi-cloud environment[J].Arabian Journal for Science and Engineering,2016,41(8):3003-3025.
[31]LENINFRED A,DHANYA D,KAVITHA S,et al.Hybrid algorithm for resource provisioning with low cost and time using improved cost-based algorithm in hybrid cloud computing[J].Journal of Intelligent & Fuzzy Systems,2019,37(3):3981-3990.
[32]CHANG Y S,FAN C T,SHEU R K,et al.An agent-based workflow scheduling mechanism with deadline constraint on hybrid cloud environment[J].International Journal of Communication Systems,2018,31(1):e3401.
[33]IRANMANESH A,NAJI H R.DCHG-TS:a deadline-constrainedand cost-effective hybrid genetic algorithm for scientific workflow scheduling in cloud computing[J].Cluster Computing,2021,24(2):667-681.
[34]LIU W,DU W,CHEN J,et al.Adaptive energy-efficient scheduling algorithm for parallel tasks on homogeneous clusters[J].Journal of Network and Computer Applications,2014,41:101-113.
[35]RAJU R,AMUDHAVEL J,KANNAN N,et al.A bio inspired Energy-Aware Multi objective Chiropteran Algorithm(EAMOCA) for hybrid cloud computing environment[C]//2014 International Conference on Green Computing Communication and Electrical Engineering(ICGCCEE).IEEE,2014:1-5.
[36]BITTENCOURT L,MADEIR A,EDMUNDO R M.A perfor-mance-oriented adaptive scheduler for dependent tasks on grids[J].Concurrency and Computation:Practice and Experience,2008,20.9:1029-1049.
[37]LIU Z,XIANG T,LIN B,et al.A data placement strategy for scientific workflow in hybrid cloud[C]//2018 IEEE 11th International Conference on Cloud Computing(CLOUD).IEEE,2018:556-563.
[38]BALDINI I,CASTRO P,CHANG K,et al.Serverless compu-ting:Current trends and open problems[M]//Research Advances in Cloud Computing.Singapore:Springer,2017:1-20.
[39]Minio:High Performance,Kubernetes-Friendly Object Storage[EB/OL].(2020-11-21)[2021-02-15].https://min.io.
[40]How Much Energy Do Data Center Really User?[EB/OL].(2020-03-17)[2021-01-26].https://energyinnovation.org/2020/03/17/how-much-energy-do-data-centers-really-use/.
[41]HADOOP,Apache.Apache Hadoop[EB/OL].(2020-07-14)[2021-2-27].https://hadoop.apache.org/.
[42]RICHARD M K.Effective heuristics for NP-hard problems[EB/OL].(2011-10-17)[2021-01-6].https://www.youtube.com/watch?v=0p5NilbKETI.
[43]KINTSAKIS A M,PSOMOPOULOS F E,SYMEONIDIS A L,et al.Hermes:Seamless delivery of containerized bioinformatics workflows in hybrid cloud (HTC) environments[J].SoftwareX,2017,6:217-224.
[44]ZHU J,LI X,RUIZ R,et al.Scheduling stochastic multi-stage jobs to elastic hybrid cloud resources[J].IEEE Transactions on Parallel and Distributed Systems,2018,29(6):1401-1415.
[45]LI C L,LI LY.Optimal scheduling across public and private clouds in complex hybrid cloud environment[J].Information Systems Frontiers,2017,19(1):1-12.
[46]LI J,DING D,LI T.Multi-objective hybrid cloud task scheduling using twice clustering[J].Journal of Zhejiang University (Engineering Science),2017,51(6):1233-1241.
[47]ZHANG M,KRINTZ C,WOLSKI R.STOIC:Serverless Teleoperable Hybrid Cloud for Machine Learning Applications on Edge Device[C]//2020 IEEE International Conference on Pervasive Computing and Communications Workshops(PerCom Workshops).IEEE,2020:1-6.
[48]HUANG H,LING Q,SHI W,et al.Collaborative resource allocation over a hybrid cloud center and edge server network[J].Journal of Computational Mathematics,2017,35(4):423-438.
[49]XU X,CAO H,GENG Q,et al.Dynamic resource provisioning for workflow scheduling under uncertainty in edge computing environment[J/OL].Concurrency and Computation:Practice and Experience,2020:e5674.https://onlinelibrary.wiley.com/doi/full/10.1002/cpe.5674?casa_token=VAyobWre1y8AAAAA%3Annavap3-V3FAZak3Qp_jDQz0gQLo_md0uu5TD_9Y9B7nC6HpBgFCrQnqOpJsPwY9tpEq9fjBSiByYoK5.
[50]LU X,LIAO Y,LIO P,et al.Privacy-preserving asynchronousfederated learning mechanism for edge network computing[J].IEEE Access,2020,8:48970-48981.
[51]Argo Project Homepage[EB/OL].(2020-10-14)[2021-02-28].https://argoproj.github.io/projects/argo/.
[52]Kubernetes Project Homepage.[EB/OL].(2020-10-14)[2021-02-28].https://kubernetes.io/.
[53]Why the need for container-native workflow?[EB/OL].(2017-09-16)[2021-02-28].https://applatix.com/benefits-container-native-workflows/.
[54]ADHIKARI M,AMGOTH T,SRIRAMAS N.A survey onscheduling strategies for workflows in cloud environment and emerging trends[J].ACM Computing Surveys (CSUR),2019,52(4):1-36.
[55]LIU X,FAN L,XU J,et al.FogWorkflowSim:an automated simulation toolkit for workflow performance evaluation in fog computing[C]//2019 34th IEEE/ACM International Confe-rence on Automated Software Engineering(ASE).IEEE,2019:1114-1117.
[1] 严磊, 张功萱, 王添, 寇小勇, 王国洪.
混合云下具有交付期约束的众包任务调度算法
Scheduling Algorithm for Bag-of-Tasks with Due Date Constraints on Hybrid Clouds
计算机科学, 2022, 49(5): 244-249. https://doi.org/10.11896/jsjkx.210300120
[2] 林潮伟, 林兵, 陈星.
边缘环境下基于模糊理论的科学工作流调度研究
Study on Scientific Workflow Scheduling Based on Fuzzy Theory Under Edge Environment
计算机科学, 2022, 49(2): 312-320. https://doi.org/10.11896/jsjkx.201000102
[3] 窦帅, 李子扬, 朱家佳, 李晓辉, 李雪松, 米琳, 杨光, 李传荣.
基于jBPM的科学试验管理系统的设计与实现
Design and Implementation of Scientific Experiment Management System Based on jBPM
计算机科学, 2021, 48(6A): 658-663. https://doi.org/10.11896/jsjkx.200600158
[4] 宁玉辉, 姚喜.
一种应急指挥系统的设计与实现
Design and Implementation of Emergency Command System
计算机科学, 2021, 48(6A): 613-618. https://doi.org/10.11896/jsjkx.201000136
[5] 宋海宁, 焦健, 刘永.
高速公路中的移动边缘计算研究
Research on Mobile Edge Computing in Expressway
计算机科学, 2021, 48(6A): 383-386. https://doi.org/10.11896/jsjkx.200900212
[6] 季琰, 戴华, 姜莹莹, 杨庚, 易训.
面向混合云的可并行多关键词Top-k密文检索技术
Parallel Multi-keyword Top-k Search Scheme over Encrypted Data in Hybrid Clouds
计算机科学, 2021, 48(5): 320-327. https://doi.org/10.11896/jsjkx.200300160
[7] 马泽华, 刘波, 林伟伟, 李加伟.
无服务器平台资源调度综述
Survey of Resource Scheduling for Serverless Platforms
计算机科学, 2021, 48(4): 261-267. https://doi.org/10.11896/jsjkx.200800023
[8] 刘漳辉, 赵旭, 林兵, 陈星.
混合云环境下基于模糊理论的科学工作流数据布局策略
Data Placement Strategy of Scientific Workflow Based on Fuzzy Theory in Hybrid Cloud
计算机科学, 2021, 48(11): 199-207. https://doi.org/10.11896/jsjkx.200900009
[9] 马堉银, 郑万波, 马勇, 刘航, 夏云霓, 郭坤银, 陈鹏, 刘诚武.
一种基于深度强化学习与概率性能感知的边缘计算环境多工作流卸载方法
Multi-workflow Offloading Method Based on Deep Reinforcement Learning and ProbabilisticPerformance-awarein Edge Computing Environment
计算机科学, 2021, 48(1): 40-48. https://doi.org/10.11896/jsjkx.200900195
[10] 张龙信, 周立前, 文鸿, 肖满生, 邓晓军.
基于异构云计算的成本约束下的工作流能量高效调度算法
Energy Efficient Scheduling Algorithm of Workflows with Cost Constraint in Heterogeneous Cloud Computing Systems
计算机科学, 2020, 47(8): 112-118. https://doi.org/10.11896/jsjkx.200300038
[11] 孙敏, 陈中雄, 叶侨楠.
云环境下基于HEDSM的工作流调度策略
Workflow Scheduling Strategy Based on HEDSM Under Cloud Environment
计算机科学, 2020, 47(6): 252-259. https://doi.org/10.11896/jsjkx.190400047
[12] 简琤峰, 平靖, 张美玉.
面向边缘计算的Storm边缘节点调度优化方法
Edge Computing-oriented Storm Edge Node Scheduling Optimization Method
计算机科学, 2020, 47(5): 277-283. https://doi.org/10.11896/jsjkx.190600048
[13] 徐飞, 王少昌, 杨卫霞.
基于博弈论的云资源调度算法
Cloud Resource Scheduling Algorithm Based on Game Theory
计算机科学, 2019, 46(6A): 295-299.
[14] 汪晨欣, 杨家海, 庄奕, 罗念龙.
未来网络试验设施的节点资源调度算法
Node Resource Scheduling for Future Network Experimentation Facility
计算机科学, 2019, 46(12): 95-100. https://doi.org/10.11896/jsjkx.190400106
[15] 黄引豪, 马郓, 林兵, 於志勇, 陈星.
混合云环境下面向代价优化的工作流数据布局方法
Cost-driven Workflow Data Placement Method in Hybrid Cloud Environment
计算机科学, 2019, 46(11A): 354-358.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发