计算机科学 ›› 2015, Vol. 42 ›› Issue (9): 45-49.doi: 10.11896/j.issn.1002-137X.2015.09.010
高燕飞,陈俊杰,强彦
GAO Yan-fei, CHEN Jun-jie and QIANG Yan
摘要： 目前,云计算环境具有动态、异构和海量多类型任务并发等特征,随着集群规模不断增大、用户QoS不断增多,现有调度算法越来越难以适应动态变化的环境及满足用户的需求。针对Hadoop平台下现有调度器不能根据作业运行状态和资源使用情况进行动态调整的问题,提出了Hadoop下基于作业分类的动态调度算法。该算法在使用朴素贝叶斯分类算法对队列中作业进行分类的过程中,根据各个作业的类型,预先设定类别权值,将队列中的作业分类,并引入效用函数,根据用户提交时的预期完成时间QoS和作业完成情况估算其作业完成时间,实现动态设置作业优先级。实验表明,使用提出的算法不仅能有效减少 作业的分类时间,而且能明显提高 动态性和用户QoS。
| [1] 朱晓敏,祝江汉,马满好.异构集群系统中具有 QoS 需求的实时任务容错调度[J].软件学报,2011,22(7):1440-1456 Zhu Xiao-min,Zhu Jiang-han,Ma Man-chao.Fault-Tolerant Scheduling for Real-time Tasks with QoS Requirements on Heterogeneous Clusters[J].Journal of Software,2011,2(7):1440-1456 [2] Yang Y,Xiang P,Mantor M,et al.CPU-Assisted GPGPU on Fused CPU-GPU Architecture[C]∥the 8th International Symposium on High Performance Computer Architecture(HPCA-18).2012:103-114 [3] Dutta D,Joshi R C.A genetic:algorithm approach to cost-based multi-QoS job scheduling in cloud computing environment[C]∥ICWET.2011:422-427 [4] 王凯,侯紫峰.动态调整虚拟机权重参数的调度方法[J].计算机研究与发展,2012,48(11):2094-2102 Wang Kai,Hou Zi-feng.An Adaptive Scheduling Method of Weight Parameter Adjustment on Virtual Machines [J].Journal of Computer Research and Development,2012,8(11):2094-2102 [5] 王守信,张莉,李鹤松.一种基于云模型的主观信任评价方法[J].软件学报,2010,21(6):1341-1352 Wang Shou-xin,Zhang Li,Li He-song.Evaluation Approach of Subjective Trust Based on Cloud Model[J].Journal of Software,2010,1(6):1341-1352 [6] Rasooli A,Down D.An Adaptive Scheduling Algorithm for Dynamic Heterogeneous Hadoop Systems [C]∥Proceedings of the 2011 Conference of the Center for Advanced Studies on Collaborative Research.IBM Corp.,2011:30-44 [7] Chen Q,Zhang D,Guo M,et al.Samr:A self-adaptive mapreduce scheduling algorithm in heterogeneous environment[C]∥2010 IEEE 10th International Conference on Computer and Information Technology(CIT).IEEE,2010:2736-2743 [8] 陈全,邓倩妮.异构环境下动态的Map-Reduce调度[J].计算机工程与科学,2009,31(S1):168-171,175 Chen Quan,Deng Qian-ni.Self-Adaptive Map-reduce Scheduling Under Heterogeneous Environment[J].Computer Engineering & Science,2009,1(S1):168-171,5 [9] Mao Hong,Hu Sheng-qiu,Zhang Zhen-zhong,et al.A Load-Driven Task Scheduler with Adaptive DSC for MapReduce[C]∥GREENCOM.2011:28-33 [10] Kaushik R T,Bhandarkar M.Greenhdfs:towards an energy-conserving,storage-efficient,hybrid hadoop compute cluster[C]∥Proceedings of the USENIX Annual Technical Conference.2010 [11] Lu P,Lee Y C,Wang C,et al.Workload Characteristic Oriented Scheduler for MapReduce[C]∥2012 IEEE 18th International Conference on Parallel and Distributed Systems(ICPADS).IEEE,2012:156-163 [12] 李强,郝沁汾,肖利民,等.云计算中虚拟机放置的动态管理与多目标优化[J].计算机学报,2011,34(12):2253-2264 Li Qiang,Hao Qin-fen,Xiao Li-min,et al.An Integrated Approach to Automatic Management of Virtualized Resources in Cloud Environments [J].Chinese Journal of Computers,2011,4(12):2253-2264 [13] Sharma B,Chudnovsky V,Hellerstein J L,et al.Modeling and synthesizing task placement constraints in Google compute clusters[C]∥SOCC.2011 [14] Deng Ke-feng,Song Jun-qiang,Ren Kai-jun,et al.Graph-Cut Based Coscheduling Strategy Towards Efficient Execution of Scientific Workflows in Collaborative Cloud Environments[C]∥GRID.2011:34-41 [15] Dhok J,Varma V.Using pattern classification for task assignment in mapreduce[C]∥International Institute of Information Technology.Hyderabad,India,2005 [16] Polo J,Castillo C,Carrera D,et al.Resource-aware Adaptive Scheduling for MapReduce Clusters[C]∥ACM/IFIP/USENIX 12th International Middleware Conference(Middleware 2011).2011 |
| No related articles found! |
|
||
首页