计算机科学 ›› 2011, Vol. 38 ›› Issue (11): 67-70.

• 计算机网络与信息安全 • 上一篇    下一篇

基于LATE的Hadoop数据局部性改进调度算法

李丽英,唐卓,李仁发   

  1. (湖南大学计算与通信学院 长沙410082)
  • 出版日期:2018-12-01 发布日期:2018-12-01
  • 基金资助:
    本文受国家工信部核高基项目(2009ZX01038-001)资助。

New Improvement of the Hadoop Relevant Data Locality Scheduling Algorithm Based on LATE

LI Li-ying,TANG Zhuo,LI Ren-fa   

  • Online:2018-12-01 Published:2018-12-01

摘要: 调度问题是目前云计算研究中的热点问题,其目的是如何协同云计算资源,使其得到充分合理的利用。数据局部性是特定云平台Hadoop的主要特性之一。针对该特性,在Hadoop原有调度算法LATE的基础上提出了一种基于数据局部性的改进算法,以解决数据局部性带来的慢任务备份执行时读取数据要占用大部分时间而影响其处理速率的问题。最后,对该算法进行了实验及性能分析,并验证了算法在提高任务的响应时间和整个系统吞吐率方面有很大改进。

关键词: Hadoop, MapRcducc, LATE,数据局部性

Abstract: In the present,scheduling problem is a hot cloud computation research issues, and the purpose is to coordinate the cloud computation resources to be fully rational use. Data locality is one of the main properties in the particular cloud platform for Hadoop. The paper discussed the property, proposed a new improvement of the Hadoop relevant data locality scheduling algorithm based on LAhE. hhe algorithm mainly sows the bakcup of slow task performance problem which arises during the implementation of data read,taking most of the time and envently influencing its processing speed. Finally, carried on experiment to the algorithm and analyzed the funcation, verified the algorithm to improve the response time and the whole system throughput.

Key words: Hadoop, MapReduce, LATE, Date locality

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!