摘要: 数据密集型应用越来越多,如何将大数据在数据中心实现有效放置变得日益重要。研究了大数据的放置模型。影响大数据放置的因素主要有:能耗、异构节点的服务能力及具有关联计算的数据集。基于这3个因素设计了一种节能、充分考虑异构节点服务能力及提升MapReduce处理Join连接的效率的大数据放置模型。该模型将有效实现大数据的有效放置管理,同时也为未来软件定制数据中心奠定了基础。
[1] 覃雄派,王会举,李芙蓉,等.数据管理技术的新格局[J].软件学报,2013,4(2):175-197 [2] Kaushik R T,Bhandarkar M.GreenHDFS:towards an energy-conserving,storage-efficient,hybrid Hadoop compute cluster[C]∥HotPower’10 Proceedings of the 2010International Conference on Power Aware Computing and Systems.2010:1-5 [3] Arasanal R M,Rumani D U.Improving MapReduce Perform-ance through Complexity and Performance Based Data Placement in Heterogeneous Hadoop Clusters[C]∥Distributed Computing and Internet Technology,Lecture Notes in Computer Science.2013,7753:115-125 [4] Xie J,Yin S,Ruan X,et al.Improving MapReduce performance through data placement in heterogeneous Hadoop clusters[C]∥Proceedings of IPDPS Workshops.2010:1-9 [5] 王俊伟.大规模多媒体存储系统中数据放置与调度策略的研究[D].长沙:国防科学技术大学,2005 [6] 林伟伟.一种改进的Hadoop数据放置策略[J].华南理工大学学报:自然科学版,2012(01):152-158 [7] Mohamed Y,Tian Yuan-yuan,zcan F,et al.CoHadoop:flexi-ble data placement and its exploitation in Hadoop[J].Procee-dings of the VLDB Endowment VLDB2011,2011,4(9):575-585 [8] Dittrich J,Quian’e-Ruiz J A,Jindal A,et al.Hadoop++:Ma-king a yellow elephant run like a cheetah (without it even noticing)[J].PVLDB2010,2010,3(1/2):518-529 [9] Zhao Yan-rong,Wang Wei-ping,Meng Dan,et al.A data locality optimization algorithm for large-scale data processing in Hadoop[C]∥ISCC 2012.2010:655-661 [10] 赵彦荣,王伟平,孟丹,等.基于Hadoop 的高效连接查询处理算法CHMJ[J].软件学报,2012,3(8):2032-2041 |
No related articles found! |
|