计算机科学 ›› 2018, Vol. 45 ›› Issue (6A): 411-414.

• 大数据与数据挖掘 • 上一篇    下一篇

MapReduce框架下的Skyline云资源选择算法

齐玉东,何诚,司维超   

  1. 海军航空工程学院 山东 烟台264001
  • 出版日期:2018-06-20 发布日期:2018-08-03
  • 作者简介:齐玉东(1973-),男,博士,副教授,主要研究领域为指挥信息系统;何 诚(1993-),男,硕士生,主要研究领域为网络作战辅助决策;司维超(1984-),男,博士,讲师,主要研究领域为装备信息化,E-mail:674690154@qq.com。

Cloud Resource Selection Algorithm by Skyline under MapReduce Frame

QI Yu-dong,HE Cheng,SI Wei-chao   

  1. Naval Aeronautical Engineering Institute,Yantai,Shandong 264001,China
  • Online:2018-06-20 Published:2018-08-03

摘要: 文中研究了MapReduce框架下的云资源选择算法。该算法利用概率筛选的方法计算资源节点属于Skyline结果集的概率,并通过阈值过滤来降低MapReduce中的心跳频率,以达到优化网络流量的目的。

关键词: MapReduce, Skyline, 网络优化, 资源选择

Abstract: This paper researched a cloud resource selected algorithm under the MapReduce frame,which uses a method of possibility filtrate to figure the possibility of a resource nod belonging to the Skyline results set.It filtrates information by value seted in advance,to reduce the frequency of heartbeat in the MapReduce frame eventually,to optimize the flow of network.

Key words: MapReduce, Network majorization, Resource selection, Skyline

中图分类号: 

  • TP391
[1]朱琳,关佶红,周水庚.Skyline计算研究综述[J].计算机工程与应用,2008,44(6):160-165.
[2]BORZSONYI S,KOSSMANN D,STOCKER K.The Skyline operator[C]∥IEEE International Conference on Data Engineering(ICDE).2001:421-430.
[3]魏小娟,杨婧,李翠平,等.Skyline查询处理[J].软件学报,2008,19(6):1386-1400.
[4]PAPADIAS D,TAO Y,FU G,SEEGER B.ProgressiveSkyline computation in database system[J].ACM Transcations on Database System,2005,30(1):41-82.
[5]姜罕盛.云环境下数据安置策略与Skyline查询研究[D].天津:天津理工大学,2014.
[6]王媛,王意洁,邓瑞鹏,等.云计算环境下的容错并行Skyline查询算法研究[J].计算机科学与探索,2011,5(9):804-814.
[7]张波良,周水庚,关佶红.MapReduce框架下的Skyline计算[J].计算机科学与探索,2011,5(5):385-397.
[8]史恒亮.云计算任务调度研究[D].南京:南京理工大学,2012.
[9]范东来.Hadoop海量数据处理技术详解与项目实战[M].北京:人民邮电出版社,2016.
[10]关国栋.Hadoop容错调度技术研究[D].成都:西南交通大学,2016.
[11]DEAN J.Sanjay Ghem awant.MapReduce:simplified data processing on large clusters[M].ACM,2008:1-27.
[12]TAN K L,ENG P K,OOI B C.Efficient progressive Skyline computation[C]∥Proc of VLDB.2001:301-310.
[13]陈敏.OPNET网络仿真[M].北京:清华大学出版社,2004.
[14]DESHPANDE A,GUESTRIN C,HONG W,et al.Exploiting correlated attributes in acquisitional query processing[C]∥21st International Conference on Data Engineering.Washington,USA,2005:143-154.
[1] 刘卫明, 安冉, 毛伊敏.
基于聚类和WOA的并行支持向量机算法
Parallel Support Vector Machine Algorithm Based on Clustering and WOA
计算机科学, 2022, 49(7): 64-72. https://doi.org/10.11896/jsjkx.210500040
[2] 朱润泽, 秦小麟, 刘嘉琛.
基于查询对象的路网Skyline查询中Why-not问题的研究
Study on Why-not Problem in Skyline Query of Road Network Based on Query Object
计算机科学, 2021, 48(6): 57-62. https://doi.org/10.11896/jsjkx.200700016
[3] 张元鸣, 虞家睿, 蒋建波, 陆佳炜, 肖刚.
面向MapReduce的中间数据传输流水线优化机制
Intermediate Data Transmission Pipeline Optimization Mechanism for MapReduce Framework
计算机科学, 2021, 48(2): 41-46. https://doi.org/10.11896/jsjkx.191000103
[4] 王妍, 韩笑, 曾辉, 刘荆欣, 夏长清.
边缘计算环境下服务质量可信的任务迁移节点选择
Task Migration Node Selection with Reliable Service Quality in Edge Computing Environment
计算机科学, 2020, 47(10): 240-246. https://doi.org/10.11896/jsjkx.190900054
[5] 龚彤艳,张广婷,贾海鹏,袁良.
一种偶数基Cooley-Tukey FFT高性能实现方法
High-performance Implementation Method for Even Basis of Cooley-Tukey FFT
计算机科学, 2020, 47(1): 31-39. https://doi.org/10.11896/jsjkx.190900179
[6] 周剑刚, 秦小麟, 张珂珩, 许建秋.
基于道路网的多移动用户动态Skyline查询
Dynamic Skyline Query for Multiple Mobile Users Based on Road Network
计算机科学, 2019, 46(9): 73-78. https://doi.org/10.11896/j.issn.1002-137X.2019.09.009
[7] 王童, 马文平, 罗维.
基于区块链的信息共享及安全多方计算模型
Information Sharing and Secure Multi-party Computing Model Based on Blockchain
计算机科学, 2019, 46(9): 162-168. https://doi.org/10.11896/j.issn.1002-137X.2019.09.023
[8] 冯贵兰, 李正楠, 周文刚.
大数据分析技术在网络领域中的研究综述
Research on Application of Big Data Analytics in Network
计算机科学, 2019, 46(6): 1-20. https://doi.org/10.11896/j.issn.1002-137X.2019.06.001
[9] 魏亮, 林子雨, 赖永炫.
DFTS:面向大数据集的Top-k Skyline查询算法
DFTS:A Top-k Skyline Query for Large Datasets
计算机科学, 2019, 46(5): 150-156. https://doi.org/10.11896/j.issn.1002-137X.2019.05.023
[10] 王晓霞, 孙德才.
一种基于Q-sample的局部相似连接并行算法
Q-sample-based Local Similarity Join Parallel Algorithm
计算机科学, 2019, 46(12): 38-44. https://doi.org/10.11896/jsjkx.190100240
[11] 胡赢双, 陆亿红.
基于MapReduce的强连通网格聚类算法
Cell Clustering Algorithm Based on MapReduce and Strongly Connected Fusion
计算机科学, 2019, 46(11A): 204-207.
[12] 张滨, 乐嘉锦.
基于列存储的MapReduce分布式Hash连接算法
Hash Join in MapReduce Distributed Environment Based on Column-store
计算机科学, 2018, 45(6A): 471-475.
[13] 周华平,刘光宗,张贝贝.
基于索引偏移的MapReduce聚类负载均衡策略
Load Balancing Strategy of MapReduce Clustering Based on Index Shift
计算机科学, 2018, 45(5): 303-309. https://doi.org/10.11896/j.issn.1002-137X.2018.05.053
[14] 王华进,黎建辉,沈志宏,周园春.
基于ORC元数据的Hive Join查询Reducer负载均衡方法
ORC Metadata Based Reducer Load Balancing Method for Hive Join Queries
计算机科学, 2018, 45(3): 158-164. https://doi.org/10.11896/j.issn.1002-137X.2018.03.025
[15] 缪丰羽, 王宏志, 阮群生.
一种基于MapReduce的不确定图上的相似性连接方法
Method of Similarity Join on Uncertain Graphs Using MapReduce
计算机科学, 2018, 45(12): 299-307. https://doi.org/10.11896/j.issn.1002-137X.2018.12.048
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!