计算机科学

计算机科学 ›› 2019, Vol. 46 ›› Issue (6A): 1-5.

• 综述研究 •    下一篇

分布式消息系统研究综述

吴璨1,2, 王小宁1, 肖海力1, 曹荣强1, 赵一宁1, 迟学斌1,2   

  1. 中国科学院计算机网络信息中心 北京1001901;
    中国科学院大学 北京1000492
  • 出版日期:2019-06-14 发布日期:2019-07-02
  • 通讯作者: 王小宁(1981－),女,博士,副研究员,主要研究方向为网格计算、云计算和分布式系统,E-mail:wxn@sccas.cn
  • 作者简介:吴 璨(1992－),女,博士生,主要研究方向为高性能计算、可视化与网格技术,E-mail:wucan@sccas.cn;肖海力(1978－),男,博士,研究员,主要研究方向为网格技术和分布式系统;曹荣强(1982－),男,博士,副研究员,主要研究方向为网格计算和云计算;赵一宁(1983－),男,博士,助理研究员,主要研究方向为分布式系统与大数据分析;迟学斌(1963－),男,博士,研究员,博士生导师,主要研究方向为高性能计算、可视化与网格技术。
  • 基金资助:
    本文受国家重点研发计划课题(2018YFB0204001),中国科学院青年创新促进会(2017216)资助。

Survey on Distributed Message System

WU Can1,2, WANG Xiao-ning1, XIAO Hai-li1, CAO Rong-qiang1, ZHAO Yi-ning1, CHI Xue-bin1,2   

  1. Computer Network Information Center,Chinese Academy of Sciences,Beijing 100190,China1;
    University of Chinese Academy of Sciences,Beijing 100049,China2
  • Online:2019-06-14 Published:2019-07-02

PDF (PC)

269

摘要： 随着大数据时代的到来,各类软硬件系统的高并发访问、海量数据处理等需求越来越多,系统的高可用、易伸缩、可扩展成为系统研发的首要目标,分布式系统应运而生,提供了满足高性能需求的解决方案。然而,系统分布式地部署在不同的计算机上,使得系统间的消息通信成为重要问题。文章综述了4种流行的开源分布式消息系统,对比分析了RabbitMQ,Kafka,ActiveMQ和RocketMQ的架构及性能,为科研人员和系统开发者选择分布式消息系统提供了参考意见。

关键词: 分布式消息系统, RabbitMQ, Kafka, ActiveMQ, RocketMQ

Abstract: With the advent of the Big Data Era,there have been increasing demands for the high concurrent access and mass data processing of all kinds of hardware and software systems.High availability,extensibility and scalability are the main driving forces for system development.Distributed systems emerge as new age comes,providing solutions for higher performance requirements.However,as distributed systems are deployed on different computers,message communication between the systems has become an important problem.This paper provided an overview of the research progress of four popular open source distributed message systems:RabbitMQ,Kafka,ActiveMQ and RocketMQ.The architecture and performance of them are compared and analyzed,providing information and references for researchers and developers when choosing distributed message systems as an option.

Key words: Distributed message system, RabbitMQ, Kafka, ActiveMQ, RocketMQ

中图分类号: 

  • TP311
[1] Distributed Systems Concepts and Design.George Coulouris [M].America:Addison-Wesley,2012.
[2] WOOD I.Distributed Message Transmission System and Me-thod:WO,EP1477034[P].2004.
[3] GE Y,LIANG X X,PAN Z,et al.MESSAGE PARSING IN A DISTRIBUTED STREAM PROCESSING SYSTEM:U.S.Patent Application 15/258,629[P].2018-3-8.
[4] CONSULTANT M P C T.Chapter 3.Point-to-Point(PTP) and Point-to-Multipoint(PMP) Wireless Systems and Antennas[M]∥Wireless Access Networks:Fixed Wireless Access and WLL Networks-Design and Operation.John Wiley & Sons,Ltd,2001:41-56.
[5] EUGSTER P T.The many faces of publish/subscribe[J].ACM Computing Surveys(CSUR),2003,35(2):114-131.
[6] GUPTA A,SAHIN O D,AGRAWAL D,et al.Meghdoot:Content-Based Publish/Subscribe over P2P Networks[M]∥Middleware 2004.Springer Berlin Heidelberg,2004:254-273.
[7] SURHONE L M,TIMPLEDON M T,MARSEKEN S F,et al.RabbitMQ[M].Betascript Publishing,2010.
[8] VIDELA A,WILLIAMS J J W.RabbitMQ in action:distributed messaging for everyone.Manning Publications Co.,2012.
[9] ROSTANSKI M,GROCHLA K,SEMAN A.Evaluation of highly available and fault-tolerant middleware clustered architectures using RabbitMQ[C]∥Computer Science and Information Systems.IEEE,2014:879-884.
[10] AYANOGLU E,AYTAS Y,NAHUM D.Mastering RabbitMQ[M].Packt Publishing,2016,1:76-77.
[11] TKRAMER J.Advanced message queuing protocol(AMQP).Belltown Media,2009.
[12] VIDELA A,VIDELA A.RabbitMQ in Action[M].Manning Publications,2012.
[13] GILLES.Kafka:toward a minor literature[M].University of Minnesota Press,1986.
[14] ESMAILI K S,ESMAILI K S.Kafka versus RabbitMQ:A comparative study of two industry reference publish/subscribe implementations:Industry Paper[C]∥ACM International Confe-rence on Distributed and Event-Based Systems.ACM,2017:227-238.
[15] PROULX T,HEINE S J.Connections from Kafka:exposure to meaning threats improves implicit learning of an artificial grammar[J].Psychological Science,2009,20(9):1125.
[16] COMPUTING C C,WANG M H,LI Y,et al.Redesigning Kafka Message Queue System:Toward a Decentralized Stateful Broker System[J].2017.
[17] GARG N.Apache Kafka[M].Packt Publishing Ltd,2013.
[18] HENJES R,SCHLOSSER D,MENTH M,et al.Throughput Performance of the ActiveMQ JMS Server[M]∥Kommunikation in VerteiltenSystemen(KiVS).DBLP,2007:113-124.
[19] IONESCU V M.The analysis of the performance of RabbitMQ and ActiveMQ[C]∥Roedunet International Conference-Networking in Education and Research.IEEE,2015:132-137.
[20] KLEIN A F,STEFANESCU M,SAIED A,et al.An experimental comparison of ActiveMQ and OpenMQ brokers in asynchronous cloud environment[C]∥Fifth International Confe-rence on Digital Information Processing and Communications.IEEE,2015:24-30.
[21] HENJES R,MENTH M,HIMMLER V.Impact of Complex Filters on the Message Throughput of the ActiveMQ JMS Ser-ver[M]∥Managing Traffic Performance in Converged Networks.Springer Berlin Heidelberg,2007:192-203.
[22] CHEN G,DU Y,QIN P,et al.Design and Realization of ActiveMQ Based High Performance Message Cluster[C]∥International Conference on Chemical Science and Engineering.2013.
[23] ZHANG Y,XU L X.Feature and Configuration Research of Ac-tiveMQ[J].Computer Programming Skills & Maintenance,2011,12:6-13.
[24] SURHONE L M,TENNOE M T,HENSSONOW S F,et al.OpenWire (binary protocol).Betascript Publishing,2010.
[25] SURHONE L M,TENNOE M T,HENSSONOW S F,et al.WS-Notification[M].Betascript Publishing,2010.
[26] MILLER J.A Feature Set for the Extensible Messaging and Presence Protocol (XMPP).China Educational Technology,2015,17:2025-2032.
[27] SNYDER B,BOSANAC D,DAVIES R.ActiveMQ in Action [M].Manning Publications Co,2011.
[28] LIAO J,ZHUANG X,FAN R,et al.Toward a General Distributed Messaging Framework for Online Transaction Processing Applications[J].IEEE Access,2017,PP(99):1.
[29] WANG X R,WANG X R,FENG J.From Alibaba to Apache:RocketMQ’s Past,Present,and Future[EB/OL].https://www.infoq.com/articles/alibaba-apache-rocketmq.
[30] MA Y,YAN R,SUN J,et al.A MQTT Protocol Message Push Server Based on RocketMQ[C]∥International Conference on Intelligent Computation Technology and Automation.2017:295-298.
[31] The Apache Software Foundation.RocketMQArchitecture[EB/OL].http://rocketmq.apache.org/docs/rmq-arc.
[32] HINTJENS P.ZeroMQ:messaging for many applications[M].O’Reilly Media,Inc.,2013.
[33] FENGPING P,JIANZHENG C.Distributed system based on ZeroMQ[J].Electronic Test,2012,7(7):24-29.
[34] HINTJENS P.ZeroMQ:云时代极速消息通信库.北京:电子工业出版社,2015.
[35] CARLSON J L.Redis in action[M].Manning Publications Co.,2013.
[36] CARLSON J L.Redis in Action.Manning Publications Co.,2012.
[1] 易佳,薛晨,王树鹏. 分布式流数据加载和查询技术优化[J]. 计算机科学, 2017, 44(5): 172-177.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] 王欢,张云峰,张艳. 一种基于CFDs规则的修复序列快速判定方法[J]. 计算机科学, 2018, 45(3): 311 -316 .
[2] 伍建辉,黄中祥,李武,吴健辉,彭鑫,张生. 城市道路建设时序决策的鲁棒优化[J]. 计算机科学, 2018, 45(4): 89 -93 .
[3] 刘琴. 计算机取证过程中基于约束的数据质量问题研究[J]. 计算机科学, 2018, 45(4): 169 -172 .
[4] 崔琼,李建华,王宏,南明莉. 基于节点修复的网络化指挥信息系统弹性分析模型[J]. 计算机科学, 2018, 45(4): 117 -121, 136 .
[5] 韩奎奎,谢在鹏,吕鑫. 一种基于改进遗传算法的雾计算任务调度策略[J]. 计算机科学, 2018, 45(4): 137 -142 .
[6] 陆佳炜,马俊,张元鸣,肖刚. 面向全局社交服务网的Web服务聚类方法[J]. 计算机科学, 2018, 45(3): 204 -212 .
[7] 吕涛,郝泳涛. 基于相似性匹配和聚类的K线模式可盈利性研究[J]. 计算机科学, 2018, 45(3): 182 -188 .
[8] 戴文静, 袁家斌. 隐含子群问题的研究现状[J]. 计算机科学, 2018, 45(6): 1 -8 .
[9] 蔡莉,梁宇,朱扬勇,何婧. 数据质量的历史沿革和发展趋势[J]. 计算机科学, 2018, 45(4): 1 -10 .
[10] 梁俊斌,周翔,王田,李陶深. 移动低占空比无线传感网中数据收集的研究进展[J]. 计算机科学, 2018, 45(4): 19 -24, 52 .
渝ICP备16013121号-4
版权所有 © 2020 《计算机科学》杂志社
地址：重庆市渝北区洪湖西路18号    邮编：401121  
联系电话：023-63500828（编辑部）  023-67039623（财务、发票）   E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发