计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (6): 111-117.doi: 10.11896/jsjkx.231000140

• 数据库&大数据&数据科学 • 上一篇    下一篇

CDES:数据驱动的云数据库效能评估方法

韩宇捷1, 徐志杰1, 杨定裕2, 黄波1, 郭健美1   

  1. 1 华东师范大学数据科学与工程学院 上海 200062
    2 阿里巴巴集团 杭州 311121
  • 收稿日期:2023-10-20 修回日期:2024-04-01 出版日期:2024-06-15 发布日期:2024-06-05
  • 通讯作者: 郭健美(jmguo@dase.ecnu.edu.cn)
  • 作者简介:(yjhan@stu.ecnu.edu.cn)
  • 基金资助:
    国家自然科学基金面上项目(62272167)

CDES:Data-driven Efficiency Evaluation Methodology for Cloud Database

HAN Yujie1, XU Zhijie1, YANG Dingyu2, HUANG Bo1, GUO Jianmei1   

  1. 1 School of Data Science and Engineering,East China Normal University,Shanghai 200062,China
    2 Alibaba Group,Hangzhou 311121,China
  • Received:2023-10-20 Revised:2024-04-01 Online:2024-06-15 Published:2024-06-05
  • About author:HAN Yujie,born in 2000,postgraduate,is a member of CCF(No.C4735G).His main research interests include FinOps and system optimization.
    GUO Jianmei,born in 1981,Ph.D,professor,Ph.D supervisor,is a member of CCF(No.55252S).His main research interests include software engineering,performance engineering,cloud computing and software product lines.
  • Supported by:
    National Natural Science Foundation of China(62272167).

PDF (PC)

511

摘要： 在大规模云生产环境中在线评估数据库效能,对云厂商进一步优化云成本至关重要。为了评估云数据库的使用效能,提出了一种数据驱动的、基于计算与存储指标融合的云数据库效能评估方法CDES。该方法根据云数据库实例负载行为和性能画像,从计算和存储两方面选取影响云数据库成本与效能的主要指标,再结合云监控平台采集的数据,评估云数据库实例与集群的线上实际使用效能。基于CDES评估结果,进一步提出了云数据库效能优化的治理方案,提供效能优化建议,引导用户减少闲置资源。CDES已被部署在某大型互联网企业生产环境中,并用于其OLTP云数据库产品的效能评价。实验结果表明,所提方法能有效评估超过5 000个云数据库实例的集群的效能并引导治理,单位业务量下实例最高能节省40.74%的成本。

关键词: 云计算, 云数据库, 效能评估, 云成本优化

Abstract: Evaluating database efficiency in a large-scale cloud production environment is crucial for cloud vendors to optimize the cloud cost.In order to evaluate the use efficiency of cloud database,this paper proposes CDES,a data-driven cloud database efficiency evaluation method based on the fusion of computing and storage indicators.According to the load behavior and perfor-mance profile of the cloud database instance,this method selects the main metrics that affect the cost and efficiency of the cloud database from two aspects of computing and storage,and then combines the data collected by the cloud monitoring platform to evaluate the efficiency of the cloud database instance and cluster.Based on the evaluation results of CDES,the governance scheme of cloud database efficiency is further proposed,together with the governance optimization suggestions to guide users to improve the efficiency of resource utilization and reduce idle resources.Finally,CDES has been deployed in the production environment of a large Internet enterprise and used for the performance evaluation of the cloud OLTP database product.The results show that the proposed method can effectively evaluate the efficiency and guide governance of the cluster with more than 5 000 cloud database instances,and the governance results can save 46.15% of the instance cost at most.

Key words: Cloud computing, Cloud database, Efficiency evaluation, FinOps

中图分类号: 

  • TP311
[1]AL SHEHRI W.Cloud Database Database as a Service[J].International Journal of Database Management Systems,2013,5(2):1-12.
[2]Flexera.2023 state of the cloud[EB/OL].https://info.flexera.com/CM-REPORT-State-of-the-Cloud/.
[3]Amazon Web Service,Inc.AWS Billing Console[EB/OL].(2023-05-29)[2023-05-29].https://aws.amazon.com/cn/aws-cost-management/aws-billing/.
[4]Microsoft Inc.Microsoft CostManagement[EB/OL].(2023-05-29)[2023-05-29].https://azure.microsoft.com/zh-cn/produ-cts/cost-management/.
[5]VAN RENEN A,LEIS V.Cloud Analytics Benchmark[J].Proceedings of the VLDB Endowment,2023,16(6):1413-1425.
[6]HWANG K,BAI X,SHI Y,et al.Cloud Performance Modeling with BenchmarkEvaluation of Elastic Scaling Strategies[J].IEEE Transactions on Parallel and Distributed Systems,2016,27(1):130-143.
[7]VERBITSKI A,GUPTA A,SAHA D,et al.Amazon Aurora:Design Considerations for High Throughput Cloud-Native Relational Databases[C]//Proceedings of the 2017 ACM International Conference on Management of Data.2017.
[8]VERBITSKI A,GUPTA A,SAHA D,et al.Amazon Aurora:On Avoiding Distributed Consensus for I/Os,Commits,and Membership Changes[C]//Proceedings of the 2018 International Conference on Management of Data.2018.
[9]DEPOUTOVITCH A,CHEN C,CHEN J,et al.Taurus database:how to be fast,available,and frugal in the cloud[C]//Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data.2020.
[10]Alibaba Cloud.Databases[EB/OL].(2023-06-13)[2023-06-13].https://www.alibabacloud.com/en/product/databases.
[11]CAI B,ZHAO L,ZHOU X,et al.On evaluatingthe resourceusage effectiveness of Multi-tenant cloud Storage[J].Journal of Systems Architecture,2019,98:403-412.
[12]XIAO W,BAO W,ZHU X,et al.Cost Minimization Method for Multi-Source Big Data Processing in Clouds[J].Journal of Software,2017,28(3):544-562.
[13]ANTONOPOULOS P,BUDOVSKI A,DIACONU C,et al.Socrates:The New SQL Server in the Cloud[C]//Proceedings of the 2019 International Conference on Management of Data.2019.
[14]CHAWLA S,DEEP S,KOUTRISW P,et al.Revenue maximization for query Pricing[J].Proceedings of the VLDB Endowment,2019,13(1):1-14.
[15]MÜLLER I,MARROQUÍN R,ALONSO G.Lambada:Interactive Data Analytics on Cold Data Using Serverless Cloud Infrastructure[C]//Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data.2020:115-130.
[16]DUAN S,THUMMALA V,BABU S.Tuning database configuration parameterswith ITuned[J].Proceedings of the VLDB Endowment,2009,2(1):1246-1257.
[17]ZHANG J,LIU Y,ZHOU K,et al.An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning[C]//Proceedings of the 2019 International Conference on Management of Data.2019.
[18]VAN AKEN D,PAVLO A,GORDON G J,et al.Automatic Database Management System Tuning Through Large-scale Machine Learning[C]//Proceedings of the 2017 ACM Interna-tional Conference on Management of Data.2017.
[19]ZHU Y,LIU J,GUO M,et al.BestConfig:Tapping the Per-formance Potential of Systems via Automatic Configuration Tuning[C]//Proceedings of the 2017 Symposium on Cloud Computing.2017.
[20]CAO R,BAO L,CUI J,et al.Survey of Approaches to Parameter Tuning for Database Systems[J].Journal of Computer Research and Development,2023,60(3):635-653.
[21]ZHANG J,LIU Y,ZHOU K,et al.An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning[C]//Proceedings of the 2019 International Confe-rence on Management of Data.2019.
[22]GREENBERG A,HAMILTON J,MALTZ D A,et al.The cost of a Cloud[J].ACM SIGCOMM Computer Communication Review,2008,39(1):68-73.
[23]LANG W,SHANKAR S,PATEL J M,et al.Towards Multi-Tenant Performance SLOs[J].IEEE Transactions on Know-ledge & Data Engineering,2014,26(6):1447-1463.
[24]MOELLER J,YE Z,LIN K,et al.Toto:Benchmarking the Efficiency of a Cloud Service[C]//Proceedings of the 2021 International Conference on Management of Data.2021.
[25]Alibaba Cloud.ESSDs[EB/OL].(2023-06-13)[2023-06-13].https://www.alibabacloud.com/help/en/ecs/user-guide/essds.
[26]MA M,YIN Z,ZHANG S,et al.Diagnosing root causes of inter-mittent slow queries in cloud Databases[J].Proceedings of the VLDB Endowment,2020,13(8):1176-1189.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发