Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (6): 111-117.doi: 10.11896/jsjkx.231000140

• Database & Big Data & Data Science • Previous Articles     Next Articles

CDES:Data-driven Efficiency Evaluation Methodology for Cloud Database

HAN Yujie1, XU Zhijie1, YANG Dingyu2, HUANG Bo1, GUO Jianmei1   

  1. 1 School of Data Science and Engineering,East China Normal University,Shanghai 200062,China
    2 Alibaba Group,Hangzhou 311121,China
  • Received:2023-10-20 Revised:2024-04-01 Online:2024-06-15 Published:2024-06-05
  • About author:HAN Yujie,born in 2000,postgraduate,is a member of CCF(No.C4735G).His main research interests include FinOps and system optimization.
    GUO Jianmei,born in 1981,Ph.D,professor,Ph.D supervisor,is a member of CCF(No.55252S).His main research interests include software engineering,performance engineering,cloud computing and software product lines.
  • Supported by:
    National Natural Science Foundation of China(62272167).

PDF (PC)

484

Abstract: Evaluating database efficiency in a large-scale cloud production environment is crucial for cloud vendors to optimize the cloud cost.In order to evaluate the use efficiency of cloud database,this paper proposes CDES,a data-driven cloud database efficiency evaluation method based on the fusion of computing and storage indicators.According to the load behavior and perfor-mance profile of the cloud database instance,this method selects the main metrics that affect the cost and efficiency of the cloud database from two aspects of computing and storage,and then combines the data collected by the cloud monitoring platform to evaluate the efficiency of the cloud database instance and cluster.Based on the evaluation results of CDES,the governance scheme of cloud database efficiency is further proposed,together with the governance optimization suggestions to guide users to improve the efficiency of resource utilization and reduce idle resources.Finally,CDES has been deployed in the production environment of a large Internet enterprise and used for the performance evaluation of the cloud OLTP database product.The results show that the proposed method can effectively evaluate the efficiency and guide governance of the cluster with more than 5 000 cloud database instances,and the governance results can save 46.15% of the instance cost at most.

Key words: Cloud computing, Cloud database, Efficiency evaluation, FinOps

CLC Number: 

  • TP311
[1]AL SHEHRI W.Cloud Database Database as a Service[J].International Journal of Database Management Systems,2013,5(2):1-12.
[2]Flexera.2023 state of the cloud[EB/OL].https://info.flexera.com/CM-REPORT-State-of-the-Cloud/.
[3]Amazon Web Service,Inc.AWS Billing Console[EB/OL].(2023-05-29)[2023-05-29].https://aws.amazon.com/cn/aws-cost-management/aws-billing/.
[4]Microsoft Inc.Microsoft CostManagement[EB/OL].(2023-05-29)[2023-05-29].https://azure.microsoft.com/zh-cn/produ-cts/cost-management/.
[5]VAN RENEN A,LEIS V.Cloud Analytics Benchmark[J].Proceedings of the VLDB Endowment,2023,16(6):1413-1425.
[6]HWANG K,BAI X,SHI Y,et al.Cloud Performance Modeling with BenchmarkEvaluation of Elastic Scaling Strategies[J].IEEE Transactions on Parallel and Distributed Systems,2016,27(1):130-143.
[7]VERBITSKI A,GUPTA A,SAHA D,et al.Amazon Aurora:Design Considerations for High Throughput Cloud-Native Relational Databases[C]//Proceedings of the 2017 ACM International Conference on Management of Data.2017.
[8]VERBITSKI A,GUPTA A,SAHA D,et al.Amazon Aurora:On Avoiding Distributed Consensus for I/Os,Commits,and Membership Changes[C]//Proceedings of the 2018 International Conference on Management of Data.2018.
[9]DEPOUTOVITCH A,CHEN C,CHEN J,et al.Taurus database:how to be fast,available,and frugal in the cloud[C]//Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data.2020.
[10]Alibaba Cloud.Databases[EB/OL].(2023-06-13)[2023-06-13].https://www.alibabacloud.com/en/product/databases.
[11]CAI B,ZHAO L,ZHOU X,et al.On evaluatingthe resourceusage effectiveness of Multi-tenant cloud Storage[J].Journal of Systems Architecture,2019,98:403-412.
[12]XIAO W,BAO W,ZHU X,et al.Cost Minimization Method for Multi-Source Big Data Processing in Clouds[J].Journal of Software,2017,28(3):544-562.
[13]ANTONOPOULOS P,BUDOVSKI A,DIACONU C,et al.Socrates:The New SQL Server in the Cloud[C]//Proceedings of the 2019 International Conference on Management of Data.2019.
[14]CHAWLA S,DEEP S,KOUTRISW P,et al.Revenue maximization for query Pricing[J].Proceedings of the VLDB Endowment,2019,13(1):1-14.
[15]MÜLLER I,MARROQUÍN R,ALONSO G.Lambada:Interactive Data Analytics on Cold Data Using Serverless Cloud Infrastructure[C]//Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data.2020:115-130.
[16]DUAN S,THUMMALA V,BABU S.Tuning database configuration parameterswith ITuned[J].Proceedings of the VLDB Endowment,2009,2(1):1246-1257.
[17]ZHANG J,LIU Y,ZHOU K,et al.An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning[C]//Proceedings of the 2019 International Conference on Management of Data.2019.
[18]VAN AKEN D,PAVLO A,GORDON G J,et al.Automatic Database Management System Tuning Through Large-scale Machine Learning[C]//Proceedings of the 2017 ACM Interna-tional Conference on Management of Data.2017.
[19]ZHU Y,LIU J,GUO M,et al.BestConfig:Tapping the Per-formance Potential of Systems via Automatic Configuration Tuning[C]//Proceedings of the 2017 Symposium on Cloud Computing.2017.
[20]CAO R,BAO L,CUI J,et al.Survey of Approaches to Parameter Tuning for Database Systems[J].Journal of Computer Research and Development,2023,60(3):635-653.
[21]ZHANG J,LIU Y,ZHOU K,et al.An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning[C]//Proceedings of the 2019 International Confe-rence on Management of Data.2019.
[22]GREENBERG A,HAMILTON J,MALTZ D A,et al.The cost of a Cloud[J].ACM SIGCOMM Computer Communication Review,2008,39(1):68-73.
[23]LANG W,SHANKAR S,PATEL J M,et al.Towards Multi-Tenant Performance SLOs[J].IEEE Transactions on Know-ledge & Data Engineering,2014,26(6):1447-1463.
[24]MOELLER J,YE Z,LIN K,et al.Toto:Benchmarking the Efficiency of a Cloud Service[C]//Proceedings of the 2021 International Conference on Management of Data.2021.
[25]Alibaba Cloud.ESSDs[EB/OL].(2023-06-13)[2023-06-13].https://www.alibabacloud.com/help/en/ecs/user-guide/essds.
[26]MA M,YIN Z,ZHANG S,et al.Diagnosing root causes of inter-mittent slow queries in cloud Databases[J].Proceedings of the VLDB Endowment,2020,13(8):1176-1189.
[1] LI Yuhang, TAN Ruixiong, CHAI Yunpeng. Study on Method for Collaborative Tuning Resources and Parameters of Cloud Database [J]. Computer Science, 2024, 51(6): 104-110.
[2] LIU Daoqing, HU Hongchao, HUO Shumin. N-variant Architecture for Container Runtime Security Threats [J]. Computer Science, 2024, 51(6): 399-408.
[3] LIU Xuanyu, ZHANG Shuai, HUO Shumin, SHANG Ke. Microservice Moving Target Defense Strategy Based on Adaptive Genetic Algorithm [J]. Computer Science, 2023, 50(9): 82-89.
[4] LI Yinghao, GUO Haogong, LIU Panpan, XIANG Yihao, LIU Chengming. Cloud Platform Load Prediction Method Based on Temporal Convolutional Network [J]. Computer Science, 2023, 50(7): 254-260.
[5] ZAHO Peng, ZHOU Jiantao, ZHAO Daming. Cloud Computing Load Prediction Method Based on Hybrid Model of CEEMDAN-ConvLSTM [J]. Computer Science, 2023, 50(6A): 220300272-9.
[6] LI Jinliang, LIN Bing, CHEN Xing. Reliability Constraint-oriented Workflow Scheduling Strategy in Cloud Environment [J]. Computer Science, 2023, 50(10): 291-298.
[7] GAO Shi-yao, CHEN Yan-li, XU Yu-lan. Expressive Attribute-based Searchable Encryption Scheme in Cloud Computing [J]. Computer Science, 2022, 49(3): 313-321.
[8] MA Xin-yu, JIANG Chun-mao, HUANG Chun-mei. Optimal Scheduling of Cloud Task Based on Three-way Clustering [J]. Computer Science, 2022, 49(11A): 211100139-7.
[9] ZHOU Qian, DAI Hua, SHENG Wen-jie, HU Zheng, YANG Geng. Research on Verifiable Keyword Search over Encrypted Cloud Data:A Survey [J]. Computer Science, 2022, 49(10): 272-278.
[10] WANG Zheng, JIANG Chun-mao. Cloud Task Scheduling Algorithm Based on Three-way Decisions [J]. Computer Science, 2021, 48(6A): 420-426.
[11] PAN Rui-jie, WANG Gao-cai, HUANG Heng-yi. Attribute Access Control Based on Dynamic User Trust in Cloud Computing [J]. Computer Science, 2021, 48(5): 313-319.
[12] CHEN Yu-ping, LIU Bo, LIN Wei-wei, CHENG Hui-wen. Survey of Cloud-edge Collaboration [J]. Computer Science, 2021, 48(3): 259-268.
[13] JIANG Hui-min, JIANG Zhe-yuan. Reference Model and Development Methodology for Enterprise Cloud Service Architecture [J]. Computer Science, 2021, 48(2): 13-22.
[14] WANG Wen-juan, DU Xue-hui, REN Zhi-yu, SHAN Di-bin. Reconstruction of Cloud Platform Attack Scenario Based on Causal Knowledge and Temporal- Spatial Correlation [J]. Computer Science, 2021, 48(2): 317-323.
[15] MAO Han-yu, NIE Tie-zheng, SHEN De-rong, YU Ge, XU Shi-cheng, HE Guang-yu. Survey on Key Techniques and Development of Blockchain as a Service Platform [J]. Computer Science, 2021, 48(11): 4-11.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.