计算机科学 ›› 2024, Vol. 51 ›› Issue (6): 104-110.doi: 10.11896/jsjkx.231000156

• 数据库&大数据&数据科学 • 上一篇    下一篇

云数据库资源与参数协同调优方法研究

李雨航1,3, 谭睿雄2, 柴云鹏2,3   

  1. 1 数据工程与知识工程教育部重点实验室 北京 100872
    2 数据库与商务智能教育部工程研究中心 北京 100872
    3 中国人民大学信息学院 北京 100872
  • 收稿日期:2023-10-20 修回日期:2024-04-03 出版日期:2024-06-15 发布日期:2024-06-05
  • 通讯作者: 柴云鹏(ypchai@ruc.edu.cn)
  • 作者简介:(liyuhang@ruc.edu.cn)
  • 基金资助:
    国家自然科学基金(61972402,61972275)

Study on Method for Collaborative Tuning Resources and Parameters of Cloud Database

LI Yuhang1,3, TAN Ruixiong2, CHAI Yunpeng2,3   

  1. 1 Key Laboratory of Data Engineering and Knowledge Engineering,Ministry of Education,Beijing 100872,China
    2 Engineering Research Center of Database and Business Intelligence,Ministry of Education,Beijing 100872,China
    3 School of Information,Renmin University of China,Beijing 100872,China
  • Received:2023-10-20 Revised:2024-04-03 Online:2024-06-15 Published:2024-06-05
  • About author:LI Yuhang,born in 2000,postgraduate.Her main research interest is database systems.
    CHAI Yunpeng,born in 1983,Ph.D,professor,Ph.D supervisor,is a member of CCF(No.25233M).His main research interests include key-value sto-rage systems,the emerging storage devices,and cloud resource allocation.
  • Supported by:
    National Natural Science Foundation of China(61972402,61972275).

摘要: 云数据库中存在许多配置项,包括数据库内部的配置参数以及部署环境的虚拟机资源配置,这些配置项共同决定了数据库的读写性能和资源消耗。在资源弹性伸缩的云环境下,用户关注数据库的服务性能和资源消耗成本。然而,由于配置项众多且负载变化快速,寻找最优的配置项组合变得困难。文中针对负载动态变化的在线调优场景提出了CoTune,一种协同调节云数据库资源与参数的快速调优方法。该方法针对OLTP型动态负载,通过迭代调节云虚拟机资源配置和数据库参数配置,在保障服务质量的前提下降低资源消耗。该方法的创新点如下:首先,在每个调优周期内,采用三阶段方案对资源配额和数据库参数进行调节,优先保障服务质量;其次,根据数据库参数对不同资源的影响进行分类,减小搜索空间,快速调节参数;最后,在数据库参数调节的强化学习模型中,设计特定的奖励函数,快速获取奖励值,加快调节频率。实验结果表明,该方法相比同时调节资源和参数、单独调节资源等方法,能够在保障服务质量的前提下降低资源消耗。通过快速迭代调优,能够应对负载变化的挑战,并在动态负载环境中实现更高效的资源利用。

关键词: 云数据库, 参数调优, 服务质量, 资源消耗

Abstract: In cloud databases,there are numerous configuration options,including internal database parameters and virtual machine resource configuration for the environment deployment,which collectively determine the database’s read/write performance and resource consumption.In the cloud environment with elastic resources,users are concerned about both the database’s service performance and resource consumption costs.However,due to the large number of configuration options and rapid workload changes,finding the optimal combination of configurations becomes challenging.To address the online tuning scenario with dynamically changing workloads,this paper proposes CoTune,a fast tuning method for coordinating cloud database resources and parameters.This method focuses on OLTP workloads and iteratively adjusts the configurations of virtual machine resources and database parameters to minimize resource consumption while ensuring service quality.The method introduces several key innovations:firstly,it adopts a three-stage approach within each tuning cycle to adjust resource quotas and database parameters,prioritizing service quality;secondly,it classifies the impact of database parameters on different resources,reducing the search space and enabling rapid parameter adjustments;and finally,it incorporates a reinforcement learning model for database parameter tuning,with a specific reward function designed to quickly obtain reward values and accelerate the tuning frequency.Experimental results demonstrate that,compared to approaches that simultaneously tune resources and parameters or solely focus on resource tuning,the proposed method reduces resource consumption while maintaining service quality.Through rapid iterative tuning,it effectively addresses the challenges posed by workload variations and achieves more efficient resource utilization in dynamic workload environments.

Key words: Cloud database, Parameter tuning, Quality-of-Service, Resource consumption

中图分类号: 

  • TP392
[1]RENOUARD J.MySQLTuner[OL].https://github.com/major/MySQLTuner-perl.
[2]KWAN E,LIGHTSTONE S,STORM A,et al.Automatic configuration for IBM DB2 universal database[J].Proceedings of IBM Perf Technical Report,2002,1(1):1-13.
[3]ZHU Y,LIU J,GUO M,et al.Bestconfig:tappingthe perfor-mance potential of systems via automatic configuration tuning[C]//Proceedings of the 2017 Symposium on Cloud Computing.2017:338-350.
[4]DUAN S,THUMMALA V,BABU S.Tuning database configuration parameters with ituned[J].Proceedings of the VLDB Endowment,2009,2(1):1246-1257.
[5]ZHANG B,VAN AKEN D,WANG J,et al.A demonstration of the ottertune automatic database management system tuning service[J].Proceedings of the VLDB Endowment,2018,11(12):1910-1913.
[6]SIEGMUND N,GREBHAHN A,APELS,et al.Performance-influence models for highly configurable systems[C]//Procee-dings of the 2015 10th Joint Meeting on Foundations of Software Engineering.2015:284-294.
[7]NAIR V,MENZIES T,SIEGMUND N,et al.Using bad lear-ners to find good configurations[C]//Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering.2017:257-267.
[8]ZHANG J,LIU Y,ZHOU K,et al.An end-to-end automatic cloud database tuning system using deep reinforcement learning[C]//Proceedings of the 2019 International Conference on Ma-nagement of Data.2019:415-432.
[9]LI G,ZHOU X,LI S,et al.Qtune:A query-aware database tu-ning system with deep reinforcement learning[J].Proceedings of the VLDB Endowment,2019,12(12):2118-2130.
[10]ZHANG X,WU H,CHANG Z,et al.Restune:Resource oriented tuning boosted by meta-learning for cloud databases[C]//Proceedings of the 2021 International Conference on Management of Data.2021:2102-2114.
[11]Google Inc.Kubernetes[EB/OL].(2023-02-14)[2023-03-15].https://kubernetes.io.
[12]Amazon Inc.Amazon Elastic Compute Cloud[EB/OL].(2023-01-29)[2023-03-15].https://aws.amazon.com/cn/ec2/.
[13]Flexera Software.RightScale Cloud Management[EB/OL].(2022-12-05)[2023-03-15].http://docs.rightscale.com.
[14]RAO J,BU X,XU C Z,et al.VCONF:a reinforcement learning approach to virtual machines auto-configuration[C]//Procee-dings of the 6th International Conference on Autonomic Computing.2009:137-146.
[15]BARRETT E,HOWLEY E,DUGGAN J.Applying reinforce-ment learning towards automating resource allocation and application scalability in the cloud[J].Concurrency and Computation:Practice and Experience,2013,25(12):1656-1674.
[16]KLIMOVIC A,LITZ H,KOZYRAKIS C.Selecta:Heterogeneous cloud storage configuration for data analytics[C]//2018 {USENIX} Annual Technical Conference({USENIX}{ATC} 18).2018:759-773.
[17]MAHGOUB A,MEDOFF A,KUMAR R,et al.OPTIMUS-CLOUD:Heterogeneous configuration optimization for distributed databases in the cloud[C]//Proceedings of the 2020 USENIX Conference on Usenix Annual Technical Conference.2020:189-204.
[18]LI G L,ZHOU X H,SUN J,et al.A Survey of Database Technologies Based on Machine Learning[J].Chinese Journal of Computers,2020,43(11):2019-2049.
[19]LILLICRAP T P,HUNT J J,PRITZEL A,et al.Continuouscontrol with deep reinforcement learning[J].arXiv:1509.02971,2015.
[20]B2W Digital 2017.B2W Digital[OL].https://www.b2wdigi-tal.com.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!