Computer Science ›› 2024, Vol. 51 ›› Issue (6): 104-110.doi: 10.11896/jsjkx.231000156

• Database & Big Data & Data Science • Previous Articles     Next Articles

Study on Method for Collaborative Tuning Resources and Parameters of Cloud Database

LI Yuhang1,3, TAN Ruixiong2, CHAI Yunpeng2,3   

  1. 1 Key Laboratory of Data Engineering and Knowledge Engineering,Ministry of Education,Beijing 100872,China
    2 Engineering Research Center of Database and Business Intelligence,Ministry of Education,Beijing 100872,China
    3 School of Information,Renmin University of China,Beijing 100872,China
  • Received:2023-10-20 Revised:2024-04-03 Online:2024-06-15 Published:2024-06-05
  • About author:LI Yuhang,born in 2000,postgraduate.Her main research interest is database systems.
    CHAI Yunpeng,born in 1983,Ph.D,professor,Ph.D supervisor,is a member of CCF(No.25233M).His main research interests include key-value sto-rage systems,the emerging storage devices,and cloud resource allocation.
  • Supported by:
    National Natural Science Foundation of China(61972402,61972275).

Abstract: In cloud databases,there are numerous configuration options,including internal database parameters and virtual machine resource configuration for the environment deployment,which collectively determine the database’s read/write performance and resource consumption.In the cloud environment with elastic resources,users are concerned about both the database’s service performance and resource consumption costs.However,due to the large number of configuration options and rapid workload changes,finding the optimal combination of configurations becomes challenging.To address the online tuning scenario with dynamically changing workloads,this paper proposes CoTune,a fast tuning method for coordinating cloud database resources and parameters.This method focuses on OLTP workloads and iteratively adjusts the configurations of virtual machine resources and database parameters to minimize resource consumption while ensuring service quality.The method introduces several key innovations:firstly,it adopts a three-stage approach within each tuning cycle to adjust resource quotas and database parameters,prioritizing service quality;secondly,it classifies the impact of database parameters on different resources,reducing the search space and enabling rapid parameter adjustments;and finally,it incorporates a reinforcement learning model for database parameter tuning,with a specific reward function designed to quickly obtain reward values and accelerate the tuning frequency.Experimental results demonstrate that,compared to approaches that simultaneously tune resources and parameters or solely focus on resource tuning,the proposed method reduces resource consumption while maintaining service quality.Through rapid iterative tuning,it effectively addresses the challenges posed by workload variations and achieves more efficient resource utilization in dynamic workload environments.

Key words: Cloud database, Parameter tuning, Quality-of-Service, Resource consumption

CLC Number: 

  • TP392
[1]RENOUARD J.MySQLTuner[OL].https://github.com/major/MySQLTuner-perl.
[2]KWAN E,LIGHTSTONE S,STORM A,et al.Automatic configuration for IBM DB2 universal database[J].Proceedings of IBM Perf Technical Report,2002,1(1):1-13.
[3]ZHU Y,LIU J,GUO M,et al.Bestconfig:tappingthe perfor-mance potential of systems via automatic configuration tuning[C]//Proceedings of the 2017 Symposium on Cloud Computing.2017:338-350.
[4]DUAN S,THUMMALA V,BABU S.Tuning database configuration parameters with ituned[J].Proceedings of the VLDB Endowment,2009,2(1):1246-1257.
[5]ZHANG B,VAN AKEN D,WANG J,et al.A demonstration of the ottertune automatic database management system tuning service[J].Proceedings of the VLDB Endowment,2018,11(12):1910-1913.
[6]SIEGMUND N,GREBHAHN A,APELS,et al.Performance-influence models for highly configurable systems[C]//Procee-dings of the 2015 10th Joint Meeting on Foundations of Software Engineering.2015:284-294.
[7]NAIR V,MENZIES T,SIEGMUND N,et al.Using bad lear-ners to find good configurations[C]//Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering.2017:257-267.
[8]ZHANG J,LIU Y,ZHOU K,et al.An end-to-end automatic cloud database tuning system using deep reinforcement learning[C]//Proceedings of the 2019 International Conference on Ma-nagement of Data.2019:415-432.
[9]LI G,ZHOU X,LI S,et al.Qtune:A query-aware database tu-ning system with deep reinforcement learning[J].Proceedings of the VLDB Endowment,2019,12(12):2118-2130.
[10]ZHANG X,WU H,CHANG Z,et al.Restune:Resource oriented tuning boosted by meta-learning for cloud databases[C]//Proceedings of the 2021 International Conference on Management of Data.2021:2102-2114.
[11]Google Inc.Kubernetes[EB/OL].(2023-02-14)[2023-03-15].https://kubernetes.io.
[12]Amazon Inc.Amazon Elastic Compute Cloud[EB/OL].(2023-01-29)[2023-03-15].https://aws.amazon.com/cn/ec2/.
[13]Flexera Software.RightScale Cloud Management[EB/OL].(2022-12-05)[2023-03-15].http://docs.rightscale.com.
[14]RAO J,BU X,XU C Z,et al.VCONF:a reinforcement learning approach to virtual machines auto-configuration[C]//Procee-dings of the 6th International Conference on Autonomic Computing.2009:137-146.
[15]BARRETT E,HOWLEY E,DUGGAN J.Applying reinforce-ment learning towards automating resource allocation and application scalability in the cloud[J].Concurrency and Computation:Practice and Experience,2013,25(12):1656-1674.
[16]KLIMOVIC A,LITZ H,KOZYRAKIS C.Selecta:Heterogeneous cloud storage configuration for data analytics[C]//2018 {USENIX} Annual Technical Conference({USENIX}{ATC} 18).2018:759-773.
[17]MAHGOUB A,MEDOFF A,KUMAR R,et al.OPTIMUS-CLOUD:Heterogeneous configuration optimization for distributed databases in the cloud[C]//Proceedings of the 2020 USENIX Conference on Usenix Annual Technical Conference.2020:189-204.
[18]LI G L,ZHOU X H,SUN J,et al.A Survey of Database Technologies Based on Machine Learning[J].Chinese Journal of Computers,2020,43(11):2019-2049.
[19]LILLICRAP T P,HUNT J J,PRITZEL A,et al.Continuouscontrol with deep reinforcement learning[J].arXiv:1509.02971,2015.
[20]B2W Digital 2017.B2W Digital[OL].https://www.b2wdigi-tal.com.
[1] HAN Yujie, XU Zhijie, YANG Dingyu, HUANG Bo, GUO Jianmei. CDES:Data-driven Efficiency Evaluation Methodology for Cloud Database [J]. Computer Science, 2024, 51(6): 111-117.
[2] LI Lu-lu, DONG Qing-kuan, CHEN Meng-meng. Cloud-based Lightweight RFID Group Tag Authentication Protocol [J]. Computer Science, 2019, 46(1): 182-189.
[3] ZHANG Gang, GAO Jun-peng, LI Hong-wei. Research on Stochastic Resonance Characteristics of Cascaded Three-steady-state and Its Application [J]. Computer Science, 2018, 45(9): 146-151.
[4] XU Yang, CHEN Yi, HUANG Lei, XIE Xiao-yao. Crowd Counting Method Based on Multilayer BP Neural Networks and Non-parameter Tuning [J]. Computer Science, 2018, 45(10): 235-239.
[5] YOU Chuan-chuan and ZHANG Gui-gang. A Kind of Efficient Search Method Based on Big Data [J]. Computer Science, 2013, 40(6): 183-186.
[6] SHI Heng-liang,BAI Guang-yi,TANG Zhen-min, LIU Chuan-ling. Cloud Database Dynamic Route Scheduling Based on Ant Colony Optimization Algorithm [J]. Computer Science, 2010, 37(5): 143-145.
[7] . [J]. Computer Science, 2008, 35(8): 284-286.
[8] WU Ren-Yong ,ZHU Guang-Xi (Department of Electronics & Information Engineering, Huazhong University of Science & Technology, Wuhan, 430074). [J]. Computer Science, 2006, 33(11): 18-20.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!