Computer Science ›› 2025, Vol. 52 ›› Issue (8): 109-117.doi: 10.11896/jsjkx.250100151

• Database & Big Data ) Data Science • Previous Articles     Next Articles

Query Optimization Algorithm Based on Learning to Rank

YU Yang, PENG Yuwei   

  1. School of Computer Science,Wuhan University,Wuhan 430061,China
  • Received:2025-01-23 Revised:2025-03-07 Online:2025-08-15 Published:2025-08-08
  • About author:YU Yang,born in 2000,postgraduate.His main research interests include database,query optimization,and AI4DB.
    PENG Yuwei,born in 1980,Ph.D,associate professor.His main research in terests include database systems,big data,and digital watermarking.
  • Supported by:
    National Key R&D Program of China(2023YFB4503604).

Abstract: Query optimization is a key aspect of relational databases.In the traditional query optimization process,cardinality estimation of join and filter operations in a query is usually required in order to obtain a better execution plan.However,due to the inaccuracy of cardinality estimation,the results of query optimization are often unsatisfactory.Currently,some researches have been conducted to improve the cardinality estimation through machine learning-based methods and have made some progress.This paper finds that although these methods perform better in dealing with filtering predicates of numerical types in queries,they are ineffective for other complex filtering predicates.To address this problem,this paper proposes a query optimization algorithm based on learning to rank.The algorithm is capable of intelligently evaluating and ranking multiple execution plans for a single query to select the best plan for execution.The query optimization algorithm iteratively mines the better execution plans and collaborates with machine learning methods to finally filter out the optimal plan.Experimental results show that the proposed algorithm outperforms current learning-based query optimization algorithms on regular datasets and is more significant on complex datasets.

Key words: Query optimization, Plan generation, Learning to rank, Database, Join order, Join type, Scan type

CLC Number: 

  • TP392
[1]LEIS V,GUBICHEV A,MIRCHEV A,et al.How Good Are Query Optimizers,Really?[J].Proceedings of the VLDB Endowment,2015,9(3):204-215.
[2]WU P,CONG G.A Unified Deep Model of Learning from both Data and Queries for Cardinality Estimation[C]//International Conference on Management of Data,Virtual Event(SIGMOD '21).ACM,2021:2009-2022.
[3]WU Z,SHAIKHHA A.BayesCard:A Unified Bayesian Framework for Cardinality Estimation[J].arXiv:2012.14743,2020.
[4]YANG Z,KAMSETTY A,LUAN S,et al.NeuroCard:One Cardinality Estimator for All Tables[J].Proceedings of the VLDB Endowment,2020,14(1):61-73.
[5]YANG Z,LIANG E,KAMSETTY A,et al.Deep Unsupervised Cardinality Estimation[J].Proceedigns of the VLDB Endowment,2019,13(3):279-292.
[6]ZHANG J,ZHANG C,LI G,et al.AutoCE:An Accurate and Efficient Model Advisor for Learned Cardinality Estimation[C]//39th IEEE International Conference on Data Engineering(ICDE 2023).IEEE,2023:2621-2633.
[7]ZHU R,WU Z,HAN Y,et al.FLAT:Fast,Lightweight andAccurate Method for Cardinality Estimation[J].Proceedings VLDB Endowment,2021,14(9):1489-1502.
[8]MARCUS R,NEGI P,MAO H,et al.Bao:Making LearnedQuery Optimization Practical[C]//SIGMOD.ACM,2021:1275-1288.
[9]ZHU R,CHEN W,DING B,et al.LERO:A Learning-to-Rank Query Optimizer[J].Proceedings of the VLDB Endowment,2023,16(6):1466-1479.
[10]CHEN X,CHEN H,LIANG Z,et al.LEON:A new framework for ml-aided query optimization[J].Proceedings VLDB Endowment,2023,16:2261-2273.
[11]LAN H,BAO Z,PENG Y.A Survey on Advancing the DBMS Query Optimizer:Cardinality Estimation,Cost Model,and Plan Enumeration[J].Data Sci.Eng.,2021,6(1):86-101.
[12]OLKEN F,ROTEM D.Random Sampling from Database Files:A Survey[C]//Statistical and Scientific Database Management,5th International Conference SSDBM,Charlotte,NC(Lecture Notes in Computer Science,Vol.420).Springer,1990:92-111.
[13]CORMODE G,GAROFALAKIS M N,HAAS P J,et al.Synopses for Massive Data:Samples,Histograms,Wavelets,Sketches[J].Found.Trends Databases,2012,4(1/2/33):1-294.
[14]HASAN S,THIRUMURUGANATHAN S,AUGUSTINE J,et al.Deep Learning Models for Selectivity Estimation of Multi-Attribute Queries[C]//Proceedings of the 2020 International Conference on Management of Data,SIGMOD Conference 2020.ACM,2020:1035-1050.
[15]HILPRECHT B,SCHMIDT A,KULESSA M,et al.DeepDB:Learn from Data,not from Queries![J].Proceedings of the VLDB Endowment,2020,13(7):992-1005.
[16]KIPF A,KIPF T,RADKE B,et al.Learned Cardinalities:Estimating Correlated Joins with Deep Learning[J].arXiv:1809.00677,2018.
[17]SUN J,LI G,TANG N.Learned Cardinality Estimation for Similarity Queries[C]//International Conference on Management of Data(SIGMOD '21).ACM,2021:1745-1757.
[18]ILYAS I F,BESKALES G,SOLIMAN M A.Probabilistic query optimization in database systems[J].ACM SIGMOD Record,2008,37(3):4-11.
[19]ROY D,PANDA P,ROY K.Tree-CNN:A Deep Convolutional Neural Network for Lifelong Learning[J].arXiv:1802.05800,2018.
[20]MOU L,LI G,ZHANG L,et al.Convolutional Neural Networks over Tree Structures for Programming Language Processing[C]//AAAI.AAAI Press,2016:1287-1293.
[21]MARCUS R,PAPAEMMANOUIL O.Plan-Structured DeepNeural Network Models for Query Performance Prediction[J].Proceedings VLDB Endowment,2019,12(11):1733-1746.
[1] MENG Xiangfu, LI Zihan, SHI Jiasheng, GUO Jianwei, ZHAO Liang, GUO Sicong, WANG Peizhuang. Factor Query Language-Basic Language of Factor Database [J]. Computer Science, 2025, 52(6A): 240600027-8.
[2] CHEN Wangxu, WEN Hao, NI Yang. Application of Requirements Traceability in Code Static Analysis [J]. Computer Science, 2025, 52(6A): 241000024-5.
[3] WANG Yun, ZHAO Jianming, GUO Yifeng, ZHOU Huanhuan, ZHOU Wuai, ZHANG Wanzhe, FENG Jianhua. Automation and Security Strategies and Empirical Research on Operation and Maintenance of Digital Government Database [J]. Computer Science, 2025, 52(6A): 240500045-8.
[4] WANG Qing, YANG Wanzhe, ZHANG Cong. Research on Fusion Optimization Method of Heterogeneous Data Dictionary in Grass-roots SocialGrid Governance [J]. Computer Science, 2025, 52(6A): 240400074-7.
[5] ZUO Shun, LI Yongkun, XU Yinlong. Study on Collaborative Data Persistence in NewSQL Databases [J]. Computer Science, 2025, 52(1): 131-141.
[6] LI Chunyu, DENG Long, LI Yongkun, XU Yinlong. Application-aware Disaggregated Storage Design for Remote Memory Graph Database [J]. Computer Science, 2025, 52(1): 151-159.
[7] LIU Yumeng, ZHAO Yijing, WANG Bicong, WANG Chao, ZHANG Baomin. Advances in SQL Intelligent Synthesis Technology [J]. Computer Science, 2024, 51(7): 40-48.
[8] LI Yuhang, TAN Ruixiong, CHAI Yunpeng. Study on Method for Collaborative Tuning Resources and Parameters of Cloud Database [J]. Computer Science, 2024, 51(6): 104-110.
[9] HAN Yujie, XU Zhijie, YANG Dingyu, HUANG Bo, GUO Jianmei. CDES:Data-driven Efficiency Evaluation Methodology for Cloud Database [J]. Computer Science, 2024, 51(6): 111-117.
[10] CHEN Xinyang, CHEN Hanze, ZHOU Jiasheng, HUANG Jiaqing, YU Jiashuo, ZHU Longlong, ZHANG Dong. IntervalSketch:Approximate Statistical Method for Interval Items in Data Stream [J]. Computer Science, 2024, 51(4): 4-10.
[11] XU Haiyang, LIU Hailong, YANG Chaoyun, WANG Shuo, LI Zhanhuai. MMOS:Memory Resource Sharing Methods to Support Overselling in Multi-tenant Databases [J]. Computer Science, 2024, 51(2): 27-35.
[12] ZOU Chunling, ZHU Zhengzhou. Fusion Model of Housekeeping Service Course Recommendation Based on Knowledge Graph [J]. Computer Science, 2024, 51(2): 47-54.
[13] YANG Zhenkai, CAO Yibing, ZHAO Xinke, ZHENG Jingbiao. Temporal Hierarchical Data Management Based on Nested Intervals Scheme in Relational Database [J]. Computer Science, 2023, 50(6A): 220500290-5.
[14] ZHOU Mingxing, YAN Xiangzhou, YU Jing, GAO Changju, CHEN Yunwen, JI Daqi, JIN Ke. Unbiased Deep Learning to Rank Algorithm for Suggestion Auto-completion [J]. Computer Science, 2023, 50(11A): 220800179-5.
[15] HUANG Zhicheng, LIU Xianhui. Microservice Splitting Approach Based on Database Table [J]. Computer Science, 2023, 50(11A): 230200102-7.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!