Abstract: This paper proposes a multi-level SQL optimization framework tailored for business scenarios.By decoupling business logic from general-purpose computation,it establishes a collaborative architecture comprising a front-end rule engine and a back-end execution engine.The front-end SQL optimizer focuses on business-specific optimizations,while the back-end SQL execution engine is dedicated to general big data processing.By encapsulating and integrating existing unified batch-stream proces-sing engines,the framework achieves high computational efficiency.The optimization strategies are systematically divided into two levels:rule-based optimizations oriented toward business characteristics,and general execution optimizations based on the computational framework.This division enhances both modularity and iterability of the optimization strategies.The design and implementation delve into a rule-engine-driven front-end SQL optimizer,which works in conjunction with a general big data execution engine.This synergy enables business-oriented enhancements such as push-down operations for external data sources,consolidation of multiple production tasks,and effectively addresses common practical issues including data skew and code generation limits.Experiments,including external data source push-down tests,multi-task consolidation tests,and data skew load optimization tests,demonstrate performance improvements of 49.8% in field pruning and 79.75% in task merging across multiple real-world scenarios.These results indicate a substantial overall enhancement in the execution performance of big data production tasks.Furthermore,the core design philosophy of decoupling business-specific and general-purpose computation offers an innovative solution for efficient SQL optimization and execution in big data contexts.

Key words: Multi-level SQL optimization, Business rule engine, Query optimizer as a service, External data source pushdown, Data skew mitigation