Abstract: Offline tasks are delay-tolerant workloads without strict requirements on completion time,typically including batch processing or machine learning tasks.With the development of deep learning technology,deep learning tasks have become one of the important parts of offline workloads in cloud data centers.Accurate runtime prediction of offline tasks improves resource utilization during idle periods of online tasks.However,deep learning models exhibit diverse architectures and vast scale differences.Factors such as batch sizes,hyperparameters and operator characteristics during training also significantly affect task execution time.Existing methods struggle to comprehensively account for all these factors:configuration-based methods ignore the internal execution mechanism of the algorithm;operator-based methods neglect the impact of computation graph structure;graph-based methods either face excessive model complexity with graph neural networks or lose dependency information when simplifying to topological sequences.In view of the deficiencies of the topological sequence methods,this paper proposes the MDOT(Multi-dimensional Operator Transformer) algorithm to convert the computational graph into an operator sequence according to topological sorting.Based on this sequence of operators,MDOT uses Transformer to fuse the three-dimensional information of the operators:operator type,operator configuration,and computational load to perform multi-dimensional operator encoding,more comprehensively modeling the execution characteristics of the operators.Secondly,in order to capture the dependencies of the computational graph,MDOT designs a graph position encoding mechanism,which captures the relationships between operator sequences through the self-attention of the Transformer and models the mutual influence of operators in terms of running time.Experimental results show that MDOT outperforms existing methods in predicting the training time of deep learning tasks,with the mean absolute error and root mean square error being 25% and 45% lower than those of suboptimal models,respectively.

Key words: Cloud computing, Execution time prediction, Deep learning, Operator, Offline task