Abstract: Flexible job shop scheduling problem(FJSP),as a variant of the job shop scheduling problem,has become an important research topic in the intelligent transformation of modern manufacturing industry due to the wide applicability.In recent years,deep reinforcement learning(DRL) has been applied to solve flexible job shop scheduling problems.However,the characteristic that operations can be assigned to multiple compatible machines with different processing times brings additional complexity to decision making and state representation.This paper proposes an end-to-end deep reinforcement learning framework based on an improved attention mechanism and proximal policy optimization algorithm to solve the FJSP.Considering the characteristics of heterogeneous disjunction graph structure,it designs a double-layer attention network based on hierarchical attention,including node-level attention layers and type-level attention layers,to fully extract the complex information between operations and machines to support high-quality scheduling decisions.Experimental results on synthetic and public datasets show that the proposed method outperforms traditional priority dispatching rules and currently state-of-the-art DRL methods in both of performance and generalization ability while maintaining high efficiency.

Key words: Flexible job shop scheduling problem, Deep reinforcement learning, Graph attention network, Attention mechanism