Abstract: Causal effect estimation aims to calculate the magnitude of the causal effect of the treatment variable on the outcome variable.The existing prevalent causal effect estimation methods are mainly applicable to static data or a single time point in time series,and cannot effectively estimate the cumulative impact of the treatment variable on the outcome variable over a long period of time.To solve this problem,the long-term causal effect estimation method based on traditional reinforcement learning fits the long-term potential outcomes through linear basis functions,thereby calculating the long-term causal effect.However,due to the limited expressive power of linear basis functions in complex scenarios,existing methods cannot accurately identify weak causal effects,and at the same time,there will be significant performance degradation problems when the data dimension increases.In response to the above problems,this paper proposes a long-term causal effect estimation method based on deep reinforcement lear-ning.This method uses the dueling network to estimate long-term potential outcomes,which can effectively estimate the impact of the treatment variable on the outcome variable,thereby greatly improving the algorithm’s ability to identify weak causal effects.Meanwhile,the proposed method avoids the biases that occur when estimating long-term potential outcomes due to improper selection of basis functions.Experimental results show that the proposed method outperforms existing algorithms on statistical synthetic datasets and order scheduling simulation datasets.

Key words: Long-term causal effect estimation, Potential outcome model, Deep reinforcement learning