Abstract: In offline Markov decision process sequences,the definition of state similarity affects empirical reachability modeling.Geometric,temporal,and local-dynamics distances focus on proximity in observation space,temporal adjacency,or short-horizon transition differences,respectively,and may not jointly capture empirical reaching difficulty and similarity of future evolution.A framework combining state-level and distribution-level analysis is developed:the isolation kernel(IK) is used for state retrieval,offline k-nearest-neighbor goal relabeling,and graph planning,while the isolation distributional kernel(IDK) compares H-step successor-state sets.Local ranking,goal relabeling,graph planning,and H-step successor-region matching are evaluated on PointMaze-Umaze-v2,PointMaze-Large-v2,and AntMaze-Umaze-diverse-v1 from D4RL.IK achieves the highest Spearman correlations in the PointMaze-Large and AntMaze relabeling tasks,whereas Mahalanobis distance performs better on PointMaze-Umaze,indicating that the advantage is task dependent.At H=10,IDK achieves AUROC values of 0.9734,0.9938,and 0.9955 on the three datasets;compared with the order-sensitive raw H-step distance,the difference is small on PointMaze-Umaze and more pronounced on AntMaze.Sensitivity analyses show that performance depends on the representation,partition scale,future-window length,and planning acceptance threshold.Repeated random partitions reduce estimation variability conditional on the offline state pool but do not remove coverage bias;mean embeddings are permutation invariant and cannot preserve the full visitation order.The planning results are metric-level diagnostics and do not establish improved end-to-end offline-policy return.The empirical findings in this study mainly apply to maze environments with explicit geometric bottlenecks and usable trajectory coverage.

Key words: Isolation kernel, Isolation distributional kernel, Markov decision processes, Reachability, Offline reinforcement lear-ning, State similarity