Abstract: GNNs have demonstrated strong performance in graph representation learning due to their message-passing mechanism.However,they encounter challenges such as over-smoothing and insufficient capture of multi-hop neighbor information when processing heterogeneous graphs.The GNN-Transformer collaborative contrastive learning framework(GTC) combines the local information aggregation capability of GNNs with the global information modeling capability of Transformers.This framework implements self-supervised heterogeneous graph representation learning through cross-view contrastive learning,effectively mitigating the over-smoothing problem that GNNs experience during neighbor information aggregation.This study enhances the GNN branch of the GNN-Transformer collaborative contrastive learning framework by incorporating the node-level and semantic-level attention mechanisms from the heterogeneous graph attention network(HAN).This optimization enables more effective capture of information from different node and edge types in heterogeneous graphs during neighbor aggregation.Experiments on the ACM dataset demonstrate that the improved GNN-Transformer collaborative contrastive learning framework achieves superior performance in node classification and node clustering tasks.For node classification,with 20,40,and 60 labeled nodes,the mo-del exhibits average improvements of 0.52%,3.07%,and 3.14% in AUC,macro-F1,and micro-F1 scores,respectively.In node clustering,normalized mutual information(NMI) and adjusted rand index(ARI) increase by 6.57% and 6.38%,respectively.These results confirm that HAN's hierarchical attention mechanism enables finer neighbor aggregation and metapath semantic fusion in heterogeneous graph representation learning,offering a novel approach to alleviating the over-smoothing problem in GNNs.

Key words: Graph neural networks, Heterogeneous graphs, Contrastive learning, Attention mechanism, Self-supervised