Graph representation learning via enhanced GNNs and transformers

In recent years, graph transformers (GTs) have captured increasing attention within the graph domain. To address the prevalent deficiencies in local feature learning and edge information utilization inherent to GTs, we propose EHDGT, a novel graph representation learning method based on enhanced graph neural networks (GNNs) and Transformers. Initially, considering that edges encapsulate structural information, we enhance the original graph by superimposing edge-level positional encoding based on node-level random walk positional encoding, thus optimizing the utilization of this information. Subsequently, enhancements are applied to both GNNs and Transformers: for GNNs, we employ encoding strategies on subgraphs of the original graph, thereby augmenting their proficiency to process local information. For Transformers, we incorporate edges into the attention calculation and introduce a linear attention mechanism, significantly reducing the model's complexity. Ultimately, to exploit the synergies between GNNs and Transformers while maintaining a balance between local and global features, we propose a gate-based fusion mechanism for the dynamic integration of their outputs. Experimental results across multiple datasets demonstrate that EHDGT significantly outperforms traditional message-passing networks and achieves strong performance compared to several existing GTs. This paper further explores the application of the proposed model to improving the quality of the wine industry knowledge graph, experiments show that using link prediction as a downstream task of graph representation learning based on Graph Transformer achieves excellent results, significantly enhancing the completeness and semantic quality of the wine industry knowledge graph, thereby increasing its practical value in digitalization of the wine industry.