Entity Linking in Real-World News Streams: An End-to-End Deep Learning Approach for Disambiguating Named Entities Across Heterogeneous Sources

The process of disambiguating named entities in the varied and unstructured news text is a major challenge in natural language understanding. EL is a daunting task due to the constantly changing content, the mixed-language source of real-world news streams, and the contextual ambiguity of realworld news streams. This paper proposes a novel end-to-end DL framework, SA-ESF (Symmetrical Attention-based Entity Linking using Structural Features), designed specifically to address these complexities. The model exploits symmetrical BiLSTM networks with two attention mechanisms to learn bidirectional semantic relevance between mentions and candidate entities. In addition, it has dense structural features (such as entity descriptions and contextual embeddings) and uses prior link probabilities estimated on Wikipedia to improve disambiguation accuracy. SA-ESF is trained with contrastive learning and positive and hard negative samples, which makes it robust to cross-source and cross-language evaluation. For benchmark datasets, experiments demonstrate that SA-ESF outperforms the state-of-the-art models with the average F1score of 89.7, accuracy of 91.2%, and precision of 90.3%. The results indicate the superior performance, strength and scale of the model in resolving complicated entity disambiguation problems in dynamic, multilingual and heterogeneous news settings.