Textual embeddings with word-type-weighted word2vec and graph neural networks

Authors: T. Ladin, L. Korel, Martin Holeňa
Publication year: 2025
Publication type: Konferenční příspěvek (mezinárodní konf.)
Periodical: Proceedings of the 25th Conference Information Technologies – Applications and Theory (ITAT 2025)
Bibliographic citation: LADIN, T., KOREL, L., HOLEŇA, Martin. Textual embeddings with word-type-weighted word2vec and graph neural networks. In: CIENCIALOVÁ, L., ČECHOVÁ, M., DENTI, L., HOLEŇA, M., MRÁZ, F., OSTERTÁG, R., PARDUBSKÁ, D., PLÁTEK, M., STANEK, M., eds. Proceedings of the 25th Conference Information Technologies – Applications and Theory (ITAT 2025). Aachen: Technical University & CreateSpace Independent Publishing, 2025, S. 105-113. CEUR Workshop Proceedings, 4092. ISSN 1613-0073. Dostupné z: https://ceur-ws.org/Vol-4092/paper8.pdf
ID: 643745
Online record: View online record

Abstract

The increasing use of neural networks for semantic text analysis highlights the need for more efficient methods without compromising quality. We propose a lightweight approach combining traditional word embeddings with graph convolutional networks (GCNs) to improve sentence similarity recognition. By incorporating syntactic information, such as parts of speech and grammatical functions, our method reduces computational demands at least 2.5 times while maintaining accuracy, when tested statistically indifferent to larger models.