Textual embeddings with word-type-weighted word2vec
Abstract
The increasing use of artificial neural networks for knowledge processing often lacks precise knowledge representation. To address this issue, we propose using a word-type-weighted Word2Vec model to achieve more accurate representations of individual words within sentences. Our approach incorporates weighting vector embeddings of words based on parts-of-speech predictions generated by the spaCy library. Experimental results demonstrate that, compared to simple Word2Vec, our model enhances the accuracy of recognizing the semantics of a sentence, while maintaining significantly lower computational requirements than large language models and various variants of Transformer.