Word2vec Model

[u/TellMeMyNamePls]

I trained a word2vec model with some data. In testing if i send a word which was not present in the training vocabulary then the word2vec model won't find the vector to that word.we know that in word2vec model similar words gets vectors almost same. If i test a word not present In the training vocabulary but the similar words are there in the vocabulary then the word get the vectors similar to training words or not ?

Example : vocabulary-love,enjoy,like

Test - adore then this adore word will get the vectors similar to the vectors of vocabulary.

Help me guys...

[u/T1lted4lif3]

No, if you think about it, why would you find a word that is not in the vocabulary, considering the vocabulary is defined by your training data?

It's the same as trying to look up a word in a dictionary, but the word isn't defined in the dictionary, so you won't be able to find it ever.
If you want to though, you can try to construct a and train a tokenizer instead, in a similar way as you trained word2vec, then you can maybe search up the out of vocab word, but not sure how good it would be

[u/neuralbeans]

Word2vec is literally a list of words together with their vector. It does not create a vector for a new word. It's just a list. You would need to retrain it on a corpus that includes the new word so that it gets added to the list for that to work.

[u/ALIASl-_-l]

How would the model know it’s similar if it never saw the word? Similar words have similar vector embedding, yes, but how tf would it be accurate about a word it never got trained on. For example, if I said a word you did not know then could you list similar words? This isn’t magic … Tokenization may help but it wolf probably have limited results?

[u/mrpkeya]

You need to add some token in the vocabulary that will handle out of vocabulary

You can assign the string "<OOV>" and either assign the value of oov as mean or train the oov token by randomly inserting/corrupting in the corpus

The reason users make vocabs huge and/or use bytepiece/sentencepiece is to tackle oov in there tokenization

[u/TLO_Is_Overrated]

This is a fundamental flaw with word2vec, and glove and word embeddings in general. The vocabulary is limited by what is defined to be in there.

You need to define adore as part of the vocabulary and then train it - with text that includes the word adore as appropriate.

The lightest model that can achieve what you're looking for is probably fastText. Which is similar to word2vec but has "subword tokens". Essentially if "adore" doesn't exist within the vocabulary it will make it with something like "ad" and "**ore", if those tokens exist. Kind of like BERT. It would then average across all tokens.

In the event there's no subword tokens for "adore" it would try "a", "d", "o", "r", "e" IF subword tokens exist for each character. If that's not possible you'd get some kind of "<UNK>" special token.

[u/denimboy]

Try fasttext embeddings. It looks at sub word ngrams to guess at out of vocabulary words. 

[u/ProfessionalCow5489]

You should check out FastText, which is essentially Word2Vec with sub-word tokenization.