Word2vec (Mikolov et al. 2013) is a framework for learning word vectors
Idea:
- We have a large corpus (“body”) of text
- Every word in a fixed vocabulary is represented by a vector
- Go through each position t in the text, which has a center word c and context (“outside”) words o
- Use the similarity of the word vectors for c and o to calculate the probability of o given c (or vice versa)
- Keep adjusting the word vectors to maximize this probability
- 手推Word2Vec损失函数与梯度下降：
