Word2Vec
Word2Vec trains static word vectors with skip-gram or CBOW and negative sampling, making distributional embeddings cheap enough to run on billion-word crawls.
Definition
Skip-gram predicts context words from a center word; CBOW does the reverse. Negative sampling turns the softmax over V into a set of binary problems. The result is a lookup table, not a contextual encoder.
Why it matters
Word2Vec industrialised embeddings. It is still a strong, cheap baseline for retrieval and featureization when you do not need polysemy or sentence-level context.
Case
Google News word2vec vectors became the default features for 2014-era classifiers. They fail on sense disambiguation and on any domain whose crawl they never saw.
How to read it
Check window size, min-count, and whether OOV is handled. A desk-specific crawl often beats a famous Google News dump.
Key papers
Landmark references with real publisher or arXiv links — not token-overlap noise.
- Efficient Estimation of Word Representations in Vector SpaceMikolov, Chen, Corrado, Dean · 2013 · ICLR workshopCBOW and skip-gram.
- Distributed Representations of Words and Phrases and their CompositionalityMikolov, Sutskever, Chen, Corrado, Dean · 2013 · NeurIPSNegative sampling and phrase vectors.