Contrastive Learning
Contrastive learning pulls representations of related pairs together and pushes unrelated pairs apart. It is the pretraining idea behind SimCLR, CLIP, and many embedding models.
Definition
Given an anchor, a positive, and negatives, a contrastive loss (InfoNCE, triplet) raises the similarity of the positive relative to the rest. Positives can be two views of an image, a caption–image pair, or a query–doc pair.
Why it matters
This is how modern retrieval embeddings are trained. If your “semantic search” quality is poor, the contrastive pairs and negatives are the first place to look — not the ANN index.
Case
SimCLR learned ImageNet features without labels by agreeing two augmentations of the same image. CLIP did the same across image–text pairs and unlocked zero-shot classification.
How to read it
Ask what a positive pair is and where negatives come from. In-batch negatives are convenient and biased. Hard-negative mining changes the product.
Key papers
Landmark references with real publisher or arXiv links — not token-overlap noise.
- A Simple Framework for Contrastive Learning of Visual RepresentationsChen, Kornblith, Norouzi, Hinton · 2020 · ICMLSimCLR.