ArcCSE: Angular Margin Contrastive Learning for Self‑Supervised Text Representation

Learning high-quality text representations is a fundamental NLP task, but pretrained models such as BERT often yield suboptimal results on semantic similarity evaluations when used without fine‑tuning.

This paper proposes ArcCSE, a novel self‑supervised framework that introduces an angular margin into contrastive learning and explicitly models semantic partial‑order relations among sentences.

ArcCSE comprises pairwise and triple‑wise relationship modeling. Pairwise positive pairs are obtained via dropout‑based augmentation, while triple‑wise triplets are constructed by masking different spans of a sentence to generate entailment‑like relations. The new Angular Margin Contrastive Loss (ArcCon) replaces the conventional NT‑Xent loss, providing a tighter decision boundary and greater robustness to noise.

Extensive experiments on unsupervised STS benchmarks and SentEval transfer tasks demonstrate that ArcCSE consistently surpasses SimCSE and other state‑of‑the‑art self‑supervised methods, achieving higher alignment and uniformity scores and improving downstream classification performance.

The method has been deployed in Alibaba’s content‑understanding platforms, enhancing retrieval and recommendation in scenarios such as Taobao Live and Xianyu.