Introducing C2LLM, a contrastive code LLM series that replaces mean and EOS pooling with a multi‑head attention pooling module, achieving top scores on the MTEB‑Code benchmark across 12 tasks and demonstrating cost‑effective, high‑precision code retrieval for both production and AI agent applications.
The article introduces three BGE vector models—BGE‑Code‑v1, BGE‑VL‑v1.5, and BGE‑VL‑Screenshot—detailing their architectures, open‑source resources, benchmark results on CoIR, Code‑RAG, MMEB, and MVRB, and their impact on code and multimodal retrieval research.
Fast.ai's Kwaipilot team unveiled OASIS, a 1.3B‑parameter code‑embedding model that, using only 5 million tokens, outperforms larger OpenAI embeddings across CodeSearchNet, CoSQA and AdvTest benchmarks, thanks to repository‑level program analysis, synthetic data generation, and a fused loss function.
The article analyzes D2LLM’s teacher‑student bi‑encoder architecture and Codefuse‑CGE’s PMA‑enhanced code embedding, showing how both models surpass BERT dual encoders and LLM cross‑encoders in accuracy, efficiency, and storage cost across semantic and code search benchmarks.
