Semantic Embedding with Large Language Models: A Comprehensive Survey

This survey reviews the evolution of semantic embedding—from Word2vec and GloVe to BERT, Sentence‑BERT, and recent contrastive methods—then examines how large language models improve embeddings via synthetic data generation and backbone architectures, detailing techniques such as contrastive prompting, in‑context learning, knowledge distillation, and discussing resource, privacy, and interpretability challenges.

In-Context LearningNLPcontrastive learning

0 likes · 27 min read

Semantic Embedding with Large Language Models: A Comprehensive Survey

Architects' Tech Alliance

Sep 4, 2024 · Fundamentals

Why Bigger Transformers Win: Scaling Laws and Parallel Computing Essentials

The article explains OpenAI's 2020 Scaling Laws that show larger transformer models, more data, and greater compute consistently improve performance, introduces the concept of emergent abilities at critical size thresholds, and outlines the core principles of parallel computing such as multi‑processor usage, task decomposition, concurrent execution, and inter‑processor communication.

communicationconcurrencyemergent abilities

0 likes · 6 min read

Why Bigger Transformers Win: Scaling Laws and Parallel Computing Essentials

Kuaishou Tech

Dec 20, 2023 · Artificial Intelligence

SAMP: A Model Inference Toolkit of Post-Training Quantization for Text Processing via Self-Adaptive Mixed-Precision

SAMP is an adaptive mixed-precision inference toolkit that automatically controls floating-point and integer operations to accelerate model inference while maintaining computational accuracy.

AI inferenceNLP accelerationmixed-precision computing

0 likes · 9 min read

SAMP: A Model Inference Toolkit of Post-Training Quantization for Text Processing via Self-Adaptive Mixed-Precision

Baidu Geek Talk

Nov 9, 2023 · Artificial Intelligence

Deep Learning Model Architecture Evolution in Baidu Search

The article chronicles Baidu Search’s Model Architecture Group’s evolution of deep‑learning‑driven search, detailing the shift from inverted‑index to semantic vector indexing, the use of transformer‑based models for text and image queries, large‑scale offline/online pipelines, and extensive GPU‑centric optimizations such as pruning, quantization and distillation, all aimed at delivering precise, cost‑effective results to hundreds of millions of users.

ErnieGPU inferenceModel Optimization

0 likes · 14 min read

Deep Learning Model Architecture Evolution in Baidu Search

Kuaishou Tech

Oct 20, 2021 · Artificial Intelligence

HiT: Hierarchical Transformer with Momentum Contrast for Video-Text Retrieval

This paper proposes HiT, a hierarchical transformer model with momentum contrast for video-text retrieval, addressing limitations in existing multimodal learning methods by introducing hierarchical cross-modal contrast matching and momentum cross-modal contrast to improve retrieval performance.

HCMMCCMoCo

0 likes · 9 min read

HiT: Hierarchical Transformer with Momentum Contrast for Video-Text Retrieval