JD Tech Talk
Mar 5, 2025 · Artificial Intelligence
GLM: General Language Model Pretraining with Autoregressive Blank Infilling
GLM introduces a unified pretraining framework that combines autoregressive blank‑filling with 2D positional encoding and span‑shuffle, achieving superior performance over BERT, T5 and GPT on a range of NLU and generation tasks such as SuperGLUE, text‑filling, and language modeling.
2D positional encodingNLUautoregressive
0 likes · 27 min read