Tagged articles
Language Model
56 articles · Page 1 of 1
Microsoft Research and MIT propose Next Latent Prediction Transformer (NextLat), which adds a latent state prediction loss during training to compress history into belief states, achieving 98.7% valid route generation on Manhattan taxi data, 54.8% accuracy on Countdown arithmetic, and up to 3.3x speculative decoding speedup on a 1.3B model.
Language ModelLatent DynamicsMIT
0 likes · 15 min read

Princeton researchers introduce QUEEN, a 4B-parameter model that combines a strong chess engine with a language model via gated cross-attention bridges, trained through a four-stage QA curriculum and iterative search distillation to achieve 2697 Elo while generating natural-language move explanations.
Explainable AILanguage ModelLeela Chess Zero
0 likes · 13 min read


21CTOAug 15, 2026 · Backend Development
Alex Morley‑Finch created the open‑source Marlin search engine using a $10 cloud GPU and under 1 GB of storage, crawling 560,000 homepages, employing a lightweight LLM for categorisation, and revealing challenges with noisy tags and scalability while showing a weekend‑sized, low‑cost alternative to mainstream search.
Language ModelMarlinWeb Crawling
0 likes · 6 min read

Meta’s Brain2Qwerty v2 non‑invasive brain‑computer interface now decodes whole sentences in real time with an average word‑accuracy of 61% (up to 78%), surpassing prior methods, and the paper details its Conformer‑Aligner‑LLM architecture, open‑source releases, and remaining challenges such as device size and clinical‑grade precision.
Language ModelMEGNon-invasive
0 likes · 10 min read

A new study shows that reallocating the feed‑forward network capacity toward the early layers of a Transformer—without adding parameters or FLOPs—lowers perplexity by up to 1.84 points, and the same technique improves performance across several modern LLM architectures.
FFN widthLanguage ModelPerplexity
0 likes · 9 min read


SuanNiJun 14, 2026 · Artificial Intelligence
HRM-Text-1B, a brain‑inspired hierarchical language model, achieves strong benchmark scores while using only 0.1% of the training tokens of comparable models, cutting compute costs by 96‑432× through a novel H/L module architecture, MagicNorm stabilization, and a focused instruction‑response training objective.
Efficient PretrainingHRM-TextLanguage Model
0 likes · 9 min read


AIWalkerMay 18, 2026 · Artificial Intelligence
The article analyzes ByteDance's Cola DLM, a fully open‑source continuous diffusion language model that abandons token‑centric generation in favor of latent semantic representations, detailing its architecture, training strategy, scaling stability, and how it compares with the earlier ELF model.
ByteDanceCola DLMLanguage Model
0 likes · 14 min read


CodeNotesMay 4, 2026 · Artificial Intelligence
This article explains what a token is, how it differs from characters or words, its role in AI model costs, speed, context limits, and quality, and offers practical tips for managing tokens through context engineering to control expenses and improve performance.
AIContext WindowLanguage Model
0 likes · 11 min read

The article dissects the leaked Claude Opus 4.7 system prompt, revealing ten intertwined design decisions—from treating psychological reconstruction as a danger signal to dynamic safety‑policy upgrades—that together shape the model’s self‑restraint, tool‑use, memory handling, and risk‑aware behavior.
AI safetyClaudeLanguage Model
0 likes · 8 min read

The article explains why using formal, standard language makes AI‑generated workplace messages sound robotic and presents a three‑step protocol—high‑quality phrase extraction, persona‑mapping prompts, and forbidden‑word rules—to feed the model with emotionally intelligent corpora for more natural communication.
AI Prompt EngineeringLanguage ModelPrompt Design
0 likes · 5 min read

This guide explains why AI‑generated messages often feel robotic, presents a set of prompt templates that inject emotion, relationship, and cultural context into LLM outputs, and offers a risk‑assessment checklist to ensure safe, high‑impact workplace communication.
AILanguage ModelPrompt engineering
0 likes · 6 min read

A recent paper from Sapienza University's GLADIA Lab shows that mainstream Transformer language models are injective, enabling a novel SIPIT algorithm to recover original text from hidden states with perfect accuracy, while extensive experiments confirm the models retain all input information.
InjectiveInvertibilityLanguage Model
0 likes · 11 min read

The article explains that high‑quality prompts are essential for efficient AI output and presents practical, step‑by‑step techniques—defining roles, supplying context and format, setting tone, breaking tasks into outlines, iterating with feedback, and giving examples—to help anyone harness AI as a collaborative partner.
AILanguage ModelProductivity
0 likes · 4 min read

A Minecraft developer built CraftGPT, a 5‑million‑parameter language model that runs entirely on Redstone circuits, demonstrating how the game’s Turing‑complete logic system can implement a transformer‑style AI with billions of in‑game blocks.
AIGame ComputingLanguage Model
0 likes · 9 min read

DeepSeek’s latest V3.1 model unexpectedly injects the Chinese character “极” into generated text, a token‑ID mix‑up that breaks code compilation, JSON parsing, and academic writing, with users tracing the issue to adjacent token IDs and two main hypotheses of dataset contamination or model shortcut.
AI safetyDeepSeekLanguage Model
0 likes · 4 min read

Google’s newly released Gemma 3 270M is a compact 270‑million‑parameter language model that combines a large token vocabulary, energy‑efficient INT4 quantization, strong instruction‑following, and production‑ready checkpoints, making it ideal for fine‑tuning, on‑device deployment, and a wide range of low‑latency AI tasks.
Gemma 3Google AILanguage Model
0 likes · 7 min read

This article explains the fundamental distinctions between Causal Language Models and Prefix Language Models, detailing their definitions, attention‑mask designs, underlying design philosophies, and practical scenarios where each architecture excels.
AIAttention MaskCausal LM
0 likes · 7 min read

GLM introduces a unified pretraining framework that combines autoregressive blank‑filling with 2D positional encoding and span‑shuffle, achieving superior performance over BERT, T5 and GPT on a range of NLU and generation tasks such as SuperGLUE, text‑filling, and language modeling.
2D positional encodingGLMLanguage Model
0 likes · 27 min read

GLM introduces a universal language model that combines autoregressive blank‑filling with 2D positional encoding and span‑shuffle training, achieving superior performance over BERT, T5, and GPT across NLU, conditional and unconditional generation tasks, as demonstrated on SuperGLUE and other benchmarks.
Language ModelNLUPretraining
0 likes · 29 min read

The article explains perplexity as a metric for evaluating large language models, walks through a step‑by‑step probability calculation for a sample sentence, shows how to normalize by sentence length using the geometric mean, and demonstrates that lower perplexity indicates a more accurate and less uncertain model.
AILanguage ModelPerplexity
0 likes · 6 min read

This guide presents seven practical prompt‑engineering techniques—clear goals, structured queries, domain terminology, concrete examples, scoped questions, step‑by‑step breakdowns, and multi‑turn interactions—to help users get more accurate and useful answers from DeepSeek.
AI promptsDeepSeekEffective Questioning
0 likes · 6 min read

The article introduces two 360 AI Research Institute projects—IAA, an architecture that equips frozen language models with multimodal capabilities via plug‑in layers, and BDM, a Chinese‑native diffusion model compatible with the Stable Diffusion ecosystem—detailing their motivations, designs, benchmark results, and open‑source resources.
Chinese AI paintingLanguage ModelStable Diffusion
0 likes · 6 min read

The article traces the development of GPT series—from the 2017 Transformer breakthrough, through GPT‑1, GPT‑2, and GPT‑3’s 175 billion parameters, to later models like Codex and ChatGPT—highlighting key papers, architectural choices, and the surprising role of OpenAI’s decoder‑only approach.
GPT-3GoogleLanguage Model
0 likes · 4 min read

This step‑by‑step guide shows how to use Hugging Face's Trainer API to pre‑train an ultra‑small LLaMA‑3 model (under 20 M parameters) on the TinyStories dataset, covering model configuration, tokenizer setup, data preprocessing, collators, training arguments, and inference results.
Hugging FaceLanguage ModelLlama
0 likes · 27 min read

This guide presents 28 practical ChatGPT prompt strategies—from role‑playing experts and setting response length to crafting resumes, weekly reports, and product PRDs—helping readers boost productivity, creativity, and learning across personal and professional tasks.
AI ProductivityChatGPTLanguage Model
0 likes · 33 min read


21CTOFeb 2, 2024 · Artificial Intelligence
Recent tech headlines reveal WeChat’s iOS app ballooning to 712 MB, China’s third‑generation superconducting quantum computer “Wukong” surpassing one million remote accesses, AI2 unveiling the open‑source OLMo language model, and Google planning to retire the Bard brand in favor of Gemini, highlighting rapid shifts across mobile, quantum, and AI domains.
GoogleLanguage ModelOpen-source AI
0 likes · 7 min read

This article details HuoLaLa's development of an in‑house Automatic Speech Recognition system, covering its architecture, VAD optimization, language‑model and hot‑word enhancements, punctuation restoration, task and resource scheduling, and the resulting improvements in accuracy and cost efficiency.
ASRLanguage ModelSpeech Recognition
0 likes · 18 min read

This article reviews recent research on augmenting language and multimodal models with external knowledge sources and tool‑calling mechanisms, covering three systems—OREO‑LM for knowledge‑graph reasoning, REVEAL for multi‑source visual‑language pretraining, and AVIS for dynamic tool selection—and their experimental results and implications.
Language Modelknowledge graphmultimodal
0 likes · 28 min read

This article reviews recent research on augmenting language and vision models by incorporating external knowledge sources such as knowledge graphs, multi‑source retrieval, and dynamic tool‑calling frameworks, presenting three systems—OREO‑LM, REVEAL, and AVIS—and their experimental results.
AI researchLanguage Modelmultimodal
0 likes · 27 min read

This article reviews the evolution of game AI agents, examines how large language models (LLMs) can drive new AI behaviors in games, and discusses practical case studies across genres such as Werewolf‑style, war‑SLG, and MOBA games, concluding with challenges and future research directions.
AI agentsLLMLanguage Model
0 likes · 31 min read

This article explains the inner workings of ChatGPT, covering how large language models predict the next token using probability distributions, the role of embeddings, the transformer architecture with attention heads, training methods, loss functions, and why such a massive neural network can produce coherent, human‑like language.
ChatGPTLanguage ModelTransformer
0 likes · 79 min read


58 TechJun 21, 2023 · Artificial Intelligence
This article explains the design, implementation, and experimental evaluation of hot‑word augmentation in WeNet's GPU runtime, detailing how character‑ and word‑based language model scoring are extended to boost recognition of rare proper nouns in both streaming and non‑streaming ASR services.
ASRCTC decoderGPU
0 likes · 12 min read

This article explains what ChatGPT is, how it builds on natural-language-processing and the Transformer-based GPT series, details its model-size growth, architectural enhancements, multilingual support, and walks through the tokenization-to-generation pipeline that enables coherent AI-driven conversations.
ChatGPTGPT-3Language Model
0 likes · 8 min read


21CTOApr 16, 2023 · Artificial Intelligence
In this talk, Wu Jun explains the decades‑long evolution of language models, why ChatGPT sparked hype yet isn’t a breakthrough, how massive compute and data power it, and what practical effects it has on creators, energy use, and the tech industry.
AI historyChatGPTLanguage Model
0 likes · 20 min read

In a live talk, AI veteran Wu Jun explains why the hype around ChatGPT is overblown, traces the history of language models from the 1970s, details the massive compute and data requirements, and discusses the real impact of large‑scale AI on society and work.
AI hypeChatGPTLanguage Model
0 likes · 20 min read

In this talk, AI expert Wu Jun explains why ChatGPT has caused widespread fear, traces the historical development of language models from the 1970s to today, clarifies the massive computational and data requirements, and discusses the real impact and opportunities of large‑scale AI systems.
AI hypeChatGPTLanguage Model
0 likes · 20 min read

This article provides a comprehensive overview of how ChatGPT works, covering its probabilistic text generation, transformer architecture, embedding representations, neural network training processes, and the underlying principles that enable large language models to produce coherent and meaningful human-like language.
AIChatGPTLanguage Model
0 likes · 80 min read

This article demonstrates how to use Reinforcement Learning from Human Feedback (RLHF) with a PPO algorithm and a sentiment‑analysis model to train a language model that generates positive product reviews, covering task definition, data sampling, reward evaluation, model optimization, and experimental results.
GPTLanguage ModelPPO
0 likes · 11 min read


ArchitectFeb 19, 2023 · Artificial Intelligence
This article demonstrates how to apply Reinforcement Learning from Human Feedback (RLHF) using a sentiment‑analysis model as a reward function and Proximal Policy Optimization (PPO) to fine‑tune a language model that generates positive product reviews, complete with code snippets and experimental results.
Language ModelPPORLHF
0 likes · 10 min read

ChatGPT, a dialogue-focused language model, builds on the GPT family and employs techniques such as Reinforcement Learning from Human Feedback (RLHF), the TAMER framework, and a three-stage training pipeline (supervised fine‑tuning, reward modeling, and PPO reinforcement learning) to achieve advanced conversational capabilities.
ChatGPTGPTLanguage Model
0 likes · 7 min read

ChatGPT, OpenAI’s conversational AI launched in December 2022, builds on GPT‑3 and advanced training methods like supervised fine‑tuning and reinforcement learning from human feedback, offering versatile applications from search assistance to code generation, while also revealing notable limitations and future commercial prospects.
AIApplicationsChatGPT
0 likes · 17 min read

Prof. Qiu Xipeng’s talk highlighted how large language models can be offered as a service and efficiently adapted via in‑context learning, lightweight label‑tuning, and gradient‑free black‑box optimization, showcasing a unified asymmetric Transformer (CPT) that handles understanding, generation, ABSA and NER tasks while reducing resource demands.
Black-Box OptimizationLLMLanguage Model
0 likes · 15 min read

This article provides a comprehensive overview of natural language processing fundamentals, covering the challenges of language modeling, N‑gram and Markov assumptions, smoothing techniques such as discounting and add‑one, evaluation via perplexity, basic tasks like Chinese word segmentation, subword tokenization, POS tagging, syntactic and semantic parsing, and a range of downstream applications including information extraction, sentiment analysis, question answering, machine translation, and dialogue systems.
AILanguage ModelNLP
0 likes · 29 min read


58 TechDec 11, 2020 · Artificial Intelligence
This article explains the role of Weighted Finite State Transducers in conventional HMM‑based speech recognition, covering language models, pronunciation dictionaries, WFST definitions, semiring theory, composition and determinization operations, decoding graph construction (HCLG), lattice rescoring, and practical optimization techniques for real‑world scenarios.
ASRLanguage ModelSpeech Recognition
0 likes · 23 min read

This article provides a comprehensive, illustrated walkthrough of OpenAI's GPT‑2 language model, covering its decoder‑only Transformer architecture, self‑attention mechanisms, token processing, training data, differences from BERT, and applications beyond language modeling, enriched with visual diagrams and code snippets for deeper understanding.
AIGPT-2Language Model
0 likes · 24 min read


Didi TechNov 5, 2020 · Artificial Intelligence
DiDi’s self‑learning ASR platform lets non‑technical users upload business data, automatically train, test and deploy models with semi‑supervised learning, hot‑word updates and LSTM rescoring, creating a closed‑loop pipeline that boosted vehicle voice‑interaction accuracy from around 80 % to over 95 % within months.
AILanguage Modelacoustic model
0 likes · 14 min read


58 TechMar 2, 2020 · Artificial Intelligence
This article proposes a method to identify low-quality text in business data by training a large-scale unsupervised language model to compute sentence perplexity, converting the detection problem into a threshold decision, and details model design, challenges, optimizations, and online performance results.
BERTLanguage ModelNLP
0 likes · 13 min read

This article excerpt explains the structure and training of feed‑forward neural networks, illustrates their use in neural language models, describes deep structured semantic models for ranking tasks, and details two‑stage recommendation systems such as YouTube, covering both theoretical formulas and practical deployment considerations.
Language ModelRankingartificial intelligence
0 likes · 13 min read

This article presents a distributed system that efficiently supports large‑scale n‑gram language models for automatic speech recognition by introducing caching, a two‑level distributed index, batch processing, and a cascading fault‑tolerance mechanism, demonstrating robust scalability and low communication overhead in Tencent's WeChat ASR service.
CachingLanguage ModelN-gram
0 likes · 35 min read

Intelligent speech recognition converts spoken audio to text using a pipeline of feature extraction, acoustic and language modeling, where deep neural networks—especially CNN, LSTM, and hybrid CLDNN architectures—drive high accuracy, enabling mobile voice input, call‑center transcription, legal record keeping, and Tencent Cloud ASR’s 97% Mandarin accuracy with speaker separation and on‑premises deployment.
AILanguage ModelSpeech Recognition
0 likes · 7 min read

The article outlines the design and implementation of a multi‑turn conversational chatbot, detailing how natural‑language understanding converts user utterances into structured representations, a CNN‑LSTM language model classifies topics, intents, and sentiments, and an XML‑based answer engine orchestrates tasks and services for real‑world deployment.
AILanguage Modelchatbot
0 likes · 9 min read


58 TechJun 27, 2019 · Artificial Intelligence
This article describes the design and implementation of a spelling‑correction module for 58.com’s search engine, covering common query errors, rule‑based and statistical language‑model approaches, offline dictionary generation, n‑gram and Viterbi decoding, online workflow, and practical examples.
Language ModelQuery ProcessingViterbi algorithm
0 likes · 15 min read


58 TechFeb 20, 2019 · Artificial Intelligence
This article explains the concepts, training pipeline, deployment formats, and practical applications of language models—particularly LSTM‑based models—for evaluating and generating text quality in a real‑world rental listing platform, highlighting data preparation, model training, and online serving techniques.
LSTMLanguage ModelPerplexity
0 likes · 16 min read


58 TechJan 22, 2019 · Artificial Intelligence
The article explains why Chinese word segmentation is essential for NLP tasks, outlines its fundamental difficulties such as ambiguity and out‑of‑vocabulary words, reviews dictionary‑based, statistical, and CRF approaches, and shares practical experiences from 58 Search’s production system.
CRFLanguage ModelNLP
0 likes · 21 min read

iQIYI HomeAI tackles the bottleneck of static language models in voice assistants by separating phonetic and semantic processing, correcting ASR errors at the intent‑recognition layer with pinyin‑enhanced entity correction, thereby reducing error amplification in video‑on‑demand interactions and paving the way for adaptive, personalized voice experiences.
AILanguage ModelSpeech Recognition
0 likes · 7 min read

This article explains the fundamentals of recurrent neural networks (RNNs), their role in language modeling, various RNN architectures such as bidirectional and deep RNNs, the back‑propagation through time (BPTT) training algorithm, gradient challenges, vectorization techniques, and provides a step‑by‑step code implementation.
BPTTLanguage ModelRNN
0 likes · 21 min read
