The article dissects the leaked Claude Opus 4.7 system prompt, revealing ten intertwined design decisions—from treating psychological reconstruction as a danger signal to dynamic safety‑policy upgrades—that together shape the model’s self‑restraint, tool‑use, memory handling, and risk‑aware behavior.
The article explains why using formal, standard language makes AI‑generated workplace messages sound robotic and presents a three‑step protocol—high‑quality phrase extraction, persona‑mapping prompts, and forbidden‑word rules—to feed the model with emotionally intelligent corpora for more natural communication.
This guide explains why AI‑generated messages often feel robotic, presents a set of prompt templates that inject emotion, relationship, and cultural context into LLM outputs, and offers a risk‑assessment checklist to ensure safe, high‑impact workplace communication.
A recent paper from Sapienza University's GLADIA Lab shows that mainstream Transformer language models are injective, enabling a novel SIPIT algorithm to recover original text from hidden states with perfect accuracy, while extensive experiments confirm the models retain all input information.
A Minecraft developer built CraftGPT, a 5‑million‑parameter language model that runs entirely on Redstone circuits, demonstrating how the game’s Turing‑complete logic system can implement a transformer‑style AI with billions of in‑game blocks.
DeepSeek’s latest V3.1 model unexpectedly injects the Chinese character “极” into generated text, a token‑ID mix‑up that breaks code compilation, JSON parsing, and academic writing, with users tracing the issue to adjacent token IDs and two main hypotheses of dataset contamination or model shortcut.
Google’s newly released Gemma 3 270M is a compact 270‑million‑parameter language model that combines a large token vocabulary, energy‑efficient INT4 quantization, strong instruction‑following, and production‑ready checkpoints, making it ideal for fine‑tuning, on‑device deployment, and a wide range of low‑latency AI tasks.
This article explains the fundamental distinctions between Causal Language Models and Prefix Language Models, detailing their definitions, attention‑mask designs, underlying design philosophies, and practical scenarios where each architecture excels.
GLM introduces a unified pretraining framework that combines autoregressive blank‑filling with 2D positional encoding and span‑shuffle, achieving superior performance over BERT, T5 and GPT on a range of NLU and generation tasks such as SuperGLUE, text‑filling, and language modeling.
GLM introduces a universal language model that combines autoregressive blank‑filling with 2D positional encoding and span‑shuffle training, achieving superior performance over BERT, T5, and GPT across NLU, conditional and unconditional generation tasks, as demonstrated on SuperGLUE and other benchmarks.
The article explains perplexity as a metric for evaluating large language models, walks through a step‑by‑step probability calculation for a sample sentence, shows how to normalize by sentence length using the geometric mean, and demonstrates that lower perplexity indicates a more accurate and less uncertain model.
This guide presents seven practical prompt‑engineering techniques—clear goals, structured queries, domain terminology, concrete examples, scoped questions, step‑by‑step breakdowns, and multi‑turn interactions—to help users get more accurate and useful answers from DeepSeek.
The article introduces two 360 AI Research Institute projects—IAA, an architecture that equips frozen language models with multimodal capabilities via plug‑in layers, and BDM, a Chinese‑native diffusion model compatible with the Stable Diffusion ecosystem—detailing their motivations, designs, benchmark results, and open‑source resources.
The article traces the development of GPT series—from the 2017 Transformer breakthrough, through GPT‑1, GPT‑2, and GPT‑3’s 175 billion parameters, to later models like Codex and ChatGPT—highlighting key papers, architectural choices, and the surprising role of OpenAI’s decoder‑only approach.
This step‑by‑step guide shows how to use Hugging Face's Trainer API to pre‑train an ultra‑small LLaMA‑3 model (under 20 M parameters) on the TinyStories dataset, covering model configuration, tokenizer setup, data preprocessing, collators, training arguments, and inference results.
This guide presents 28 practical ChatGPT prompt strategies—from role‑playing experts and setting response length to crafting resumes, weekly reports, and product PRDs—helping readers boost productivity, creativity, and learning across personal and professional tasks.
Recent tech headlines reveal WeChat’s iOS app ballooning to 712 MB, China’s third‑generation superconducting quantum computer “Wukong” surpassing one million remote accesses, AI2 unveiling the open‑source OLMo language model, and Google planning to retire the Bard brand in favor of Gemini, highlighting rapid shifts across mobile, quantum, and AI domains.
This article details HuoLaLa's development of an in‑house Automatic Speech Recognition system, covering its architecture, VAD optimization, language‑model and hot‑word enhancements, punctuation restoration, task and resource scheduling, and the resulting improvements in accuracy and cost efficiency.
This article reviews recent research on augmenting language and multimodal models with external knowledge sources and tool‑calling mechanisms, covering three systems—OREO‑LM for knowledge‑graph reasoning, REVEAL for multi‑source visual‑language pretraining, and AVIS for dynamic tool selection—and their experimental results and implications.
This article reviews recent research on augmenting language and vision models by incorporating external knowledge sources such as knowledge graphs, multi‑source retrieval, and dynamic tool‑calling frameworks, presenting three systems—OREO‑LM, REVEAL, and AVIS—and their experimental results.
This article reviews the evolution of game AI agents, examines how large language models (LLMs) can drive new AI behaviors in games, and discusses practical case studies across genres such as Werewolf‑style, war‑SLG, and MOBA games, concluding with challenges and future research directions.
This article explains the inner workings of ChatGPT, covering how large language models predict the next token using probability distributions, the role of embeddings, the transformer architecture with attention heads, training methods, loss functions, and why such a massive neural network can produce coherent, human‑like language.
This article explains the design, implementation, and experimental evaluation of hot‑word augmentation in WeNet's GPU runtime, detailing how character‑ and word‑based language model scoring are extended to boost recognition of rare proper nouns in both streaming and non‑streaming ASR services.
This article explains what ChatGPT is, how it builds on natural-language-processing and the Transformer-based GPT series, details its model-size growth, architectural enhancements, multilingual support, and walks through the tokenization-to-generation pipeline that enables coherent AI-driven conversations.
In this talk, Wu Jun explains the decades‑long evolution of language models, why ChatGPT sparked hype yet isn’t a breakthrough, how massive compute and data power it, and what practical effects it has on creators, energy use, and the tech industry.
In a live talk, AI veteran Wu Jun explains why the hype around ChatGPT is overblown, traces the history of language models from the 1970s, details the massive compute and data requirements, and discusses the real impact of large‑scale AI on society and work.
In this talk, AI expert Wu Jun explains why ChatGPT has caused widespread fear, traces the historical development of language models from the 1970s to today, clarifies the massive computational and data requirements, and discusses the real impact and opportunities of large‑scale AI systems.
This article provides a comprehensive overview of how ChatGPT works, covering its probabilistic text generation, transformer architecture, embedding representations, neural network training processes, and the underlying principles that enable large language models to produce coherent and meaningful human-like language.
This article demonstrates how to use Reinforcement Learning from Human Feedback (RLHF) with a PPO algorithm and a sentiment‑analysis model to train a language model that generates positive product reviews, covering task definition, data sampling, reward evaluation, model optimization, and experimental results.
This article demonstrates how to apply Reinforcement Learning from Human Feedback (RLHF) using a sentiment‑analysis model as a reward function and Proximal Policy Optimization (PPO) to fine‑tune a language model that generates positive product reviews, complete with code snippets and experimental results.
ChatGPT, a dialogue-focused language model, builds on the GPT family and employs techniques such as Reinforcement Learning from Human Feedback (RLHF), the TAMER framework, and a three-stage training pipeline (supervised fine‑tuning, reward modeling, and PPO reinforcement learning) to achieve advanced conversational capabilities.
ChatGPT, OpenAI’s conversational AI launched in December 2022, builds on GPT‑3 and advanced training methods like supervised fine‑tuning and reinforcement learning from human feedback, offering versatile applications from search assistance to code generation, while also revealing notable limitations and future commercial prospects.
Prof. Qiu Xipeng’s talk highlighted how large language models can be offered as a service and efficiently adapted via in‑context learning, lightweight label‑tuning, and gradient‑free black‑box optimization, showcasing a unified asymmetric Transformer (CPT) that handles understanding, generation, ABSA and NER tasks while reducing resource demands.
This article provides a comprehensive overview of natural language processing fundamentals, covering the challenges of language modeling, N‑gram and Markov assumptions, smoothing techniques such as discounting and add‑one, evaluation via perplexity, basic tasks like Chinese word segmentation, subword tokenization, POS tagging, syntactic and semantic parsing, and a range of downstream applications including information extraction, sentiment analysis, question answering, machine translation, and dialogue systems.
This article explains the role of Weighted Finite State Transducers in conventional HMM‑based speech recognition, covering language models, pronunciation dictionaries, WFST definitions, semiring theory, composition and determinization operations, decoding graph construction (HCLG), lattice rescoring, and practical optimization techniques for real‑world scenarios.
This article provides a comprehensive, illustrated walkthrough of OpenAI's GPT‑2 language model, covering its decoder‑only Transformer architecture, self‑attention mechanisms, token processing, training data, differences from BERT, and applications beyond language modeling, enriched with visual diagrams and code snippets for deeper understanding.
DiDi’s self‑learning ASR platform lets non‑technical users upload business data, automatically train, test and deploy models with semi‑supervised learning, hot‑word updates and LSTM rescoring, creating a closed‑loop pipeline that boosted vehicle voice‑interaction accuracy from around 80 % to over 95 % within months.
This article proposes a method to identify low-quality text in business data by training a large-scale unsupervised language model to compute sentence perplexity, converting the detection problem into a threshold decision, and details model design, challenges, optimizations, and online performance results.
This article excerpt explains the structure and training of feed‑forward neural networks, illustrates their use in neural language models, describes deep structured semantic models for ranking tasks, and details two‑stage recommendation systems such as YouTube, covering both theoretical formulas and practical deployment considerations.
This article presents a distributed system that efficiently supports large‑scale n‑gram language models for automatic speech recognition by introducing caching, a two‑level distributed index, batch processing, and a cascading fault‑tolerance mechanism, demonstrating robust scalability and low communication overhead in Tencent's WeChat ASR service.
Intelligent speech recognition converts spoken audio to text using a pipeline of feature extraction, acoustic and language modeling, where deep neural networks—especially CNN, LSTM, and hybrid CLDNN architectures—drive high accuracy, enabling mobile voice input, call‑center transcription, legal record keeping, and Tencent Cloud ASR’s 97% Mandarin accuracy with speaker separation and on‑premises deployment.
The article outlines the design and implementation of a multi‑turn conversational chatbot, detailing how natural‑language understanding converts user utterances into structured representations, a CNN‑LSTM language model classifies topics, intents, and sentiments, and an XML‑based answer engine orchestrates tasks and services for real‑world deployment.
This article describes the design and implementation of a spelling‑correction module for 58.com’s search engine, covering common query errors, rule‑based and statistical language‑model approaches, offline dictionary generation, n‑gram and Viterbi decoding, online workflow, and practical examples.
This article explains the concepts, training pipeline, deployment formats, and practical applications of language models—particularly LSTM‑based models—for evaluating and generating text quality in a real‑world rental listing platform, highlighting data preparation, model training, and online serving techniques.
The article explains why Chinese word segmentation is essential for NLP tasks, outlines its fundamental difficulties such as ambiguity and out‑of‑vocabulary words, reviews dictionary‑based, statistical, and CRF approaches, and shares practical experiences from 58 Search’s production system.
iQIYI HomeAI tackles the bottleneck of static language models in voice assistants by separating phonetic and semantic processing, correcting ASR errors at the intent‑recognition layer with pinyin‑enhanced entity correction, thereby reducing error amplification in video‑on‑demand interactions and paving the way for adaptive, personalized voice experiences.
This article explains the fundamentals of recurrent neural networks (RNNs), their role in language modeling, various RNN architectures such as bidirectional and deep RNNs, the back‑propagation through time (BPTT) training algorithm, gradient challenges, vectorization techniques, and provides a step‑by‑step code implementation.
