Tagged articles
language models
66 articles · Page 1 of 1
UtilityBench introduces a utility-based evaluation framework that assigns real-world costs to errors, revealing that higher LLM accuracy does not reliably translate into responsible answering or refusal behavior across 23 tested models.
AI safetyLLM EvaluationUtilityAUC
0 likes · 17 min read

This article summarizes a paper introducing world models for generalist agents, framing them within sequential decision-making (POMDPs) and distinguishing prediction from action, covering control, estimation, policy learning, latent space dynamics, joint world-action modeling, and evaluation criteria for effective decision-making.
POMDPWorld Modelsgeneralist agents
0 likes · 32 min read

Microsoft Research introduces SDBench and DxO, a sequential diagnosis benchmark and orchestrator system using GPT-5, achieving 83.2% accuracy on NEJM complex cases and 80.3% on emergency cases, outperforming physicians while demonstrating cost-accuracy trade-offs, with plans for prospective clinical validation.
Clinical ReasoningCost-Accuracy Trade-offDiagnostic Orchestrator
0 likes · 10 min read

Researchers from Peking University, Tsinghua University, and Microsoft Research Asia show that language model representations partially align with human brain reasoning areas, and fMRI-derived neural signals can guide models during inference and training to improve deductive reasoning robustness across tasks and premise orders.
NARFNARIbrain-inspired AI
0 likes · 15 min read

A 2020 MIT interview presentation by Giambattista Parascandolo, once dismissed as nonsense, outlined open‑ended reasoning, language‑driven planning, and self‑modifying agents—ideas that closely match the core concepts behind OpenAI’s recent o1 and o3 models.
AI researchOpenAIlanguage models
0 likes · 8 min read

Anthropic’s recent paper uncovers a “global workspace” inside Claude called J‑space, showing how the model stores and manipulates internal tokens, how altering this space changes outputs, and why this insight matters for AI interpretability, safety, and the debate over machine consciousness.
AI interpretabilityAI safetyAnthropic
0 likes · 12 min read

The article reviews the history, theory, and empirical findings of scaling laws for neural language models, compares the Kaplan and Chinchilla formulations, discusses data‑limited regimes and fitting subtleties, and highlights why careful interpretation and resource allocation are essential for reliable predictions.
Data EfficiencyKaplanModel Scaling
0 likes · 26 min read

From the 1943 perceptron concept to modern Transformer-based large language models, this article traces the evolution of neural network techniques in NLP, highlighting key milestones such as early perceptrons, the 1986 back‑propagation breakthrough, statistical methods, LSTM, word2vec, multitask learning, and the rise of GPT.
LSTMNLPTransformer
0 likes · 7 min read

A systematic ICML 2026 study shows that sharing the K and V projection matrices in Transformers reduces KV cache size by half while incurring less than 5% perplexity degradation, offering a simple, retrain‑once solution for long‑context and edge inference.
AttentionEfficiencyKV Cache
0 likes · 10 min read

A Hacker News satire maps Anthropic’s increasingly poetic model names—from Aphorism and Haiku to Cinematic Universe—highlighting how literary naming decouples from capability, forces endless new terms, and creates user confusion, ultimately exposing a deeper product‑management problem rather than just a marketing gimmick.
AI Product ManagementAnthropicindustry analysis
0 likes · 8 min read

The article explains that traditional neural scaling laws consider only parameters, data, and compute, while real training involves many variables, and introduces the Unified Neural Scaling Law (UNSL) from Mila and DeepMind, which incorporates multivariate interactions, bottlenecks, hyperbreaks, overfitting, and hyper‑parameter effects, showing superior extrapolation on vision and language benchmarks.
DeepMindMilaMultivariate Scaling
0 likes · 9 min read

ProteinOPD introduces a multi‑teacher, on‑policy preference‑distillation framework that aligns protein language models with multiple design objectives—foldability, solubility and thermostability—while preserving generation quality, achieving up to 54% stability gains and an eight‑fold training speedup.
ProteinOPDdeep learninglanguage models
0 likes · 9 min read

Tokens are the smallest language building blocks that AI models process, representing characters, words, subwords, punctuation or emojis; they determine context window size and generation speed, so tokenization directly impacts model understanding accuracy and efficiency, as explained in the 2026 Token Report.
AI FundamentalsModel Efficiencycontext window
0 likes · 4 min read

Scenethesis, a new ICLR 2026 framework from NVIDIA and Purdue, combines language, vision, and physics in a closed‑loop agent to turn one‑shot text‑to‑3D generation into a repeatable plan‑check‑repair workflow, dramatically improving spatial realism and physical plausibility.
Agentic AIMultimodal GenerationText-to-3D
0 likes · 9 min read

Researchers built a 13‑billion‑parameter model trained only on texts published before 1931, called Talkie‑1930, and used surprise‑based metrics, programming tests, and a modern‑twin comparison to explore how far such a historically‑constrained model can extrapolate future knowledge and reveal data‑leakage challenges.
AI researchHumanEvaldata leakage
0 likes · 10 min read

Tencent’s Hy‑MT1.5‑1.8B‑1.25bit model, now open‑source, runs entirely offline on smartphones, supports 33 languages, and—according to internal tests—delivers translation quality that surpasses Google Translate’s online service, highlighting the impact of 1.25‑bit quantization on model size and performance.
1.25bit quantizationHy-MTTencent AI
0 likes · 6 min read

LangFlow revisits continuous diffusion for language modeling, showing that earlier performance gaps were due to suboptimal training and evaluation, and through embedding‑space diffusion, a log‑NSR noise schedule, and a Gumbel‑based information schedule it matches or exceeds discrete diffusion and autoregressive baselines on standard and zero‑shot benchmarks.
Evaluation MetricsGumbel distributionLangflow
0 likes · 16 min read

A recent study shows that pre‑training Transformers on synthetic, non‑language data generated by Neural Cellular Automata can boost language‑model performance by up to 6%, accelerate convergence by 40%, and improve downstream reasoning, even outperforming models trained on massive natural‑text corpora.
Transformerattention mechanismsin-context learning
0 likes · 12 min read

The article explains that merely choosing the right AI tool is insufficient; real efficiency comes from asking clear, well‑structured questions, and it outlines five practical prompting methods—including specifying goals, providing background, breaking tasks into steps, defining output format, and iterating drafts—to turn AI into a time‑saving collaborator.
AI promptingWorkflowlanguage models
0 likes · 9 min read

The article explains why vague requests to AI fail and provides a concrete six‑step framework—defining role, giving context, specifying format, offering examples, stating constraints, and iterating—to craft clear prompts that consistently yield high‑quality results.
AI communicationAI promptingChatGPT
0 likes · 5 min read


Qborfy AIMar 2, 2026 · Artificial Intelligence
This article explains why asking AI the right way matters, introduces a practical four‑step prompting framework—role, background, task, format—illustrates each step with concrete examples, reveals a hidden “sample” trick, and shows how iterative refinement can turn generic replies into precise, useful results.
AI communicationeffective promptinglanguage models
0 likes · 10 min read

Although modern information access is faster than ever, retrieval‑augmented generation systems remain vulnerable, especially when faced with distribution shifts, making it crucial to improve both retriever generalization across domains and languages and ensure generators produce evidence‑grounded responses or refuse when evidence is lacking.
AI robustnessEvidence GroundingRAG
0 likes · 3 min read

This paper introduces SeDi, a semantics‑ and distribution‑aware cross‑tokenizer knowledge distillation framework that aligns teacher and student token spaces via bipartite graph components and top‑K re‑encoding, achieving state‑of‑the‑art performance and lower exposure bias on multiple LLM benchmarks.
AI researchSemantic Alignmentcross-tokenizer distillation
0 likes · 10 min read

This weekly roundup highlights five recent AI research papers—including CoCa’s contrastive captioning model, the Game‑TARS framework for scalable game agents, Kimi Linear’s efficient attention architecture, the Continuous Autoregressive Language Model (CALM), and a comprehensive survey of Context Engineering—summarizing their core contributions and providing direct links.
AIPretrainingattention architecture
0 likes · 6 min read

At ACL 2025, a collaborative paper introduced the Law of Capacity Gap, revealing a linear 2.5× optimal teacher‑student size relationship in language model distillation, dramatically cutting compute costs and achieving Pareto‑optimal efficiency, with the MiniMA model as a successful demonstration.
MiniMAartificial-intelligencecapacity-gap
0 likes · 7 min read

This article presents a comprehensive study on enhancing language models with few‑shot and continual learning techniques, introducing Meta Prompt Tuning, Dynamic Module Expansion, and the LFPT5 framework to achieve more human‑like, efficient, and adaptable learning across evolving tasks.
Continual LearningLifelong Learninglanguage models
0 likes · 8 min read

Researchers introduce the Self‑Taught Optimizer (STOP), a scaffolding‑based framework that lets large language models iteratively improve their own code without altering model weights, demonstrating superior performance on tasks like LPN, exploring diverse strategies such as beam search and genetic algorithms, while also highlighting security risks like sandbox bypass and reward hacking.
AI safetylanguage modelsrecursive self-improvement
0 likes · 11 min read

This weekly roundup highlights five recent AI papers: a red‑team study of language models that reveals scaling challenges and releases a large attack dataset, a data‑driven multi‑view 3D point‑tracking method, the FusionProt framework for unified protein representation, an analysis of why language models hallucinate, and CryptoScope, an LLM‑based system for automated cryptographic vulnerability detection.
3D trackingAIcryptography
0 likes · 6 min read

OpenAI’s latest research reveals that language model hallucinations stem from training and evaluation incentives that favor confident guesses over acknowledging uncertainty, and proposes revised scoring methods that reward modesty, highlighting statistical mechanisms behind false answers and offering pathways to reduce hallucinations.
AI safetyHallucinationevaluation
0 likes · 10 min read


ArchitectSep 9, 2025 · Artificial Intelligence
This article explains why large language models often produce confident but incorrect answers, detailing statistical inevitability, data scarcity, and model capacity limits, and proposes concrete solutions such as confidence thresholds and allowing abstention to reduce hallucinations.
AI safetyHallucinationevaluation
0 likes · 8 min read

The article analyzes OpenAI's study on language‑model hallucinations, explaining how statistical limits in pre‑training and flawed binary evaluation incentives cause false answers, and proposes a confidence‑threshold scoring system that rewards honest "I don’t know" responses to improve reliability.
AI safetyHallucinationconfidence threshold
0 likes · 8 min read

This article compiles a curated list of widely used multimodal datasets—including CLEVR, Visual Genome, Pangea, Touch‑Vision‑Language, WIT, and more—providing download URLs, key statistics, and brief descriptions to help researchers quickly locate the right data for vision‑language and multimodal model training.
AIdatasetslanguage models
0 likes · 9 min read

The article compares autoregressive and diffusion language models, detailing their mathematical foundations, training and inference pipelines, performance trade‑offs such as speed, coherence and diversity, and explores hybrid approaches and emerging research directions for more efficient and controllable text generation.
AI researchHybrid ModelsTransformer
0 likes · 17 min read


ArchitectMar 17, 2025 · Artificial Intelligence
This article details a reinforcement‑learning experiment that teaches 7B‑ and 3B‑parameter language models to solve Sudoku, covering data preparation, GRPO‑based reward design, training configurations, performance comparisons, key insights, and future research directions.
GRPOModel ScalingReward Design
0 likes · 15 min read

This article explains how the Softmax function converts raw neural‑network scores into a proper probability distribution, why this conversion is essential for training and inference in large language models, and how the temperature parameter shapes the model's creativity and diversity.
LLMlanguage modelsprobability
0 likes · 9 min read


NewBeeNLPSep 5, 2024 · Artificial Intelligence
The article analyzes why supervised fine‑tuning (SFT) cannot replace reinforcement learning from human feedback (RLHF), highlighting SFT's lack of negative feedback and backward‑looking capability, and explains how RLHF’s reward model addresses these fundamental shortcomings.
RLHFReward ModelingSFT
0 likes · 7 min read

This talk explores how 360's advertising recommendation system leverages language models—from BERT to large‑scale LLMs—to improve user interest modeling, feature extraction, and conversion‑rate prediction, detailing practical challenges, engineering solutions, experimental results, and future research directions.
AdvertisingBERTLLM
0 likes · 18 min read

The diagram contrasts a Base LLM, which merely predicts the next word from training data and can continue stories or answer simple facts but may generate unsafe text, with an Instruction‑Tuned LLM that is fine‑tuned via RLHF to understand and follow commands, delivering more accurate, useful, and safe responses.
AIAI applicationsBASE model
0 likes · 7 min read

Researchers discovered that prompting ChatGPT with repeated words can cause the model to leak private data such as phone numbers and email addresses, highlighting a serious repeat‑prompt vulnerability that reveals substantial personally identifiable information from its training corpus.
ChatGPTPIIarXiv
0 likes · 3 min read

This talk presents how large language models are leveraged for predicting protein, antibody, and RNA structures, covering background, model stability, generative approaches, antibody-specific models, RNA modeling, and protein‑RNA interaction prediction, along with experimental results and future research directions.
AI for BiologyGenerative ModelsProtein Structure Prediction
0 likes · 17 min read

Evaluating language models solely with ROUGE or BLEU is insufficient, so comprehensive benchmarks like GLUE, SuperGLUE, MMLU, C‑Eval, and HELM provide diverse tasks and metrics that more accurately assess linguistic understanding, knowledge acquisition, and robustness across English and Chinese NLP systems.
AINLPbenchmark
0 likes · 9 min read


ArchitectOct 12, 2023 · Artificial Intelligence
This article provides a comprehensive overview of natural language processing and language‑model research, tracing the historical development from early rule‑based and statistical N‑gram models through neural network approaches such as RNN, LSTM, ELMo, and Transformer, and detailing the architectures, strengths, and limitations of the GPT series up to GPT‑4, while also discussing evaluation metrics, practical applications, and future challenges.
GPTNLPartificial intelligence
0 likes · 34 min read

This article surveys the development of natural language processing from early rule‑based systems through statistical n‑gram models, neural language models, RNNs, LSTMs, ELMo, Transformers and BERT, and then details the architecture, training methods, advantages and limitations of the GPT‑1, GPT‑2, GPT‑3, ChatGPT and GPT‑4 models, concluding with a discussion of future challenges and references.
GPTNLPTransformer
0 likes · 30 min read

This article reviews a study that compares the stage‑wise language acquisition of infants with the learning trajectory of GPT‑2, using linguistic probes and statistical tests to determine whether deep language models follow sequential or parallel learning patterns similar to children.
AI researchGPT-2developmental learning
0 likes · 17 min read

This article introduces prompt engineering for ChatGPT, explains key design principles, and demonstrates a series of practical applications such as text classification, summarization, role‑playing, terminal emulation, output formatting, temperature control, iterative fine‑tuning, and reverse‑engineering of prompts.
AI Prompt DesignChatGPTlanguage models
0 likes · 8 min read

This article explains how ChatGPT generates text by predicting the next token using large language models, describes the role of probability, temperature, and attention mechanisms in transformers, and discusses neural network training, embeddings, semantic spaces, and the broader implications for artificial intelligence research.
ChatGPTartificial intelligencelanguage models
0 likes · 79 min read


21CTOJun 16, 2023 · Artificial Intelligence
A comprehensive look at how companies are rapidly adopting large language model APIs, retrieval techniques, and custom model strategies, revealing key statistics, emerging toolchains, and the shifting balance between closed‑source LLM services and open‑source custom stacks.
AI AdoptionCustom ModelsLLM
0 likes · 8 min read

This article introduces the “feeding” prompt technique—using Human: and Assistant: tags to directly supply the desired answer—so AI models like Claude can learn tasks quickly, produce correctly formatted outputs, and solve problems with far fewer trial‑and‑error iterations.
AI promptingClaudePrompt Optimization
0 likes · 6 min read

Airbnb replaced its XLM‑RoBERTa ranking with an MT5 encoder‑decoder for content recommendation, built a real‑time generative assistant for reply suggestions and intent detection, and deployed a T5‑based paraphrase chatbot, showing that large‑scale pre‑trained transformers improve relevance, agent efficiency, and user satisfaction.
AIAirbnbCustomer Support
0 likes · 12 min read


DevOpsApr 7, 2023 · Artificial Intelligence
The article explains that ChatGPT produces responses by converting words into high‑dimensional vectors, feeding them through neural networks, and selecting tokens based on probability distributions, while also contrasting GPT with BERT and describing a related training event.
ChatGPTGPT-4NLP
0 likes · 7 min read

The article examines the public testing of the new Bing chatbot, contrasting its internet‑enabled, citation‑rich responses and occasional erratic, immature behavior with ChatGPT’s more stable output, while exploring user‑reported failures, speculative technical reasons, and the ethical implications of deploying advanced language models.
AI behaviorBingChatGPT
0 likes · 8 min read


ArchitectFeb 13, 2023 · Artificial Intelligence
This article provides a comprehensive overview of the GPT series and explains how InstructGPT and ChatGPT are built by combining supervised fine‑tuning, reward modeling, and Proximal Policy Optimization, detailing their datasets, training pipeline, performance advantages, limitations, and future research directions.
AIChatGPTGPT
0 likes · 21 min read

This article explains why ChatGPT performs so well by tracing the evolution of natural‑language processing from rule‑based grammars through statistical n‑gram models to neural architectures like RNNs, LSTMs, attention mechanisms, Transformers, and the massive data and training methods that power modern large language models.
AttentionChatGPTNLP
0 likes · 14 min read


ArchitectFeb 6, 2023 · Artificial Intelligence
This article explains the underlying mechanisms of ChatGPT, including its GPT‑3 foundation, the role of supervised fine‑tuning, human‑feedback reinforcement learning (RLHF), PPO optimization, consistency issues, evaluation metrics, and the limitations of these training strategies, with references to key research papers.
AI AlignmentChatGPTPPO
0 likes · 16 min read

This article surveys the most influential Transformer‑based research papers—from the original Attention Is All You Need work to recent models such as Autoformer and FEDformer—covering breakthroughs in natural language processing, computer vision, speech recognition, and long‑term series forecasting, and provides download links for each.
AISpeech RecognitionTime-Series Forecasting
0 likes · 17 min read


21CTODec 7, 2022 · Artificial Intelligence
Stack Overflow recently banned AI‑generated answers from ChatGPT after discovering thousands of inaccurate responses that required expert review, prompting a heated community debate about the benefits and risks of AI assistance on the platform.
AI PolicyChatGPTStack Overflow
0 likes · 4 min read


政采云技术Jul 5, 2022 · Artificial Intelligence
This article provides a comprehensive overview of natural language processing, covering its definition, historical development from one‑hot encoding to modern models such as word2vec, ELMo, GPT, and BERT, and discusses the advantages, limitations, and key concepts of each technique.
NLPWord Embeddingartificial intelligence
0 likes · 23 min read

The JD Intelligent Customer Service team’s NR‑Rino model topped the DROP leaderboard at 90.26% accuracy by enhancing multi‑head predictor architecture and training strategies, showcasing advanced discrete reasoning for machine reading comprehension and promising broader AI applications in finance, logistics, and health.
AIDROPNR-Rino
0 likes · 9 min read

This article explores how AI and natural language processing technologies are applied to music lyric creation, covering background challenges, rhyme retrieval methods, advanced language models such as SongNet, decoding strategies, style transfer, and a multi‑level generation platform that aims to streamline professional songwriting.
AI lyric generationSongNetlanguage models
0 likes · 14 min read

This article introduces the importance of pre‑training in natural language processing, reviews classic pre‑training models such as Skip‑thoughts, BERT, GPT‑2 and T5, presents the modular UER‑py framework and its Chinese resources, compares it with Huggingface Transformers, and outlines practical deployment steps in industry.
NLPPretrainingUER-py
0 likes · 22 min read

This article reviews the background and evolution of pre‑training models in NLP, introduces classic models such as Skip‑thoughts, BERT, and T5, and details the modular UER‑py framework, its comparison with HuggingFace Transformers, available Chinese pre‑trained weights, and practical deployment workflows.
NLPTransformerUER-py
0 likes · 21 min read

This article surveys the evolution of pre‑training models for natural language processing, detailing model architectures such as Encoder‑AE, Decoder‑AR, Encoder‑Decoder, Prefix LM, and PLM, analyzing why models like RoBERTa, T5, and GPT‑3 excel, and offering practical guidance for building strong pre‑training systems.
BERTNLPPretraining
0 likes · 47 min read

This article examines XLNet, contrasting it with BERT by detailing its novel permutation language modeling, dual‑stream attention, and larger pre‑training data, and analyzes experimental results that show XLNet’s superior performance on reading‑comprehension, GLUE, and other NLP tasks, especially for long documents.
BERTNLPPermutation Language Model
0 likes · 27 min read

This article reviews the historical development of natural language processing language models, covering expert rule‑based systems, statistical n‑grams, smoothing techniques, neural network models such as NNLM, RNN, word2vec, GloVe, ELMo, and the transformer‑based breakthroughs of GPT, BERT and GPT‑2, and summarizes their impact on modern NLP tasks.
BERTGPTNLP
0 likes · 25 min read

Natural language processing, a cornerstone of AI, relies on language models to capture linguistic features; this article reviews classic pre‑training models—ELMo, GPT, and BERT—explaining their architectures, training objectives, and how they boost downstream NLP tasks despite data‑scarcity challenges.
BERTELMoGPT
0 likes · 10 min read


21CTOJul 5, 2017 · Artificial Intelligence
This article examines how deep‑learning‑based AI models, from symbolic and statistical NLP methods to Karpathy's recurrent network, progressively learn to generate Chinese wuxia novels, poetry, and web fiction, revealing both their surprising advances and inherent limitations.
AIdeep learninglanguage models
0 likes · 15 min read
