DeepSeek’s distillation technology combines data and model distillation to transfer knowledge from large teacher models to compact student models, detailing its definitions, principles, key innovations, architecture, training methods, performance gains, and challenges, especially in multimodal contexts.
This article reviews the most notable open‑source reproductions of DeepSeek‑R1—including Open R1, OpenThoughts, LIMO and DeepScaleR—detailing their data pipelines, training steps, reinforcement‑learning strategies, dataset constructions, and benchmark results that demonstrate how small, high‑quality data can rival massive‑scale models.
FlexTok introduces a variable‑length 1‑D image tokenizer that can reconstruct images with as few as eight tokens, surpasses TiTok in FID and MAE across multiple token budgets, and serves as a hierarchical visual vocabulary for autoregressive image generation.
The article examines whether pre‑training scaling laws remain valid, compares Grok‑3’s architecture and training strategy with Deepseek models, and explores how different scaling approaches—pre‑training, RL‑based, and test‑time—affect the cost‑effectiveness and intelligence of large language models.
The article analyses recent breakthroughs such as OpenAI's o1, Long CoT, and test‑time search, arguing that enabling LLMs to perform self‑critique and reinforcement learning with long output sequences is essential for future AI performance, while warning against overly structured workflows.
Transfusion is a 7B‑parameter transformer that jointly trains language modeling and diffusion losses on mixed text‑image data, enabling seamless text generation, image generation, and image understanding within one model and outperforming prior multimodal approaches such as Chameleon across multiple benchmarks.
The X‑R1 release introduces fully open‑source 0.5B, 1.5B and 3B models with one‑click training scripts, LoRA fine‑tuning support, Chinese inference capabilities, detailed reward‑curve visualizations, and quick‑start instructions for both CUDA and Ascend platforms.
VARGPT is a novel multimodal large language model that unifies visual understanding and autoregressive image generation within a single architecture, extending LLaVA with next‑token and next‑scale prediction, trained through three staged data‑curated phases and achieving superior performance on numerous vision‑language benchmarks.
This article explains what reasoning language models are, when they are needed, and reviews four main techniques— inference‑time scaling, pure reinforcement learning, combined SFT + RL, and distillation—illustrated with DeepSeek‑R1’s development, cost analysis, and low‑budget alternatives.
This week’s AI roundup covers OpenAI’s fully open ChatGPT Search, the launch of Deep Research for automated multi‑step research, NetEase Youdao’s integration of DeepSeek‑R1, Figure’s robot partnership break with OpenAI, low‑cost AI model s1, OpenAI’s Stargate data‑center plans, Google’s antitrust probe, DeepSeek’s traffic surge, and top AI scientist Xu joining Alibaba.
The article reviews DeepSeek‑R1’s training methodology—including cold‑start data collection, multi‑stage RL fine‑tuning, SFT data generation, and model distillation—highlights its performance comparable to OpenAI‑o1‑1217, and discusses key contributions, reward design, successful experiments, and failed attempts.
The article reviews DeepSeek‑R1, detailing its reinforcement‑learning‑based training pipeline that uses minimal supervised data, cold‑start fine‑tuning, multi‑stage RL, rejection‑sampling SFT, and distillation to achieve reasoning performance comparable to OpenAI‑o1‑1217, while also discussing successful contributions and failed experiments.
The paper introduces MILS, a training‑free multimodal iterative LLM solver that lets large language models perceive and generate across image, video, and audio domains, achieving new state‑of‑the‑art results without any task‑specific data or fine‑tuning.
Shanghai AI Laboratory’s InternLM 3.0 upgrade demonstrates that refining data quality—measured as intelligence‑per‑token—can replace massive datasets, achieving higher reasoning and dialogue capabilities with just 4 TB of tokens, cutting training cost by over 75 % while approaching GPT‑4‑level performance.
This article introduces the fundamentals of large‑model fine‑tuning, compares popular parameter‑efficient methods such as LoRA and its variants, presents experimental results on the Qwen2.5‑7B model, and discusses current challenges and future research directions.
Microsoft outlines six 2025 AI forecasts—including more powerful models, autonomous AI agents reshaping work, AI companions aiding daily life, greener resource use, rigorous testing and customization, and AI-driven scientific breakthroughs—highlighting how these advances will transform industries, research, and everyday experiences.
Former Google CEO Eric Schmidt warns that AI is on the brink of a transformative era, highlighting three 2025 breakthroughs—unlimited context memory, autonomous AI agents, and text‑to‑action programming—while also stressing the looming risks of energy consumption, security threats, and the need for ethical safeguards.
This article compiles recent AI research breakthroughs presented at AAAI 2025 and NeurIPS 2024, summarizing eight papers on multi‑condition image generation, mixed auto‑regressive models, hallucination mitigation in vision‑language models, quantized diffusion denoising, facial part swapping, language‑guided concept vectors, attribution consistency, and video virtual try‑on, with links to each work.
This article surveys the post‑training pipelines of major open‑source large language models released this year, detailing their alignment algorithms, data synthesis, reward modeling, DPO/GRPO variants, long‑context handling, tool use, and model‑averaging techniques, and highlights emerging trends such as data‑centric pipelines and iterative weak‑to‑strong alignment.
In this extensive interview, Rich Sutton critiques the focus on transient deep learning, advocates for continuous learning, discusses the reward hypothesis, outlines research challenges, offers advice to emerging scholars, and predicts breakthroughs in AI understanding by 2030‑2040.
EchoMimicV2 is an open‑source digital‑human framework that generates high‑quality half‑body animation videos from a single reference image, an audio clip, and a hand‑gesture sequence, addressing challenges of facial portrait limits, complex condition injection, and inference latency in audio‑driven animation.
At the 2024 Global Machine Learning Conference in Beijing, 360 AI Research Institute showcased cutting‑edge multimodal large‑model research, fine‑grained open‑world object detection, and document understanding technologies, highlighting open‑source releases, real‑world deployments, and competitive achievements in AI competitions.
Tencent has open‑sourced its 389‑billion‑parameter Hunyuan Large Mixture‑of‑Experts model—featuring 52 B active parameters, 256 K token context, novel routing, KV‑cache compression, and advanced training optimizations that beat leading open‑source models—and its first text‑to‑3D/image‑to‑3D Hunyuan 3D Generation model, both downloadable via GitHub, Hugging Face, and Tencent Cloud.
This article highlights eight Meituan research papers accepted at CIKM 2024—spanning self‑supervised sequential recommendation, rating‑consistent explanation generation, CTR prediction via recommendation pre‑training, cross‑domain interest transfer, multimodal vector retrieval, design‑aware poster layout, order‑fulfillment cycle‑time forecasting, and delivery‑scope substitution—offering insights from both internal and university collaborations.
AI researchCTR predictionCross‑Domain Recommendation
This article surveys and categorizes instruction data selection techniques for large language model fine‑tuning, explaining metric‑based, trainable‑LLM, powerful‑LLM, and small‑model approaches, detailing representative papers, their pipelines, and empirical findings on data quality and diversity.
This announcement introduces three Ant Group papers accepted at EMNLP 2024—Mixture‑of‑Modules for dynamic Transformer assembly, a plug‑and‑play visual reasoning framework built via data synthesis, and a layer‑wise importance‑aware efficient fine‑tuning method for large language models—highlighting their innovations and upcoming live presentations.
AI researchEMNLP 2024Parameter-Efficient Fine-Tuning
NoteLLM-2 introduces multimodal In-Content Learning and Late Fusion to overcome visual‑modality bias in end‑to‑end fine‑tuned large representation models, delivering significant gains over baseline multimodal LLMs and traditional retrieval methods in recommendation tasks.
AI researchContrastive Learningin‑content learning
The VICTORIA algorithm, presented by Alibaba Cloud AI Platform PAI and South China University of Technology at ACM MM 2024, leverages linguistic dependency parsing to guide cross‑attention in Stable Diffusion, enabling accurate, training‑free multi‑object image editing while preserving spatial structure and achieving state‑of‑the‑art results on benchmark datasets.
The VICTORIA algorithm, presented by Alibaba Cloud's PAI team at ACM MM2024, leverages linguistic dependency parsing and cross‑attention control to overcome multi‑object editing challenges in training‑free text‑guided image editing, delivering precise, structure‑preserving results across diverse scenes.
The Beijing University of Posts and Telecommunications' Network Intelligent Research Center (NIRC) had two papers accepted to NeurIPS 2024, presenting FM-Delta, a lossless compression technique that halves storage and cuts cloud costs by over 40%, and GLAFF, a global‑local fusion framework that markedly improves the robustness of time‑series forecasting across multiple domains.
The author speculates on OpenAI’s o1 architecture, proposing that it relies on reinforcement learning guided by a generalizable, process‑supervised reward model, and outlines data collection, multi‑model generation, and training tweaks needed to realize such a system.
The author reviews Google NotebookLM, describing how it aids deep paper reading, boosts chat willingness with guided prompts, maintains conversation coherence through self‑play insights, highlights the audio‑overview feature, and reflects on AI concepts such as the "bitter lesson" and the limits of self‑play in open scenarios.
The paper introduces XPSR, a diffusion‑based image super‑resolution method that incorporates cross‑modal semantic priors from a large multimodal language model, achieving state‑of‑the‑art performance on both reference and no‑reference quality metrics across synthetic and real‑world video restoration tasks.
This article presents the Fudan University Knowledge Workshop Lab's comprehensive research on domain large models, covering background, domain adaptation, capability enhancement, collaborative workflows, challenges such as inference cost and alignment, and proposed solutions including source‑enhanced training, self‑correction mechanisms, and hybrid retrieval‑augmented generation.
The article explains how homologous small language models—trained on the same tokenizer and data as their large counterparts—serve as cheap, fast experimental platforms that can predict large‑model performance, guide pre‑training decisions, and support techniques like distillation and reward modeling.
FancyVideo is an open‑source UNet‑based video generation model that supports arbitrary resolutions, aspect ratios, styles, and motion dynamics by introducing a Cross‑frame Textual Guidance Module (CTGM) with temporal injectors, refiners, and boosters, achieving state‑of‑the‑art results on multiple benchmarks and enabling versatile applications such as video extension, backtracking, and frame interpolation.
The article presents a curated collection of Ant Group's research papers accepted at KDD2024, summarizing each paper's title, type, link, source, relevant fields, and abstract, covering topics such as graph mining, large language models, fraud detection, recommendation systems, and multimodal medical AI.
This article introduces DAFNet, a dynamic auxiliary fusion framework that enables efficient sequential editing of large language models by injecting knowledge with reduced resource costs while preserving model reliability, generalization, and mitigating hallucination, and details its dataset, architecture, and evaluation results.
The paper introduces a method that classifies user queries into ordinary and long‑tail types, applying retrieval‑augmented generation only to long‑tail queries, which improves large language model efficiency and accuracy by leveraging specialized knowledge detection metrics and an extended RAG pipeline.
AI researchECE metricRetrieval-Augmented Generation
Recent ACL 2024 papers from Alibaba Cloud’s PAI platform showcase open‑source Chinese diffusion models, an interactive multi‑turn prompt generator, a long‑tail knowledge‑aware retrieval‑augmented LLM approach, and a dynamic fusion network for sequential model editing, all integrated into cloud services.
AI researchKnowledge EditingRetrieval-Augmented Generation
This article reviews the IJCAI 2024‑presented Task‑aware Decoding (TaD) technique, explains how it mitigates large‑language‑model hallucinations when combined with Retrieval‑augmented Generation, and details experimental results, practical deployments, and future research directions.
A New York Times report reveals that a hacker accessed OpenAI's internal messaging system, exposing employee discussions on AI advancements and sparking concerns about foreign espionage, internal security practices, and the broader national‑security implications of AI technology.
This article details Ant Group's research on integrating large language models into recommendation pipelines, covering background challenges, knowledge extraction, teacher‑student distillation, experimental results, and practical Q&A for improving bias, efficiency, and cold‑start performance.
This article reviews recent step‑level DPO research, compares it with instance‑level DPO, explains the underlying Monte Carlo Tree Search formulation, and presents the author’s own replication experiments that demonstrate consistent performance gains across multiple LLM sizes on GSM8K and MATH benchmarks.
The article highlights three Meituan research papers accepted at SIGIR 2024—covering deep automated mechanism design for ad auction, a retrieval‑enhanced vertical federated recommendation framework, and disentangled contrastive hypergraph learning for next POI recommendation—and announces an online sharing event where the authors will present their work.
This article surveys the main challenges of deploying large language models, introduces key RAG optimization papers such as RAPTOR, Self‑RAG, and CRAG, and compiles practical engineering tricks—including chunking, query rewriting, hybrid and progressive retrieval—to help practitioners build more accurate and efficient RAG systems.
On June 27, 2024, Xiaohongshu’s technical team will livestream a two‑hour session across WeChat Channels, Bilibili, Douyin and Xiaohongshu, showcasing six top‑conference papers on large‑model advances—including early‑stopping and fine‑grained self‑consistency, novel evaluation methods, negative‑sample‑assisted distillation, and LLM‑based note recommendation—followed by a Q&A and recruitment briefing.
The paper introduces Free-Prompt-Editing (FPE), a novel, efficient algorithm for text‑guided image editing that leverages probe analysis of cross‑ and self‑attention maps in Stable Diffusion, demonstrates its superiority over existing methods through extensive experiments, and provides open‑source implementation for both synthetic and real‑image editing.
The paper introduces Free-Prompt-Editing, a concise and efficient algorithm that replaces self‑attention maps during denoising to achieve high‑quality text‑guided image edits without source prompts, and demonstrates its superiority over existing methods on both synthetic and real images.
AI researchFree-Prompt-Editingattention mechanisms
This article presents Fudan University's Knowledge Factory Lab research on domain large models, covering background, challenges, data selection, source‑enhanced tagging, capability improvements, self‑correction, collaborative workflows, and retrieval‑augmented generation for practical AI deployment.
This article introduces MetaGPT, an open‑source multi‑agent framework that leverages large language models to automate software development, data science, and simulation tasks, detailing its development, impact, experimental results, memory and reasoning enhancements, and comparisons with related systems.
This article reviews recent advances in generative recommendation, highlighting challenges such as item representation and multimodal fusion, and summarizing four key research papers that propose novel tokenization, collaborative integration, and transformer-based multimodal approaches to improve recommendation performance.
The article introduces 360VL, an open‑source multimodal large language model built on Llama‑3‑70B, describes its novel C‑abs bridge architecture for high‑resolution visual understanding, outlines the two‑stage training with bilingual data, and presents benchmark results showing superior performance over prior LMMs.
This talk reviews recent advances in large language models, compares them with knowledge graphs, explores how LLMs enhance knowledge extraction and completion, examines how knowledge graphs aid LLM evaluation and safe deployment, and outlines future interactive integration between the two technologies.
OpenAI introduced GPT‑4o, a free, omni‑capable multimodal model that processes text, audio, and images together, delivers near‑human response latency, showcases impressive live demos, and will soon be available via a discounted API, marking a significant step forward in end‑to‑end AI research.
Naver’s newly unveiled HyperCLOVA X large‑language model, detailed in an arXiv technical report, claims superior cross‑lingual reasoning for Asian languages, especially Korean, by pre‑training on a data mix of Korean, multilingual text and code, achieving state‑of‑the‑art translation and multilingual capabilities.
This article reviews a recent study that bridges the knowledge gap between large language models and recommendation systems by generating natural‑language auxiliary tasks, fine‑tuning the models, and achieving notable performance gains on Amazon domain benchmarks.
The DIN‑SQL approach enhances Text‑to‑SQL performance by using large language models in a decomposed in‑context learning framework with schema linking, query classification, SQL generation, and self‑correction modules, achieving state‑of‑the‑art 85.3% execution accuracy on the Spider benchmark by breaking complex queries into manageable sub‑tasks.
Elon Musk announced the open‑source release of Grok, xAI’s new large‑language‑model chatbot, while recalling his lawsuit against OpenAI; the article explains Grok’s rapid development, links to the GitHub repository, summarizes the seminal “Grokking” research paper that describes a sudden generalization breakthrough in neural networks, and provides reference links.
In a detailed interview, Tsinghua researcher Zhao Sicheng and JD Retail senior director Peng Changping discuss the latest progress in multimodal large models, their practical applications in advertising and e‑commerce, persistent challenges such as hallucinations and data alignment, and the skills engineers need to thrive in the emerging AI era.
This article examines why ensemble methods are crucial for large language models, outlines five core fusion strategies—including model integration, probability integration, graft learning, crowdsourced voting, and Mixture of Experts—provides implementation details, pseudo‑code, and discusses practical challenges and recent research advances.
This talk presents GraphTranslator, a framework that bridges pretrained graph models and large language models to enable unified handling of both predefined and open-ended graph analysis tasks by translating node representations into language tokens and training an alignment producer for node‑text pairs.
AI researchGraph Neural Networkslarge language models
OpenAI’s Sora video model unifies image and video data into latent spacetime patches via a VAE, trains on original resolutions with GPT‑4‑expanded captions, employs a Diffusion Transformer backbone for patch‑wise denoising, and demonstrates 3D‑consistent, long‑term world‑model capabilities that hint at a unified computer‑vision paradigm and steps toward AGI.
This article reviews the challenges of multi‑task learning, compares various network architectures such as hard‑parameter sharing, MMoE, CGC, and PLE, and examines loss‑balancing techniques like GradNorm, Dynamic Weight Average and task‑prioritization, offering insights on how to mitigate the “seesaw” effect and improve overall performance.
AI researchDynamic Weightinggradient normalization
OpenAI’s Sora model demonstrates large‑scale text‑conditional video generation using a diffusion transformer that operates on spatiotemporal patches, supporting variable durations, resolutions, and aspect ratios while showcasing emergent simulation abilities, flexible sampling, and multimodal editing capabilities, though it still has notable limitations.
This article reviews recent advances that enhance dual‑tower recommendation models by injecting interaction information through various knowledge‑distillation strategies and interaction‑enhanced architectures, summarizing methods such as PFD, ENDX, TRMD, VIRT, Distilled‑DualEncoder, ERNIE‑Search, ColBert, IntTower and MVKE.
The article recounts Sergey Brin’s surprising appearance as a core contributor to Google’s Gemini AI model, tracing his early technical career, his semi‑retirement from Alphabet, the company’s AI challenges after ChatGPT, and how Brin returned to help develop Gemini, highlighting internal reactions and his lasting influence.
At EMNLP 2023, the joint WeChat AI and Peking University paper 'Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning' won the Best Long Paper award, revealing that label tokens act as anchors driving information aggregation in shallow layers and prediction flow in deep layers, and proposing methods to improve and diagnose in‑context learning.
This article systematically explains how to create vertical domain large language models by continuing pre‑training on domain data, constructing fine‑tuning datasets with self‑instruct, reducing hallucinations, and integrating knowledge retrieval, while also reviewing related papers, products, and system architectures.
This article reviews recent research on augmenting language and vision models by incorporating external knowledge sources such as knowledge graphs, multi‑source retrieval, and dynamic tool‑calling frameworks, presenting three systems—OREO‑LM, REVEAL, and AVIS—and their experimental results.
MiniGPT-5 is a novel multimodal generation model using generative vokens to interleave text and image synthesis, integrating Stable Diffusion and LLMs with a two-stage training that requires no domain-specific annotations, achieving state‑of‑the‑art coherence and quality on benchmarks like CC3M, VIST, and MMDialog.
The article shares a practitioner’s transition story from reinforcement‑learning‑focused game AI to large‑model work, outlines the challenges faced during job hunting at major Chinese tech firms, and provides a curated list of 23 technical interview questions covering PPO, RLHF, dataset evaluation, model fine‑tuning, and broader LLM concepts.
This article presents MiniGPT-4, a multimodal system that combines a frozen visual encoder (Q‑Former + ViT) with an open‑source large language model (Vicuna), describes its motivation, training pipeline, demo capabilities, observed limitations, and includes a brief Q&A session.
At the 2023 Bund Conference, Ant Group presented the Large Graph Model (LGM), a research effort that combines graph computing, graph learning, and large language models to enrich heterogeneous graph data and enable more precise insights for complex digital applications, with results accepted at WWW 2023.
This article shares practical techniques for domain‑specific large model continue pre‑training, including data selection, mixing ratios with general data, multi‑task instruction pre‑training, resource‑aware fine‑tuning strategies, evaluation set design, vocabulary considerations, and deployment constraints for 7‑13B models.
Google’s recent research reveals that when small neural networks are trained for extended periods on tasks like modular addition, they can abruptly shift from memorizing training data to genuinely generalizing—a sudden “grokking” phenomenon driven by weight decay and the emergence of periodic weight structures.
This article reviews a study that compares the stage‑wise language acquisition of infants with the learning trajectory of GPT‑2, using linguistic probes and statistical tests to determine whether deep language models follow sequential or parallel learning patterns similar to children.
Baidu’s chief architect Li Shuanglong outlined how AIGC, driven by advanced large‑language and multimodal models, is already powering commercial tools such as automated copywriting, 2D digital‑human video creation and lead‑generation chatbots, while emphasizing future progress in engineering scalability, algorithmic fidelity, data quality, and scenario‑focused applications.
This article presents OPPO Research Institute's recent advances in large‑model AI, detailing the CHAOS pretrained model that topped the CLUE leaderboard, the knowledge‑enhanced training pipeline, and the GammaE model for multi‑hop reasoning over knowledge graphs, together with experimental results and practical training tips.
Leading experts discuss how causal inference intersects with large language models, exploring opportunities, challenges, industry applications, and future research directions, while sharing personal journeys into causal reasoning and offering practical advice for practitioners.
This article reviews the evolution of face detection, covering the Widely‑Face dataset, major research directions such as feature fusion, label assignment, auxiliary supervision, anchor‑free methods, NAS‑based designs, summarizes key papers from S3FD to MogFace, introduces ModelScope implementations, and outlines future challenges and opportunities.
This article presents the Hyper‑Actor Critic (HAC) framework that splits reinforcement‑learning policies into continuous hyper‑actions and effective recommendation lists, introduces alignment and supervised losses, and demonstrates superior performance on an online simulator compared to existing RL and supervised methods.
This article provides a comprehensive overview of the development of large language models, reviewing classic papers from GPT‑1 through GPT‑4, discussing open‑source implementations such as LLaMA, Alpaca, GLM, and ChatGLM, and analyzing training methods, datasets, and future research directions.
This curated collection highlights the latest open‑source LLM breakthroughs, comprehensive surveys, AutoGPT developments, distributed training pitfalls, and practical tools for AI engineers, providing concise descriptions and direct links to each resource for deeper exploration.
This article provides a comprehensive overview of major large language models—including GPT series, T5, LaMDA, LLaMA, BLOOM, and others—detailing their architectures, parameter scales, open‑source status, and the evolution of instruction‑fine‑tuning techniques that improve zero‑shot and few‑shot performance.
This article reviews the emergence phenomena observed in large language models, explains how model scale, in‑context learning and chain‑of‑thought prompting contribute to sudden performance gains, discusses small‑model alternatives, and explores the relationship between emergence and the training‑time Grokking effect.
The article examines the rapid proliferation of large language models, weighing both the opportunities for experimentation and the drawbacks of noise, and argues that establishing authoritative Chinese LLM evaluation benchmarks is essential to guide meaningful progress in the field.
This article surveys eight popular open‑source ChatGPT alternatives, detailing each model’s size, training data, performance relative to proprietary systems, and providing links to code repositories, demos, and papers for developers interested in building or researching large language models.
This article reviews the rapid progress of large language models (LLMs), covering their historical development, scaling laws, emergent abilities, core technologies such as training and alignment, resource ecosystems, evaluation methods, safety concerns, and prospective research challenges.
This article chronicles Ilya Sutskever’s two‑decade rise from a young researcher to a leading figure in artificial intelligence, highlighting his early mentorship, breakthroughs in image recognition, language translation, the founding of OpenAI, and the development of GPT and DALL‑E models.
In this DataFun interview, Huawei Noah's Ark Lab senior researcher Zhou Min discusses the state of graph machine learning in academia and industry, covering algorithmic foundations, model variants, practical applications, scalability challenges, and future directions for more universal feature extraction across domains.
The article summarizes the technical innovations behind GPT‑4, highlighting its multimodal abilities, improved alignment methods, scaling‑law‑based performance prediction, and remaining limitations, while referencing the official OpenAI technical report and community analyses.
The article explains how Google's i‑S2R and GoalsEye projects use iterative simulation‑to‑real training, behavior cloning and goal‑conditioned learning to enable robots to play table‑tennis with humans, highlighting the challenges, experimental setup, and performance improvements achieved across player skill levels.
The article explores three emerging directions for next‑generation large language models—self‑generated training data, built‑in verification with external retrieval, and massive sparse‑expert architectures—highlighting recent research, practical challenges, and their potential to reshape AI development.
The article outlines three promising research avenues—self‑generated training data, model‑driven fact‑checking, and sparse expert architectures—that could shape the next wave of large language model innovation and address current limitations such as data scarcity and hallucinations.
AI researchlarge language modelsmodel self‑improvement
This article reviews recent research on the emergent abilities of large language models—such as chain‑of‑thought reasoning, knowledge retrieval without external sources, and robustness to distribution shifts—examining scaling laws, model size thresholds, and the open questions surrounding a potential paradigm shift from fine‑tuning to in‑context learning.
AI researchchain-of-thought promptingemergent abilities
This article compiles three highly upvoted Zhihu answers that examine OpenAI's ChatGPT, discussing its breakthrough impact on NLP, visual in‑context learning, reinforcement‑learning‑from‑human‑feedback, and the broader implications for AI research and development.
Google’s new Muse model, a Transformer‑based text‑to‑image system running on TPUv4, claims to generate 256×256 images in 0.5 seconds—far faster than Imagen—while delivering unprecedented photorealism and deep language understanding through parallel decoding and large‑scale LLM‑conditioned training.
The article reviews the evolution of large language models from the pre‑GPT‑3 era to the present, analyzes the conceptual and technical gaps between Chinese and global research, and outlines key future research directions such as scaling laws, prompting techniques, multimodal training, and efficient model architectures.
During Xiaohongshu’s REDtech livestream, AI researchers examined ChatGPT’s rapid adoption, versatile task performance, and underlying large‑scale pre‑training with in‑context learning, while highlighting persistent hallucinations, weak reasoning, high costs, and limited search‑engine replacement potential, and emphasized the importance of RLHF‑driven human feedback for future multimodal AI research.