A community 9B model combining DeepSeek V4 Pro reasoning with Qwen3.5-9B via distillation and refusal adjustment reached #1 on Hugging Face, showing higher science accuracy, 36% fewer reasoning tokens, and near-zero refusal rates.
Chinese AI models DeepSeek and Kimi face widening benchmark gaps against top closed-source models per Artificial Analysis v5, while The Information alleges they distilled OpenAI's GPT-6 Astra; the article also highlights their technical achievements and argues against dismissing contributions based on unverified claims.
CMU, UC Berkeley, and Amazon introduce Lafa, a continuous flow language model that converts pretrained autoregressive models into block-level asynchronous denoisers, achieving 2.02–2.54x throughput over AR baselines at 8B scale with matched quality, using under 30B tokens for adaptation.
FastH3 Omni, a distilled version of MiniMax H3, enables local video-with-audio generation on a 36GB M4 Max Mac by using MLX and INT6 quantization to reduce peak memory from 123GB to 19.5GB, though benchmarks are limited to a single device and quality trade-offs remain unverified.
A security vulnerability in OpenCode exposed API keys for OpenAI, Anthropic, and Google Gemini, allowing hackers to run up massive bills — up to $1,700 per user — by using the stolen credentials to generate high-quality training data for their own models, prompting emergency revocations and warnings about third-party AI tool risks.
PixVerse R2 demonstrates real-time world models that generate video and audio synchronously while accepting live inputs, but faces hurdles in compute costs, long-run stability, and distribution models for interactive content.
Meta CTO Andrew Bosworth argues AI is shifting from direct manipulation to intent understanding, emphasizes interaction quality over benchmark scores, details Muse's design including visual avatars and counter-arguments, describes private processing with trusted execution environments, expresses skepticism about near-term AGI and AI consciousness, and supports open-source AI with distilled models like Muse Glimmer.
This article analyzes five major paradigm shifts in large model post-training over the past six months, covering expert distillation, online distillation as RL alternative, RLVR refinements, SFT-RL distribution mismatch, and data quality as irreducible constraint, with specific papers and metrics.
This article explains model distillation through a question rewriting example, detailing how teacher models provide demonstrations to train smaller student models, the differences between distillation, fine-tuning, and quantization, and practical pitfalls like data quality and student capacity limits.
This practical guide details seven core data sources for training a cybersecurity LLM—including academic papers, standards like MITRE ATT&CK, CVE/CWE databases, textbooks, community blogs, open datasets, and model distillation via EasyDataset—providing specific acquisition channels and reproducible Python code for each.
The article analyzes why traditional automated test frameworks struggle with micro‑service scale, identifies three hidden sources of AI testing latency, and presents a four‑layer optimization strategy—from data sampling to edge inference—that dramatically improves speed, determinism, and resource elasticity.
Researchers uncovered a critical API vulnerability that lets cheap models decode the hidden chain‑of‑thought reasoning of flagship LLMs like Claude, GPT and Gemini, demonstrating cross‑session, cross‑user, and cross‑model leakage through inexpensive API calls and exposing massive sensitive data leaks.
In a rare internal briefing, ByteDance founder Zhang Yiming ordered the Seed AI team to abandon model distillation as a shortcut for leaderboard rankings, accepting short‑term performance loss to focus on long‑term, self‑reliant AI research amid escalating US‑China tech tensions.
Karpathy argues that 99.9% of large‑model parameters are wasted on low‑quality data, advocates distilling core cognition into sub‑billion models, proposes a hierarchical model ecosystem, and envisions AI as an empowerment tool that reshapes education and lifelong learning.
Jensen Huang’s first X blog post backs open‑weight AI, arguing that open models boost safety, accelerate innovation, and give organizations greater control, while highlighting a shift toward on‑premise deployments, US regulatory scrutiny of Chinese models, and Nvidia’s strategy for supporting both proprietary and open AI workloads.
A Reddit user uncovered that Gemini 3.2 Flash silently went live, delivering single‑prompt code generation of over 2,200 lines—including interactive 3D scenes and a functional Windows 98—thanks to model distillation and sparsification that cut inference cost 15‑20× while approaching GPT‑5.5 performance, and the model is already being integrated with third‑party services ahead of the I/O 2026 showcase.
S‑Agent redefines spatial intelligence by replacing single‑shot answers with a multi‑step action chain that combines a vision‑language model for task planning, specialized depth and pose models for 3D alignment, and a spatial expert that converts geometry into usable evidence, achieving state‑of‑the‑art zero‑shot scores on MMSI‑Bench and ViewSpatial‑Bench and further improvements after distilling 29.2 k trajectories into an 8‑billion‑parameter model.
Gemini 3.2 Flash quietly appeared on the Gemini web interface, letting developers generate massive code (up to 2,200 lines) in a single prompt, leveraging model distillation and sparsification, while integrating services like Canva, Instacart and OpenTable to act as a unified AI assistant.
Google’s Gemini 3.2 Flash model quietly appeared on the web, discovered by a Reddit user, and can be triggered via the Thinking+Canvas mode to generate massive, high‑quality code—over 2,200 lines for complex 3D physics, a PS5 UI, and even a functional Windows 98 environment—while using a distilled, sparsified architecture that cuts inference cost by 15‑20× and integrates seamlessly with apps like Canva, Instacart and OpenTable ahead of the I/O 2026 conference.
Gemini 3.2 Flash silently appeared on the Gemini web UI, was first spotted by a Reddit user, and demonstrates a dramatic jump in code generation—producing up to 2,200 lines of Three.js, SVG, and even a functional Windows 98 environment—thanks to model distillation and sparsification that deliver near‑GPT‑5.5 performance at 15‑20× lower cost, while integrating apps like Canva, Instacart and OpenTable to become a full‑stack AI assistant.
Google’s Gemini 3.2 Flash model slipped into the web silently, letting developers generate massive, interactive code—over 2,200 lines from a single prompt—thanks to aggressive model distillation, sparsification, and new "Thinking+Canvas" routing, while also integrating third‑party services like Canva and Instacart.
Meta has imposed internal limits on engineers' use of external AI coding assistants Claude Code and Codex because the company fears that outputs from these tools could be unintentionally fed into its own model training and evaluation pipelines, raising legal and competitive‑risk concerns.
The article identifies the hidden “privilege illusion” that degrades on‑policy distillation when privileged information is injected, and introduces Dual On‑policy Distillation (DOPD), a dynamic two‑stream approach that separates true ability gaps from information gaps, achieving superior performance and stability across LLM and VLM benchmarks.
Meta has imposed internal limits on engineers’ use of external AI coding assistants Claude Code and Codex, fearing that outputs from these tools could be inadvertently incorporated into Meta’s own training data or evaluation pipelines, raising model‑distillation compliance and cost concerns.
The article details how a transaction platform replaced fragile manual reviews and static rule engines with a data‑flywheel, model‑distillation, and a multi‑agent AI system—automating labeling, dual‑modality risk detection, and anti‑corrosion mechanisms—to achieve a 99.96% reduction in annual financial loss.
Meta is restricting its engineers from using external AI coding tools Claude Code and Codex because the company worries that outputs from these models could unintentionally enter its own training data, creating model‑distillation compliance and cost challenges.
A Reddit user uncovered that Google silently launched Gemini 3.2 Flash, which in Fast + Canvas mode generates over 2,200 lines of code per prompt—far surpassing the previous Flash limits—thanks to aggressive model distillation and sparsification that cut inference cost 15‑20× while approaching GPT‑5.5 performance, and the model is already being integrated with apps like Canva, Instacart and OpenTable ahead of the I/O 2026 conference.
Gemini 3.2 Flash was quietly released on the web, instantly generating thousands of lines of code—including a 2200‑line Three.js demo and a full Windows 98 replica—thanks to model distillation and sparsification that deliver near‑GPT‑5.5 performance with 15‑20× lower inference cost, while also integrating with services like Canva, Instacart and OpenTable ahead of the I/O 2026 conference.
Developers discovered that Gemini 3.2 Flash silently rolled out on the web, instantly generating thousands of lines of code—from interactive 3D scenes to a functional Windows 98—thanks to aggressive model distillation and sparsification, while also integrating third‑party services and reshaping the AI competition ahead of Google I/O 2026.
Gemini 3.2 Flash was quietly released on the web, discovered by a Reddit user, and instantly demonstrated the ability to generate thousands of lines of code—including complex SVG, Three.js scenes, and even a functional Windows 98 environment—thanks to a distilled and sparsified model that rivals GPT‑5.5 performance while cutting inference cost by 15‑20×.
Google quietly released Gemini 3.2 Flash on the web, where developers discovered a hidden model that, when triggered via Thinking + Canvas, generates massive, high‑quality code—up to 2 200 lines for complex 3D, Windows 98, and PS5 UI tasks—while delivering 15‑20× lower inference cost, sub‑200 ms latency, and deep app integrations, marking a major AI industry milestone.
Gemini 3.2 Flash quietly appeared on the web, letting developers trigger a hidden model that writes over a thousand lines of code per prompt, introduces a “thinking level” feature, and achieves near‑GPT‑5.5 performance with dramatically lower inference cost, while Google rolls out deep app integrations ahead of I/O 2026.
Gemini 3.2 Flash slipped onto the Gemini web UI before the I/O event, delivering unprecedented code generation—over 2,200 lines from a single prompt—thanks to hidden model switching, aggressive distillation and sparsification, dramatically lower inference cost, and deep integration with third‑party apps, signaling a major AI product shift.
Gemini 3.2 Flash quietly appeared on the web, discovered by a Reddit user, and can be triggered by selecting Thinking + Canvas mode, instantly generating thousands of lines of sophisticated code—from SVG UI designs to Three.js 3D scenes—while claiming performance near GPT‑5.5 with 15‑20× lower inference cost and deep integration with third‑party apps ahead of the I/O conference.
Gemini 3.2 Flash quietly appeared on the web, was spotted by a Reddit user, can be triggered via Thinking + Canvas, generates thousands of lines of code in a single prompt, relies on model distillation and sparsification, and integrates third‑party apps like Canva and Instacart as Google prepares its I/O 2026 showdown.
Google’s Gemini 3.2 Flash model quietly surfaced online, instantly generating thousands of lines of complex code, outperforming its predecessor and even rivaling GPT‑5.5 in benchmarks while cutting inference costs dramatically, and it now powers an all‑in‑one AI assistant that integrates services like Canva, Instacart and OpenTable.
Google quietly released Gemini 3.2 Flash, discovered by a Reddit user, which can generate thousands of lines of code in a single prompt, leverages model distillation and sparsification to match near‑GPT‑5.5 performance while cutting inference cost 15‑20×, and now integrates with apps like Canva, Instacart and OpenTable as an all‑in‑one AI assistant.
Google’s Gemini 3.2 Flash model quietly appeared on the web, delivering unprecedented code generation—over 2,200 lines from a single prompt—thanks to model distillation and sparsification, while cutting inference cost 15‑20× and integrating with apps like Canva, Instacart and OpenTable ahead of the I/O 2026 showcase.
This work analyzes weight‑direction changes in diffusion‑model distillation, proposes a low‑rank rotation adapter (LoRaD) to model those changes, and integrates it into Variational Score Distillation as WaDi, achieving state‑of‑the‑art FID on COCO with only ~10% trainable parameters while generalizing to multiple downstream tasks.
The Qwopus 3.6‑27B‑v2 model reconstructs full step‑by‑step reasoning from compressed Claude outputs using a Trace‑Inverter, creates two high‑quality SFT datasets, and achieves 35.9% token savings, a 2.57‑point accuracy gain on MMLU‑Pro, 75.25% success on SWE‑bench, while running on a single consumer‑grade RTX 5090.
Google’s Gemini 3.2 Flash model quietly appeared before the I/O event, letting a single prompt produce over 2,200 lines of sophisticated code—including interactive 3D scenes and a functional Windows 98—while claiming near‑GPT‑5.5 performance with dramatically lower inference cost and new integrations for Canva, Instacart and OpenTable.
ActDistill, proposed by Tongji University and collaborators, reduces the inference latency, compute consumption, and action-loop speed of Vision‑Language‑Action (VLA) models by selectively distilling action‑relevant knowledge, achieving up to 1.67× speedup while preserving control quality on real robot hardware.
The preview model Qwopus3.6-27B‑v1, distilled from Claude Opus onto Qwen3.6‑27B using SFT with the Unsloth stack and a curated 12 K high‑quality inference sample set, is evaluated on agentic reasoning, front‑end design, and Canvas/WebGL tasks with an RTX 5090, and can be deployed locally via llama.cpp GGUF quantizations with detailed memory guidelines.
This guide explains how to build high‑quality agent training data using ReAct trajectories, synthesize difficult samples with a data‑flywheel, and distill the knowledge into small LLMs on Alibaba Cloud PAI, covering teacher model deployment, EasyDistill installation, data generation, task solving, rubric filtering, and final model deployment.
The article explores how AI model "distillation" can turn personal chat histories into a digital twin that reveals explicit knowledge, thinking patterns, and cognitive blind spots, while outlining practical steps to extract skill lists, mental models, and boundaries from one’s own AI conversations.
The April 7 AI briefing covers OpenAI’s leadership turnover and bold economic reform proposals, China’s AI model usage overtaking the United States, Google’s Gemma 4 achieving 85% of larger models’ scores with a 256K context, Cursor 3 ushering in an agent‑based coding era, and a joint effort by OpenAI, Anthropic and Google to combat model distillation.
The article details how Claude Opus 4.6's chain‑of‑thought data were used to distill the 27‑billion‑parameter Qwen3.5‑27B model with Unsloth and LoRA, achieving full‑context inference on a single RTX 3090/4090, while outlining performance numbers, hyper‑parameter tips, benchmark gains and the trade‑offs of losing multimodal abilities.
Notes from Jeff Dean's recent interviews reveal his engineering philosophy: build large models then distill to efficient flash models, design evaluation sets that expire at 95% saturation to guide architecture optimization, and implement long-context processing via parallel coarse-to-fine retrieval akin to search engines.
Anthropic alleges that DeepSeek, Moonshadow and MiniMax used about 24,000 fake accounts to conduct over 16 million API interactions with Claude, prompting Elon Musk to mock the company's double standards while sparking a broader debate over model‑distillation legality, API‑use contracts, and the shifting competitive dynamics of the global AI industry.
The article explains how inference‑oriented large language models shift the focus from training‑time resources to test‑time computation, detailing scaling laws, verification techniques, reinforcement‑learning pipelines such as DeepSeek‑R1, and methods for distilling reasoning abilities into smaller, consumer‑grade models.
Reinforcement Learninginference computelarge language models
The article explains how TeichAI used Claude‑Opus‑4.5 to generate a high‑quality 250‑sample reasoning dataset and distill the GLM‑4.7‑Flash model into a compact GGUF version that runs on a single consumer‑grade GPU via llama.cpp, detailing the workflow, quantization options, and practical considerations.
This article explains the TLM large‑model development platform and details how knowledge distillation—using soft labels, temperature scaling, and combined loss functions—compresses teacher models into efficient student models, with practical steps and evaluation on the platform.
The LightReasoner paper from Hong Kong University shows that small language models can guide large models on critical reasoning steps, achieving up to 90% faster inference and significant accuracy gains across multiple math benchmarks.
This article explains how a leading e‑commerce platform leveraged large language models to overcome traditional search relevance challenges, detailing the iterative workflow, model distillation, performance gains, deployment results, and future directions for smarter, more accurate product search.
In an interview before the DACon conference, Dr. Feng Ziyong reveals how Glint‑MVT and novel data‑synthesis techniques overcome distribution gaps, improve compositional understanding, and enable billion‑scale, second‑level retrieval for city‑level surveillance, while balancing model efficiency and effectiveness.
Data Synthesiscity surveillanceembedding retrieval
The article explains the technical background and suitable scenarios for Retrieval‑Augmented Generation (RAG) and model fine‑tuning, compares their strengths, discusses how they can be combined, and provides interview‑style Q&A on their capabilities, risks, and differences from model distillation.
Anthropic announced an immediate, worldwide ban on Claude for any entity controlled by Chinese capital, citing legal, regulatory and security risks, and warned that continued access could enable military use or model‑stealing, urging firms to adopt domestic alternatives.
Meta's DINOv3 is a 70‑billion‑parameter self‑supervised visual foundation model trained on 17 billion Instagram images without any labels, introducing dense feature extraction, Gram‑Anchoring to prevent feature collapse, high‑resolution adaptation, and multi‑student distillation that together enable out‑of‑the‑box performance on segmentation, depth estimation, 3D matching, and tracking while surpassing prior models such as DINOv2, CLIP, and SAM.
This guide explains how to use the EasyDistill framework and Alibaba Cloud PAI to distill large language models for high‑quality text generation, covering model deployment, SFT and DPO training data construction, code examples, configuration files, and best practices for achieving resource‑efficient, high‑performance student models.
This article examines generative data‑driven model distillation as a technique that not only compresses large language models but also improves their accuracy, addresses data‑privacy constraints, and reduces computational costs, offering a practical roadmap and real‑world results from a corporate AI platform.
The article explores how JD’s engineers across retail, logistics, and AI divisions use model distillation, data selection, intelligent routing, and advanced voice recognition to improve the 618 shopping festival experience, highlighting real‑world technical challenges, solutions, and the company’s talent development programs.
The article recounts Xiaoting’s journey from a PhD research background to leading JD.com’s ad‑fraud detection, detailing how large language models, reinforcement learning, and model distillation were applied to identify hidden address codes, reduce false‑positive rates to 0.3%, and balance accuracy with real‑time performance in a high‑traffic e‑commerce environment.
This article recounts a JD researcher’s journey from academic data‑mining competitions to building an AI‑driven, LLM‑enhanced anti‑fraud platform that balances detection accuracy, computational cost, and business value in large‑scale e‑commerce advertising.
This article recounts the journey of a JD PhD trainee who transformed academic research on anomaly detection into a production‑grade, LLM‑enhanced anti‑fraud system that identifies concealed address codes in CPS ads, detailing model design, LoRA fine‑tuning, reinforcement learning, distillation, cost‑aware deployment, and lessons learned for scalable ad risk management.
Industry AIReinforcement Learningad fraud detection
This article explains model distillation—a technique that transfers knowledge from large teacher models to compact student models—covering its motivation, core principles, key steps, practical applications, and both its advantages and limitations, illustrating how AI can be made efficient without sacrificing performance.
AI compressionknowledge transfermodel distillation
The JD Exploration Institute paper introduces Omniforce, a human‑centered, cloud‑edge collaborative AutoML system that uses model distillation, dynamic data governance, Bayesian‑optimized training, and edge deployment to cut large‑model training costs by 70% and improve inference speed by 30%, powering the JoyBuild platform for broader AI adoption.
The paper "Omniforce" from JD Exploration Research Institute presents a cloud‑edge collaborative AutoML system that uses model distillation, data governance, Bayesian training optimization, and cloud‑edge cooperation to reduce large‑model training costs by 70% and improve inference efficiency by an average of 30%, offering a reusable technical paradigm for scalable AI deployment.
This article provides a comprehensive overview of DeepSeek's model distillation technology, detailing its definition, key innovations, architecture, training methods, performance gains, and the remaining challenges such as the implicit performance ceiling and multimodal data distillation.
This article introduces DistilQwen2.5-DS3-0324, a distilled language model series that balances rapid inference with strong reasoning by applying a fast‑thinking chain‑of‑thought strategy, details its two‑stage distillation framework, evaluation on diverse benchmarks, and provides code for downloading and using the models.
On April 2, Baidu released its Wenxin X1 large model on the Qianfan platform, offering enterprise users and developers a multimodal, deep‑thinking AI with superior math, coding, and reasoning scores, low token‑price API access, batch inference, one‑click distillation, and rapid RAG/Agent application building.
This article examines how to build and improve reasoning‑capable large language models, explains the definition and use‑cases of reasoning models, details DeepSeek‑R1’s training pipeline, compares four key enhancement methods—including inference‑time scaling, pure RL, SFT + RL, and distillation—and offers budget‑friendly advice.
On February 24, 2025, 360 and Peking University unveiled Tiny‑R1‑32B‑Preview, a medium‑scale inference model that uses only 5% of the parameters yet achieves performance comparable to the 671‑billion‑parameter Deepseek‑R1, with leading results on math, programming, and scientific benchmarks.
DeepSeek‑R1 demonstrates that large‑scale reinforcement learning, especially with the novel Group Relative Policy Optimization and a rule‑based reward scheme, can markedly boost reasoning in LLMs without heavy supervised fine‑tuning, while a brief cold‑start SFT phase, two‑stage alignment, and knowledge distillation further improve performance and efficiency, despite remaining challenges such as language mixing.
The DeepSeek report details rapid model releases from 2024 onward, highlighting innovations such as model distillation, a 671 B MoE architecture, FP8 mixed‑precision, and the Janus‑Pro multimodal framework, while also documenting major cloud and chip providers' integration of these models into their services.
This article explains why the massive DeepSeek V3/R1 model (671 B parameters) is hard to deploy and introduces three key techniques—model distillation, quantization, and fine‑tuning—that can shrink, accelerate, or specialize large models, while outlining their trade‑offs and practical steps.
This article explains what reasoning language models are, outlines their strengths and weaknesses, details DeepSeek R1's three variants and their training pipelines—including pure reinforcement learning, SFT + RL, and distillation—while also discussing inference‑time scaling techniques and related research such as Sky‑T1 and TinyZero.
This article provides an in‑depth technical analysis of DeepSeek’s model distillation technology, covering its core principles, innovative data‑model fusion strategies, architecture design, training optimizations, performance benchmarks, and the remaining challenges of scaling distillation to multimodal tasks.
This article provides an in‑depth overview of DeepSeek’s model distillation technology, covering its definition, core principles, innovative data‑model distillation integration, architecture design, training strategies, performance gains, and the challenges of scaling to multimodal data.
The article explains DeepSeek's model distillation technique, covering its fundamental knowledge‑transfer principles, unique innovations such as data‑model fusion and task‑specific strategies, impressive benchmark results, practical applications in edge and online inference, existing challenges, and future research directions.
This article provides an in‑depth analysis of DeepSeek’s model distillation technology, covering its definition, core principles, innovative strategies, architecture design, training optimizations, benchmark results, efficiency gains, and the remaining challenges of applying distillation to large language models and multimodal data.
A recent study by Fei‑Fei Li’s team shows that using supervised fine‑tuning on the open‑source Qwen2.5‑32B‑Instruct model can replicate and even surpass the reasoning abilities of OpenAI’s o1‑preview at a fraction of the computational cost, demonstrating a cheap yet powerful approach to large‑language‑model development.
The article reviews DeepSeek's breakthrough in inference efficiency, explores the trade‑offs of model distillation, compares open‑source and closed‑source ecosystems, examines shifting compute demands, highlights Chinese engineering innovations, and outlines future directions for AI development.
This article presents DeepSeek‑R1 and DeepSeek‑R1‑Zero, two next‑generation LLMs trained with pure reinforcement learning and multi‑stage fine‑tuning, details their GRPO training framework, model‑distillation pipeline, open‑source release, and evaluation results that rival OpenAI’s o1‑1217 across reasoning, knowledge, and coding benchmarks.
OpenAI and Anthropic allege that DeepSeek has illegally distilled their large language models, prompting investigations, industry satire, and a detailed look at model distillation technology, its legal implications, and the broader trends shaping AI cost, scaling laws, and market dynamics.
DeepSeek‑R1’s open‑source series demonstrates that reinforcement‑learning‑only training can match top‑tier models like OpenAI‑o1, while a small amount of SFT further improves readability; the article dissects its technical report, training pipeline, reward design, distillation strategy, benchmark outcomes, and remaining challenges.
This guide walks you through the full workflow of using DistilQwen2 on Alibaba Cloud's PAI platform, covering environment setup, model deployment, fine‑tuning with SFT/DPO, evaluation, compression, and distillation, while providing practical code snippets and resource links.
The article explains how homologous small language models—trained on the same tokenizer and data as their large counterparts—serve as cheap, fast experimental platforms that can predict large‑model performance, guide pre‑training decisions, and support techniques like distillation and reward modeling.
The article provides an in‑depth analysis of LLaMA 3 405B, covering its dense Transformer architecture, three‑stage pre‑training (initial, long‑context, annealing), iterative post‑training with RM‑guided rejection sampling, the decision against MOE, and the broader implications for both large and small model development.
The article presents Ant Group's research on integrating large language models into recommendation pipelines, covering background challenges, knowledge extraction, teacher‑model distillation, efficient deployment, experimental results, and future directions to improve accuracy and reduce bias.
Hyper‑SD introduces a trajectory‑segmented consistency distillation framework that combines trajectory‑preserving and trajectory‑reconstruction strategies, integrates human‑feedback learning and score distillation, and achieves state‑of‑the‑art low‑step image generation performance on both SD1.5 and SDXL models.
SDXL‑Lightning, a new diffusion‑based text‑to‑image model from ByteDance, uses Progressive Adversarial Distillation to cut inference steps to as few as 2 while maintaining high resolution and fidelity, offering ten‑fold speed gains, open‑source access, and compatibility with SDXL, ControlNet, and ComfyUI.
Baidu's Search Architecture team details how its deep‑learning models have evolved to deliver direct answer results via semantic embeddings, describes a massive online inference pipeline that rewrites queries, ranks relevance, and classifies types, and outlines optimization techniques—including data I/O, CPU/GPU balancing, pruning, quantization, and distillation—to achieve high‑throughput, low‑latency search.
The article chronicles Baidu Search’s Model Architecture Group’s evolution of deep‑learning‑driven search, detailing the shift from inverted‑index to semantic vector indexing, the use of transformer‑based models for text and image queries, large‑scale offline/online pipelines, and extensive GPU‑centric optimizations such as pruning, quantization and distillation, all aimed at delivering precise, cost‑effective results to hundreds of millions of users.
DAMO‑YOLO is an open‑source, high‑speed and high‑precision object detection framework that leverages MAE‑NAS for low‑cost model customization, Efficient RepGFPN and HeavyNeck for enhanced multi‑scale detection, and a universal distillation technique to boost performance across model scales.
This article presents the QQ Browser search relevance team's experience integrating QQ Browser and Sogou search systems, detailing business overview, relevance system evolution, algorithm architecture, evaluation metrics, deep semantic matching, relevance calibration, and model distillation techniques to improve search relevance performance.
This article describes how Ctrip's R&D team applied deep‑learning models, BERT‑based embeddings, knowledge distillation, and term‑weighting techniques to enhance e‑commerce search intent recognition and term importance estimation, achieving high accuracy while meeting sub‑10 ms latency requirements.
The paper presents CONFLUX, a request-level fusion ranking framework that uses linear programming and cascade distillation to allocate ad impressions between contract and real-time bidding ads, improving platform revenue and ad effectiveness while addressing offline training, latency, and model drift challenges.
The article explains how transfer learning reduces data and time requirements in deep learning by reusing pretrained models for vision, natural language processing, and reinforcement learning, while discussing challenges such as overfitting, the need for progressive networks, entropy regularization, domain adaptation, multi‑task learning, and model distillation.
This article details the business background, system architecture, coarse‑ranking algorithms, dual‑tower model, model distillation, diversity‑control techniques such as DPP, and the online performance gains of the recommendation pipeline used in Tencent Music's Quanmin K‑Song platform.
FastReID, an open‑source PyTorch library from JD AI Research, offers a modular architecture, model distillation, automatic hyper‑parameter search, and multi‑task support, enabling efficient large‑scale object re‑identification across diverse applications such as security, retail, and smart infrastructure.