Tagged articles
information theory
39 articles · Page 1 of 1
The article analyzes why AI-assisted coding accelerates code generation but fails to proportionally improve delivery efficiency, using information theory to explain bottlenecks in requirement clarification, verification capacity, and code rot, and proposes organizational and platform changes to convert coding speed into reliable delivery.
AI codingAI-native organizationAmdahl's Law
0 likes · 60 min read

FlowMAS, a NeurIPS 2026 paper, uses Generative Flow Networks with curiosity-driven and information-theoretic rewards to automatically generate diverse, efficient multi-agent workflow topologies, outperforming baselines across six benchmarks with far lower computational cost.
Automated Architecture DesignCuriosity-Driven ExplorationGFlowNets
0 likes · 11 min read

The article explains how a binary‑encoding strategy lets you identify a single poisoned bottle out of 1000 using only ten mice by assigning each mouse to a bit position, sampling wines accordingly, and interpreting the death pattern as a 10‑bit address.
BitmapBloom Filterbinary
0 likes · 7 min read

The article builds an information‑theoretic framework for AI coding, showing how entropy, conditional entropy and mutual information explain why detailed prompts still fail, why new projects succeed more easily than legacy code, and how memory, retrieval and harness engineering can be evaluated to reduce the model's guesswork.
AI codingAgent MemoryHarness Engineering
0 likes · 23 min read

The article dissects the Silly Big Personality Test (SBTI), explaining its 15‑dimensional Euclidean space, threshold‑based hierarchical clustering, statistical tricks like the Barnum effect, and its viral spread as a social meme, while highlighting the scientific limits of such entertainment‑focused assessments.
Barnum effectPsychologyclustering
0 likes · 7 min read

InfoTok, a collaborative effort by Stanford, NVIDIA Cosmos, and NUS, leverages information theory and an ELBO‑based router to allocate tokens adaptively, achieving 2.3× higher compression, 11× faster inference, and superior reconstruction quality on benchmarks such as TokenBench and DAVIS.
Computer VisionELBOICLR 2026
0 likes · 11 min read

The article argues that Software Engineering 3.0 is fundamentally a continuous entropy‑reduction process, using information theory to explain how large language models turn ambiguous human intent into precise executable code, why knowledge graphs outperform documents, and how multi‑agent systems improve quality assurance.
AI agentsentropyinformation theory
0 likes · 11 min read

An in‑depth analysis of the OpenClaw‑derived Moltbook AI agent network reveals a “Self‑Evolution Trilemma” where continuous self‑evolution, complete isolation, and perpetual safety cannot coexist, supported by information‑theoretic definitions, empirical observations of cognitive decay, alignment failures, communication collapse, and proposed thermodynamic mitigation strategies.
AI safetySecurityagent networks
0 likes · 9 min read

This article explains Claude Shannon’s groundbreaking 1948 mathematical theory of communication, covering the essence of communication, the classic Shannon model, bits and entropy, source and channel coding theorems, and why digitizing information remains the most efficient way to transmit data.
Shannoncoding theoremcommunication
0 likes · 9 min read

This article builds a quantitative framework for analyzing leadership intent using vector representation, multi‑objective optimization, Shannon‑type information‑theoretic channel models, and signaling games, and illustrates the approach with three Chinese case studies that reveal how intent distortion, high entropy, and strategic misalignment affect execution.
Leadership Intentgame theoryinformation theory
0 likes · 10 min read


ITPUBNov 24, 2025 · Artificial Intelligence
In a detailed interview, the CTO of Memory Tensor (Shanghai) explains how limited memory capacity hampers large models, outlines the MemOS memory operating system, discusses information‑theoretic metrics, multimodal extensions, and reinforcement‑learning strategies for scalable, secure, and explainable AI memory management.
AI Architectureinformation theorylarge language models
0 likes · 23 min read

The article examines how shortcut tools that boost efficiency can reduce expressive diversity by lowering choice costs, creating path‑dependence, reinforcing the Matthew effect, and leveraging cognitive anchoring and network effects, ultimately eroding cultural creativity and value rationality.
cognitive biasentropyexpression diversity
0 likes · 6 min read

This article examines classic Tang poems through modern mathematical modeling, revealing how geometric optics, probability, calculus, information theory, and network analysis can be applied to literary works to uncover the underlying mathematical structures and interdisciplinary insights.
Tang Poetrycalculusgeometry
0 likes · 10 min read

This article explains the Kullback‑Leibler (KL) divergence for discrete and continuous distributions, outlines its non‑negativity and asymmetry, walks through a uniform‑distribution example, provides a simple Python demonstration, and discusses key applications in variational autoencoders, reinforcement‑learning policy optimization, and other machine‑learning contexts.
KL DivergenceVariational AutoEncoderinformation theory
0 likes · 7 min read

This article explains how large language models act like a super‑library by compressing vast amounts of text using information‑theoretic concepts, probability‑based coding, autoregressive neural networks, and arithmetic coding, while discussing accuracy, compression ratios, and theoretical limits.
AIData Compressionarithmetic coding
0 likes · 8 min read

π’s endless, non‑repeating decimal expansion behaves like a universal information reservoir: any finite digit sequence—birthdates, historic years, or arbitrary data—appears somewhere, and if the unproven normal‑number hypothesis holds, π could serve as an infinite random number generator for cryptography, art, and scientific modeling.
Picryptographydata hiding
0 likes · 6 min read

Every daily decision—from picking clothes to selecting a menu item—acts as an entropy‑reducing process, and this article explains how information theory’s concepts of entropy, entropy reduction, and information gain illuminate the nature of choice, free will, and optimal decision‑making.
Information Gainchoice optimizationdecision making
0 likes · 7 min read

In an age of information overload, understanding and applying the concept of entropy to filter and reject unnecessary data helps lower uncertainty, reduce mental load, and enable more focused, efficient decision‑making for a simpler, higher‑quality life.
attention managemententropyinformation theory
0 likes · 9 min read

This article analyzes the fundamental goals of video transcoding, presents an information‑theoretic framework for bitrate reduction, compares traditional and deep‑learning codecs, and shares Bilibili's practical system design, parameter‑decision strategies, and visual‑quality‑aware optimizations that dramatically lower bandwidth consumption.
VMAFbandwidth optimizationbitrate reduction
0 likes · 24 min read

The upcoming Shanghai Jiao‑Tong University seminar on Intelligent Media Technology will feature Bilibili’s Cai Chunlei presenting an information‑theoretic framework for jointly optimizing video transcoding pipelines, linking traditional coding, deep‑learning methods and future large‑model techniques to improve compression and guide practical system design.
AISeminarinformation theory
0 likes · 4 min read

This article explains the hierarchy of digital storage units—from the single bit up to speculative sizes like Brontobyte and beyond—detailing their values, typical examples, and real‑world analogies such as characters per byte and the amount of text each unit can hold.
Data Storagebitsbytes
0 likes · 5 min read

This article explains the concept of information entropy, its mathematical definition, how to compute it for simple events like coin flips and dice rolls, and demonstrates its role in decision‑tree learning by calculating information gain for a watermelon dataset.
Information Gaindecision treeinformation entropy
0 likes · 11 min read

This article recounts how a MIT information‑theory class assignment sparked David Huffman's invention of a bottom‑up binary‑tree coding method that revolutionized lossless data compression and became a cornerstone of modern computer science.
AlgorithmData CompressionHuffman coding
0 likes · 9 min read


21CTOApr 7, 2023 · Cloud Computing
In this thought‑leading talk, Microsoft China CTO Wei Qing explores how first‑principle thinking, information theory, and cloud‑native architectures can guide organizations through the unknown digital era, emphasizing practical engineering, data integration, and the shift from hype to real‑world technology empowerment.
Systems Engineeringcloud computingdigital transformation
0 likes · 42 min read

This article explores the concept of entropy, tracing its origins from the second law of thermodynamics through Shannon’s information entropy, illustrating its mathematical definition, physical meaning, and practical examples, and even discusses how the principle can be applied to personal growth and organizational management.
PhysicsShannon entropyentropy
0 likes · 13 min read

The article explains Shannon information content, entropy, cross‑entropy, KL‑divergence, conditional entropy and mutual information, illustrating each concept with coin‑flip and dice examples, visual formulas, and discusses their roles as loss functions and evaluation metrics in machine‑learning models.
KL Divergencecross-entropyentropy
0 likes · 8 min read


21CTOApr 7, 2022 · R&D Management
In an era dominated by black‑swan events and misinformation, the article explores how leaders can develop resilience through humility, first‑principles thinking, revisiting classic research, and collaborative sense‑making to navigate digital transformation and the challenges of a rapidly evolving information landscape.
Complexitydigital transformationfirst principles
0 likes · 10 min read

This article chronicles Jacob Ziv’s pioneering contributions to lossless data compression, tracing the evolution from early Morse code through Shannon‑Fano and Huffman to the groundbreaking LZ77 and LZ78 algorithms, and highlights his biography, major awards, and lasting influence on modern digital media and storage technologies.
Data CompressionJacob ZivLZ77
0 likes · 9 min read

The article chronicles Jacob Ziv's life and achievements, detailing the development of the LZ77 and LZ78 lossless compression algorithms, their historical context, and the lasting impact of these innovations on modern data storage and transmission technologies.
Data CompressionJacob ZivLempel-Ziv
0 likes · 8 min read


21CTOJul 24, 2021 · Fundamentals
Jacob Ziv, the 90‑year‑old Israeli pioneer honored with the 2021 IEEE Medal of Honor, co‑created the groundbreaking Lempel‑Ziv lossless compression algorithms that underpin modern formats like GIF, PNG and ZIP, and his century‑spanning career reshaped data storage, transmission, and information theory.
Data CompressionJacob ZivLempel-Ziv
0 likes · 10 min read

This article chronicles Alibaba’s three‑year evolution of frontend intelligence, detailing the origins of AI‑driven code generation, key technologies like imgCook, BizCook and UICook, the methodology, toolchain, organizational integration, and future directions for boosting development efficiency and business value.
AIautomationcode generation
0 likes · 26 min read

Former Huawei strategist Xu Wenwei outlined ten critical mathematical problems—from lossy compression limits to DNN interpretability—that must be solved to advance communication technologies beyond Shannon’s limits, highlighting the pivotal role of mathematics in the emerging intelligent, fully connected society.
Huaweicommunicationinformation theory
0 likes · 5 min read


21CTOAug 27, 2018 · Artificial Intelligence
This extensive essay critiques the book "Proving Darwin" while exploring configuration spaces, genetic and simulated‑annealing algorithms, their computational complexities, and how they compare to physical models, then delves into information theory, Turing machines, and philosophical reflections on evolution, life, and the universe.
Complexity Analysisevolutionary computationgenetic algorithms
0 likes · 31 min read


21CTOFeb 7, 2018 · Artificial Intelligence
This article explains entropy, joint entropy, conditional entropy, and related measures such as KL divergence and cross‑entropy, using intuitive coin‑flip examples and mathematical formulas to show how they quantify uncertainty and information in probability distributions.
KL Divergencecross-entropyentropy
0 likes · 14 min read

This article explains the concepts of entropy, joint entropy, conditional entropy, relative entropy (KL divergence) and cross‑entropy, illustrating their definitions, mathematical formulas, intuitive interpretations, and relationships through simple probability examples and visual diagrams.
KL Divergencecross-entropyentropy
0 likes · 14 min read

The article outlines key mathematical areas—probability & statistics, linear algebra, calculus, information theory, and convex optimization—essential for deep learning, and recommends specific lecture notes and textbooks for each topic to guide learners toward effective study.
calculusconvex optimizationdeep learning
0 likes · 4 min read


21CTODec 16, 2017 · Artificial Intelligence
This article reviews recent research that mathematically explains why deep learning, especially convolutional neural networks, achieve remarkable performance by examining core factors such as architecture, regularization, and optimization, and discusses properties like global optimality, geometric stability, and invariant representations.
Generalizationdeep learninginformation theory
0 likes · 16 min read


21CTOSep 22, 2017 · Big Data
Big data’s explosive growth, fueled by the internet and massive data collection, has transformed AI methods, reshaped industries, and sparked a new intelligent revolution by turning uncertainty into actionable insight, illustrating how data volume, multidimensionality, and completeness fundamentally change technology and business models.
data-driveninformation theorytechnology revolution
0 likes · 23 min read

The essay explores how information, translation, and hierarchical structures underpin our perception of reality, linking bio‑hacking, physics, culture, and philosophy while highlighting the quest for a unified key to connect all levels of existence.
CRISPRPhysicsfundamentals
0 likes · 12 min read
