BestHub
Sign in
Tagged articles
33 articles
Page 1 of 1
Machine Heart
Machine Heart
May 14, 2026 · Artificial Intelligence

Breaking Homogeneous Reasoning: I²B‑LPO Guides RLVR from Repeated Sampling to Effective Exploration

I²B‑LPO is an exploration‑enhancement framework for RLVR that branches rollouts at high‑entropy nodes, injects latent variables via pseudo self‑attention, and filters paths with an information‑bottleneck self‑reward, achieving up to 5.3% accuracy and 7.4% diversity improvements on multiple math reasoning benchmarks.

RLVRentropyexploration
0 likes · 14 min read
Breaking Homogeneous Reasoning: I²B‑LPO Guides RLVR from Repeated Sampling to Effective Exploration
Model Perspective
Model Perspective
Dec 3, 2025 · Fundamentals

Why Shannon’s 1948 Theory Still Powers Modern Communication

This article explains Claude Shannon’s groundbreaking 1948 mathematical theory of communication, covering the essence of communication, the classic Shannon model, bits and entropy, source and channel coding theorems, and why digitizing information remains the most efficient way to transmit data.

Shannoncoding theoremcommunication
0 likes · 9 min read
Why Shannon’s 1948 Theory Still Powers Modern Communication
Baobao Algorithm Notes
Baobao Algorithm Notes
Oct 28, 2025 · Artificial Intelligence

Why Entropy Collapse Limits LLM Reinforcement Learning and How to Fix It

The article explains how information entropy, cross‑entropy, and KL‑divergence shape reinforcement learning for large language models, describes the phenomenon of entropy collapse, compares token‑level and policy‑level entropy, and reviews recent methods like Clip‑Cov and KL‑Cov that mitigate this issue.

cross entropyentropypolicy entropy
0 likes · 11 min read
Why Entropy Collapse Limits LLM Reinforcement Learning and How to Fix It
Data Party THU
Data Party THU
Aug 19, 2025 · Artificial Intelligence

Why RL Fine‑Tuning Fails to Extend LLM Reasoning Limits: Entropy Collapse Explained

This article examines how reinforcement learning fine‑tuning influences large language model reasoning, revealing that RL primarily amplifies pre‑trained capabilities, suffers from entropy collapse, and fails to push the model’s reasoning boundary, supported by extensive experiments on scaling laws, entropy analysis, and mitigation techniques.

LLMRLRLVR
0 likes · 24 min read
Why RL Fine‑Tuning Fails to Extend LLM Reasoning Limits: Entropy Collapse Explained
Model Perspective
Model Perspective
Jan 24, 2025 · Fundamentals

Mastering Weight Generation and Multi‑Criteria Evaluation Methods for Modeling Competitions

This article reviews common weight‑generation and multi‑criteria evaluation techniques—including AHP, entropy, TOPSIS, fuzzy evaluation, CRITIC, PCA, factor analysis, and DEA—explaining their mathematical foundations, advantages, and drawbacks to help competition participants choose the most suitable method for complex decision problems.

AHPDEAMulti-Criteria Evaluation
0 likes · 16 min read
Mastering Weight Generation and Multi‑Criteria Evaluation Methods for Modeling Competitions
Model Perspective
Model Perspective
Sep 7, 2024 · Fundamentals

How Choices Reduce Uncertainty: The Hidden Role of Information Entropy

Every daily decision—from picking clothes to selecting a menu item—acts as an entropy‑reducing process, and this article explains how information theory’s concepts of entropy, entropy reduction, and information gain illuminate the nature of choice, free will, and optimal decision‑making.

Information Gainchoice optimizationdecision making
0 likes · 7 min read
How Choices Reduce Uncertainty: The Hidden Role of Information Entropy
Model Perspective
Model Perspective
Mar 29, 2024 · Fundamentals

How Reducing Information Entropy Can Simplify Your Life

In an age of information overload, understanding and applying the concept of entropy to filter and reject unnecessary data helps lower uncertainty, reduce mental load, and enable more focused, efficient decision‑making for a simpler, higher‑quality life.

attention managemententropyinformation theory
0 likes · 9 min read
How Reducing Information Entropy Can Simplify Your Life
DevOps
DevOps
Mar 11, 2024 · R&D Management

Understanding Business and Personal Debt: Structured Thinking and Management

The article explores the concept of debt in both software projects and personal health, using entropy, the 5‑why method, communication theory, and structured thinking to illustrate how recognizing, analyzing, and systematically addressing such debts can improve system reliability and personal well‑being.

Technical Debtcommunication debtentropy
0 likes · 18 min read
Understanding Business and Personal Debt: Structured Thinking and Management
21CTO
21CTO
Jan 25, 2024 · Fundamentals

Why Software Architecture Fails: Entropy, Broken Windows, and Sustainable Design

This article explains how entropy, the broken‑window effect, and accumulated complexity erode software systems, compares behavior and architecture value dimensions, and presents fundamental principles such as Ockham's razor, SOLID, layering, and anti‑corruption layers to build maintainable, low‑cost architectures.

SOLIDSoftware Architecturecomplexity
0 likes · 17 min read
Why Software Architecture Fails: Entropy, Broken Windows, and Sustainable Design
DevOps
DevOps
Jan 12, 2024 · Operations

Why Building a Never‑Failing System Is Impossible and How to Pursue Continuous High Availability

The article analyses why truly never‑failing systems cannot exist—citing entropy and Murphy’s laws—examines the organizational and technical obstacles to continuous high availability, and offers practical cultural and engineering practices such as testing, code review, monitoring, and regular system health checks to mitigate risk.

Murphy's LawOperationsSRE
0 likes · 14 min read
Why Building a Never‑Failing System Is Impossible and How to Pursue Continuous High Availability
Tencent Cloud Developer
Tencent Cloud Developer
Jan 10, 2024 · Operations

The Challenges of Building Continuously Available Systems: Entropy, Murphy's Law, and the 'Divine Doctor Paradox'

Building continuously available systems in 2023 is hampered by entropy‑driven technical debt and Murphy’s Law failures, and the “Divine Doctor Paradox” shows that successful availability work goes unnoticed while blame follows any outage, making cultural commitment—not just technology—the essential solution.

Murphy's LawSRETechnical Debt
0 likes · 14 min read
The Challenges of Building Continuously Available Systems: Entropy, Murphy's Law, and the 'Divine Doctor Paradox'
Model Perspective
Model Perspective
Dec 7, 2023 · Fundamentals

Why Entropy Always Increases: Understanding the Second Law of Thermodynamics

The article explains the entropy increase law (the second law of thermodynamics), illustrating how isolated systems become more disordered, why heat flows from hot to cold, and how everyday examples like cooling coffee or refrigerators demonstrate local order at the cost of greater overall disorder.

Physicsentropyheat transfer
0 likes · 6 min read
Why Entropy Always Increases: Understanding the Second Law of Thermodynamics
FunTester
FunTester
Nov 27, 2023 · Industry Insights

From Software Debt to Body Debt: Entropy, 5 Whys, and Structured Thinking

The article explores why technical and business debt inevitably grow, using entropy theory, 5‑why analysis, Conway's law, and communication complexity, then draws parallels to personal health, offering structured thinking methods to identify, prioritize, and eliminate hidden liabilities in both software projects and everyday life.

5 why analysisSoftware EngineeringTechnical Debt
0 likes · 20 min read
From Software Debt to Body Debt: Entropy, 5 Whys, and Structured Thinking
Tencent Cloud Developer
Tencent Cloud Developer
Nov 21, 2023 · Fundamentals

Understanding Business Debt and Physical Debt: Structured Thinking and Management

The article argues that both business and personal health debts inevitably accumulate like entropy, but can be managed through structured thinking—identifying, categorizing, prioritizing, and continuously monitoring and feedback‑driven actions such as mindfulness, exercise, and disciplined system maintenance—to reduce complexity and sustain growth.

Conway's lawFeedback LoopTechnical Debt
0 likes · 18 min read
Understanding Business Debt and Physical Debt: Structured Thinking and Management
FunTester
FunTester
Nov 5, 2023 · Backend Development

Why java.util.UUID.randomUUID Can Block Threads and How to Fix It

The article explains how java.util.UUID.randomUUID relies on OS entropy, can block threads when entropy is low, shows a real‑world thread‑dump example, and provides three remedies: upgrading the JDK, installing haveged, or switching to /dev/urandom.

BlockingJDKJava
0 likes · 5 min read
Why java.util.UUID.randomUUID Can Block Threads and How to Fix It
DaTaobao Tech
DaTaobao Tech
May 22, 2023 · Artificial Intelligence

Statistical and Machine Learning Metrics for Data Analysis

The article presents a practical toolbox of statistical and machine‑learning metrics—including short‑term growth rates, CAGR, Excel forecasting functions, Wilson score adjustment, sigmoid decay weighting, correlation coefficients, KL divergence, elbow detection with KneeLocator, entropy‑based weighting, PCA, and TF‑IDF—offering concise formulas and code snippets for data analysis without deep theory.

PCAcorrelationdata analysis
0 likes · 12 min read
Statistical and Machine Learning Metrics for Data Analysis
Model Perspective
Model Perspective
Oct 8, 2022 · Fundamentals

What Is Entropy? From Thermodynamics to Information Theory Explained

This article explores the concept of entropy, tracing its origins from the second law of thermodynamics through Shannon’s information entropy, illustrating its mathematical definition, physical meaning, and practical examples, and even discusses how the principle can be applied to personal growth and organizational management.

PhysicsShannon entropyentropy
0 likes · 13 min read
What Is Entropy? From Thermodynamics to Information Theory Explained
Python Programming Learning Circle
Python Programming Learning Circle
Sep 15, 2022 · Fundamentals

Time Series Analysis in Python: Visualization, FFT, Entropy, PCA and Autocorrelation

This article demonstrates how to analyze and visualize time‑series sensor data in Python using libraries such as NumPy, Pandas, Matplotlib, Seaborn and Scikit‑learn, covering data import, preprocessing, mean‑std plots, boxplots, Fourier transforms, entropy calculation, PCA dimensionality reduction and autocorrelation analysis.

Data visualizationFFTPCA
0 likes · 17 min read
Time Series Analysis in Python: Visualization, FFT, Entropy, PCA and Autocorrelation
Model Perspective
Model Perspective
Jun 19, 2022 · Artificial Intelligence

How Decision Trees Work: From Entropy to Gini Index Explained

This article introduces decision tree algorithms, explains their role in supervised learning for classification and regression, details the construction process, compares information gain and Gini index for attribute selection, and reviews popular tree methods such as ID3, C4.5, and CART with illustrative examples.

C4.5CARTGini Index
0 likes · 7 min read
How Decision Trees Work: From Entropy to Gini Index Explained
Model Perspective
Model Perspective
Jun 13, 2022 · Artificial Intelligence

Understanding Decision Trees: From Basic Process to Watermelon Example

This article explains the fundamentals of decision tree learning, describing its recursive construction, the criteria for splitting nodes using information gain based on entropy, and walks through a classic watermelon dataset example to illustrate how attributes are selected and the final tree is built.

ID3 algorithmInformation Gainclassification
0 likes · 8 min read
Understanding Decision Trees: From Basic Process to Watermelon Example
Code DAO
Code DAO
May 6, 2022 · Fundamentals

Information Theory Foundations for Machine Learning and Deep Learning

The article explains Shannon information content, entropy, cross‑entropy, KL‑divergence, conditional entropy and mutual information, illustrating each concept with coin‑flip and dice examples, visual formulas, and discusses their roles as loss functions and evaluation metrics in machine‑learning models.

KL divergencecross entropyentropy
0 likes · 8 min read
Information Theory Foundations for Machine Learning and Deep Learning
KooFE Frontend Team
KooFE Frontend Team
Sep 2, 2021 · R&D Management

How Conway’s Law and Entropy Shape Modern Tech Team Structures

The article explores how Conway’s Law and the entropy increase principle explain the dynamic nature of organizational structures in tech companies, highlighting the impact of communication patterns on system design, resource allocation, and team efficiency.

Conway's lawSoftware Architectureentropy
0 likes · 6 min read
How Conway’s Law and Entropy Shape Modern Tech Team Structures
Python Programming Learning Circle
Python Programming Learning Circle
Aug 10, 2021 · Artificial Intelligence

Building a Decision Tree Model in Python Using Entropy and Gini Impurity

This tutorial walks through creating, visualizing, and exporting two Python decision‑tree classifiers—one using entropy and the other using Gini impurity—by installing required packages, preparing a simple dataset, training the models with scikit‑learn, and rendering the trees with Graphviz.

Gini ImpurityGraphvizdecision tree
0 likes · 11 min read
Building a Decision Tree Model in Python Using Entropy and Gini Impurity
Tongcheng Travel Technology Center
Tongcheng Travel Technology Center
Aug 4, 2021 · Backend Development

Designing Complex System Architecture: From Entropy to Microservices and the Middle Platform

The article explains how the thermodynamic entropy principle inspires software architecture design, emphasizes abstract thinking and domain‑driven design, compares monolithic and microservice approaches, outlines microservice benefits and distributed‑system design principles, and discusses the complementary role of middle‑platform architecture in modern backend development.

Domain-Driven DesignMicroservicesSoftware Architecture
0 likes · 9 min read
Designing Complex System Architecture: From Entropy to Microservices and the Middle Platform
NetEase Smart Enterprise Tech+
NetEase Smart Enterprise Tech+
Mar 9, 2021 · Information Security

How Server‑Side Device Fingerprinting Boosts Security and Stability

Device fingerprinting uniquely identifies devices using collected data; this article explains how uniqueness and stability are measured, shows probability‑based calculations for single and combined fields, discusses the shortcomings of client‑side methods, and details a server‑side multi‑algorithm approach that improves security and stability.

Securityanti-frauddecision tree
0 likes · 11 min read
How Server‑Side Device Fingerprinting Boosts Security and Stability
Programmer DD
Programmer DD
Dec 3, 2019 · Operations

Why SSH Login Stalls on New UCloud Hosts: Entropy Delays and Kernel Bugs

A user reported extremely slow first SSH login on a UCloud “High‑Kernel Ubuntu 18.04” instance, which was traced to slow entropy‑pool initialization in the Linux kernel, a libssl 1.1.1 bug, and a kernel configuration issue, leading to a fix involving disabling MOTD or enabling CONFIG_RANDOM_TRUST_CPU.

Linux kernelSSHcloud hosting
0 likes · 11 min read
Why SSH Login Stalls on New UCloud Hosts: Entropy Delays and Kernel Bugs
DevOpsClub
DevOpsClub
May 25, 2018 · Backend Development

What Entropy Theory Reveals About Software Architecture Evolution and Technical Debt

This article applies the thermodynamic concept of entropy to software systems, explaining how entropy increase manifests as technical debt and how deliberate entropy reduction through architectural evolution, micro‑services, DevOps practices, and organizational changes can sustain system vitality and productivity.

MicroservicesTechnical Debtentropy
0 likes · 18 min read
What Entropy Theory Reveals About Software Architecture Evolution and Technical Debt
DevOps
DevOps
May 23, 2018 · Operations

Applying the Second Law of Thermodynamics to Software Architecture, Technical Debt, and Evolution

The article explores how the concept of entropy from the second law of thermodynamics maps onto software systems and organizations, describing entropy increase and reduction, negative entropy, four governing rules, technical debt, progressive architecture patterns such as the strangler and refactoring approaches, and real‑world examples like Amazon’s evolution to microservices.

DevOpsMicroservicesTechnical Debt
0 likes · 17 min read
Applying the Second Law of Thermodynamics to Software Architecture, Technical Debt, and Evolution
Tencent Cloud Developer
Tencent Cloud Developer
Mar 19, 2018 · Artificial Intelligence

Basic Concepts of Decision Trees

Decision trees are tree-structured classifiers that split data using attributes chosen for maximal purity measured by Gini impurity or entropy, with algorithms like ID3 selecting splits by information gain, while overfitting is mitigated through constraints and pruning techniques such as REP, PEP, and CCP.

Gini ImpurityID3Information Gain
0 likes · 13 min read
Basic Concepts of Decision Trees
21CTO
21CTO
Feb 7, 2018 · Artificial Intelligence

Demystifying Entropy: From Basic Concepts to Cross‑Entropy and KL Divergence

This article explains entropy, joint entropy, conditional entropy, and related measures such as KL divergence and cross‑entropy, using intuitive coin‑flip examples and mathematical formulas to show how they quantify uncertainty and information in probability distributions.

KL divergencecross entropyentropy
0 likes · 14 min read
Demystifying Entropy: From Basic Concepts to Cross‑Entropy and KL Divergence