Tagged articles
entropy
39 articles · Page 1 of 1
The article traces mandatory complex password rules to Bill Burr's 2003 NIST guide based on a 1980s paper, reveals how they create predictable patterns, cites xkcd's entropy comparison, and notes Burr's 2017 regret and NIST's updated guidelines, yet many sites still enforce the outdated rules.
AuthenticationBill BurrNIST
0 likes · 7 min read

A detailed technical investigation reveals that a simple macro‑comparison bug in Coldcard firmware reduced entropy to about 40 bits, enabling attackers to enumerate seed space, derive vulnerable addresses, and steal roughly $38 million in Bitcoin within minutes, while the hardware TRNG remained unused.
ColdcardRNGSecurity
0 likes · 18 min read

The article builds an information‑theoretic framework for AI coding, showing how entropy, conditional entropy and mutual information explain why detailed prompts still fail, why new projects succeed more easily than legacy code, and how memory, retrieval and harness engineering can be evaluated to reduce the model's guesswork.
AI codingAgent MemoryHarness Engineering
0 likes · 23 min read


TonyBaiJun 25, 2026 · Information Security
In Go 1.27 the standard uuid.NewV7() function produces UUIDs with a constant “7000” segment when compiled to WebAssembly, because browsers deliberately reduce high‑precision timers for Spectre‑style mitigations, causing the 12‑bit random field to collapse to zero and dramatically lowering entropy and collision resistance.
GoUUIDv7Wasm
0 likes · 10 min read

I²B‑LPO is an exploration‑enhancement framework for RLVR that branches rollouts at high‑entropy nodes, injects latent variables via pseudo self‑attention, and filters paths with an information‑bottleneck self‑reward, achieving up to 5.3% accuracy and 7.4% diversity improvements on multiple math reasoning benchmarks.
Math ReasoningRLVRentropy
0 likes · 14 min read

The article argues that Software Engineering 3.0 is fundamentally a continuous entropy‑reduction process, using information theory to explain how large language models turn ambiguous human intent into precise executable code, why knowledge graphs outperform documents, and how multi‑agent systems improve quality assurance.
AI agentsentropyinformation theory
0 likes · 11 min read

This article explains Claude Shannon’s groundbreaking 1948 mathematical theory of communication, covering the essence of communication, the classic Shannon model, bits and entropy, source and channel coding theorems, and why digitizing information remains the most efficient way to transmit data.
Shannoncoding theoremcommunication
0 likes · 9 min read

The article examines how shortcut tools that boost efficiency can reduce expressive diversity by lowering choice costs, creating path‑dependence, reinforcing the Matthew effect, and leveraging cognitive anchoring and network effects, ultimately eroding cultural creativity and value rationality.
cognitive biasentropyexpression diversity
0 likes · 6 min read

The article explains how information entropy, cross‑entropy, and KL‑divergence shape reinforcement learning for large language models, describes the phenomenon of entropy collapse, compares token‑level and policy‑level entropy, and reviews recent methods like Clip‑Cov and KL‑Cov that mitigate this issue.
cross-entropyentropypolicy entropy
0 likes · 11 min read

The article explores how AI‑generated code can be both fascinating and frustrating, using control theory concepts such as variety, the law of requisite variety, and Shannon entropy to explain the hidden risks and propose ways to manage complexity in software development.
AIcode generationcontrol theory
0 likes · 9 min read

This article examines how reinforcement learning fine‑tuning influences large language model reasoning, revealing that RL primarily amplifies pre‑trained capabilities, suffers from entropy collapse, and fails to push the model’s reasoning boundary, supported by extensive experiments on scaling laws, entropy analysis, and mitigation techniques.
LLMRLRLVR
0 likes · 24 min read

The article argues that software engineering is fundamentally a discipline for managing complexity, entropy, and information flow, and that understanding its first‑principles—trade‑offs, abstraction, process, feedback, and automation—is essential to avoid ritualistic practices and deliver sustainable value.
ComplexityFeedbackabstraction
0 likes · 10 min read

This article reviews common weight‑generation and multi‑criteria evaluation techniques—including AHP, entropy, TOPSIS, fuzzy evaluation, CRITIC, PCA, factor analysis, and DEA—explaining their mathematical foundations, advantages, and drawbacks to help competition participants choose the most suitable method for complex decision problems.
AHPDEAMulti-Criteria Evaluation
0 likes · 16 min read

Every daily decision—from picking clothes to selecting a menu item—acts as an entropy‑reducing process, and this article explains how information theory’s concepts of entropy, entropy reduction, and information gain illuminate the nature of choice, free will, and optimal decision‑making.
Information Gainchoice optimizationdecision making
0 likes · 7 min read

In an age of information overload, understanding and applying the concept of entropy to filter and reject unnecessary data helps lower uncertainty, reduce mental load, and enable more focused, efficient decision‑making for a simpler, higher‑quality life.
attention managemententropyinformation theory
0 likes · 9 min read


DevOpsMar 11, 2024 · R&D Management
The article explores the concept of debt in both software projects and personal health, using entropy, the 5‑why method, communication theory, and structured thinking to illustrate how recognizing, analyzing, and systematically addressing such debts can improve system reliability and personal well‑being.
communication debtentropyfeedback loops
0 likes · 18 min read


21CTOJan 25, 2024 · Fundamentals
This article explains how entropy, the broken‑window effect, and accumulated complexity erode software systems, compares behavior and architecture value dimensions, and presents fundamental principles such as Ockham's razor, SOLID, layering, and anti‑corruption layers to build maintainable, low‑cost architectures.
ComplexityDesign PrinciplesSOLID
0 likes · 17 min read


DevOpsJan 12, 2024 · Operations
The article analyses why truly never‑failing systems cannot exist—citing entropy and Murphy’s laws—examines the organizational and technical obstacles to continuous high availability, and offers practical cultural and engineering practices such as testing, code review, monitoring, and regular system health checks to mitigate risk.
High AvailabilityMurphy's LawOperations
0 likes · 14 min read

Building continuously available systems in 2023 is hampered by entropy‑driven technical debt and Murphy’s Law failures, and the “Divine Doctor Paradox” shows that successful availability work goes unnoticed while blame follows any outage, making cultural commitment—not just technology—the essential solution.
High AvailabilityMurphy's LawSRE
0 likes · 14 min read

The article explains the entropy increase law (the second law of thermodynamics), illustrating how isolated systems become more disordered, why heat flows from hot to cold, and how everyday examples like cooling coffee or refrigerators demonstrate local order at the cost of greater overall disorder.
Physicsentropyheat transfer
0 likes · 6 min read

The article explores why technical and business debt inevitably grow, using entropy theory, 5‑why analysis, Conway's law, and communication complexity, then draws parallels to personal health, offering structured thinking methods to identify, prioritize, and eliminate hidden liabilities in both software projects and everyday life.
5 why analysisbusiness communicationentropy
0 likes · 20 min read

The article argues that both business and personal health debts inevitably accumulate like entropy, but can be managed through structured thinking—identifying, categorizing, prioritizing, and continuously monitoring and feedback‑driven actions such as mindfulness, exercise, and disciplined system maintenance—to reduce complexity and sustain growth.
Conway's LawHealth Managemententropy
0 likes · 18 min read

The article explains how java.util.UUID.randomUUID relies on OS entropy, can block threads when entropy is low, shows a real‑world thread‑dump example, and provides three remedies: upgrading the JDK, installing haveged, or switching to /dev/urandom.
JDKJavaUUID
0 likes · 5 min read

The article presents a practical toolbox of statistical and machine‑learning metrics—including short‑term growth rates, CAGR, Excel forecasting functions, Wilson score adjustment, sigmoid decay weighting, correlation coefficients, KL divergence, elbow detection with KneeLocator, entropy‑based weighting, PCA, and TF‑IDF—offering concise formulas and code snippets for data analysis without deep theory.
PCAcorrelationdata analysis
0 likes · 12 min read

This article explores the concept of entropy, tracing its origins from the second law of thermodynamics through Shannon’s information entropy, illustrating its mathematical definition, physical meaning, and practical examples, and even discusses how the principle can be applied to personal growth and organizational management.
PhysicsShannon entropyentropy
0 likes · 13 min read

This article demonstrates how to analyze and visualize time‑series sensor data in Python using libraries such as NumPy, Pandas, Matplotlib, Seaborn and Scikit‑learn, covering data import, preprocessing, mean‑std plots, boxplots, Fourier transforms, entropy calculation, PCA dimensionality reduction and autocorrelation analysis.
FFTPCAPython
0 likes · 17 min read

This article introduces decision tree algorithms, explains their role in supervised learning for classification and regression, details the construction process, compares information gain and Gini index for attribute selection, and reviews popular tree methods such as ID3, C4.5, and CART with illustrative examples.
C4.5CARTGini Index
0 likes · 7 min read

This article explains the fundamentals of decision tree learning, describing its recursive construction, the criteria for splitting nodes using information gain based on entropy, and walks through a classic watermelon dataset example to illustrate how attributes are selected and the final tree is built.
ClassificationID3 algorithmInformation Gain
0 likes · 8 min read

The article explains Shannon information content, entropy, cross‑entropy, KL‑divergence, conditional entropy and mutual information, illustrating each concept with coin‑flip and dice examples, visual formulas, and discusses their roles as loss functions and evaluation metrics in machine‑learning models.
KL Divergencecross-entropyentropy
0 likes · 8 min read

The article explores how Conway’s Law and the entropy increase principle explain the dynamic nature of organizational structures in tech companies, highlighting the impact of communication patterns on system design, resource allocation, and team efficiency.
Conway's Lawentropyorganizational structure
0 likes · 6 min read

This tutorial walks through creating, visualizing, and exporting two Python decision‑tree classifiers—one using entropy and the other using Gini impurity—by installing required packages, preparing a simple dataset, training the models with scikit‑learn, and rendering the trees with Graphviz.
Gini ImpurityGraphvizdecision tree
0 likes · 11 min read

The article explains how the thermodynamic entropy principle inspires software architecture design, emphasizes abstract thinking and domain‑driven design, compares monolithic and microservice approaches, outlines microservice benefits and distributed‑system design principles, and discusses the complementary role of middle‑platform architecture in modern backend development.
Design Principlesdomain-driven designentropy
0 likes · 9 min read

Device fingerprinting uniquely identifies devices using collected data; this article explains how uniqueness and stability are measured, shows probability‑based calculations for single and combined fields, discusses the shortcomings of client‑side methods, and details a server‑side multi‑algorithm approach that improves security and stability.
Anti-FraudSecurityServer-Side
0 likes · 11 min read

A user reported extremely slow first SSH login on a UCloud “High‑Kernel Ubuntu 18.04” instance, which was traced to slow entropy‑pool initialization in the Linux kernel, a libssl 1.1.1 bug, and a kernel configuration issue, leading to a fix involving disabling MOTD or enabling CONFIG_RANDOM_TRUST_CPU.
Linux kernelSSHSystem Performance
0 likes · 11 min read

This article applies the thermodynamic concept of entropy to software systems, explaining how entropy increase manifests as technical debt and how deliberate entropy reduction through architectural evolution, micro‑services, DevOps practices, and organizational changes can sustain system vitality and productivity.
entropymicroservicestechnical debt
0 likes · 18 min read


DevOpsMay 23, 2018 · Operations
The article explores how the concept of entropy from the second law of thermodynamics maps onto software systems and organizations, describing entropy increase and reduction, negative entropy, four governing rules, technical debt, progressive architecture patterns such as the strangler and refactoring approaches, and real‑world examples like Amazon’s evolution to microservices.
DevOpsentropymicroservices
0 likes · 17 min read

Decision trees are tree-structured classifiers that split data using attributes chosen for maximal purity measured by Gini impurity or entropy, with algorithms like ID3 selecting splits by information gain, while overfitting is mitigated through constraints and pruning techniques such as REP, PEP, and CCP.
AlgorithmGini ImpurityID3
0 likes · 13 min read


21CTOFeb 7, 2018 · Artificial Intelligence
This article explains entropy, joint entropy, conditional entropy, and related measures such as KL divergence and cross‑entropy, using intuitive coin‑flip examples and mathematical formulas to show how they quantify uncertainty and information in probability distributions.
KL Divergencecross-entropyentropy
0 likes · 14 min read

This article explains the concepts of entropy, joint entropy, conditional entropy, relative entropy (KL divergence) and cross‑entropy, illustrating their definitions, mathematical formulas, intuitive interpretations, and relationships through simple probability examples and visual diagrams.
KL Divergencecross-entropyentropy
0 likes · 14 min read
