How Information Theory Guides AI Coding: Fighting Entropy to Optimize Prompts and Agents

Introduction

AI coding concepts evolve rapidly (Context Engineering, RAG, Agent Memory, etc.). The author proposes a simple, information‑theoretic framework that helps developers stop chasing every new term and instead focus on the underlying logic of "information" and "uncertainty".

Information‑theoretic Foundations

Three core concepts are introduced:

Entropy H(X) – measures the size of the possibility space of a random variable.

Conditional entropy H(Y|X) – the remaining uncertainty of Y after observing X.

Mutual information I(X;Y) – how much uncertainty X removes from Y.

Examples illustrate these ideas: asking an LLM to write a login function has high H(Y) because the language, authentication method, error handling, etc., are all uncertain. Adding concrete details such as UserService and ApiError reduces H(Y|X) and increases I(X;Y).

A Useful Decomposition

The relationship can be expressed as: H(Y) = I(X;Y) + H(Y|X) In AI coding: H(Y) – total code possibilities determined by the task. I(X;Y) – how much the provided context eliminates uncertainty. H(Y|X) – the part the model must guess from its own prior.

Improving quality means either increasing I(X;Y) (more useful context) or decreasing H(Y|X) (reducing the guesswork).

Real‑world Corrections

1. Low entropy ≠ correctness. LLMs are trained to minimize cross‑entropy on public data, producing a distribution P. During inference they output the most probable token sequence, which can be confidently wrong when the business’s true distribution Q differs from P. Examples show a model calling userClient.getProfile() (common pattern) while the project requires profileService.fetchByUid().

2. Context is not unlimited. Adding more tokens can drown useful signals and introduce contradictory information, raising H(Y|X). Effective context must be high‑density; irrelevant or stale content harms performance.

Two‑layer Failure Model

The framework defines two independent failure dimensions:

Coverage layer – how much the context reduces entropy (metric: I(X;Y)/|Context|).

Fill layer – how well the model’s prior P matches the real business distribution Q (metric: cross‑entropy H(Q,P)).

High I(X;Y) with low H(Y|X) but large H(Q,P) leads to confidently wrong code; low I(X;Y) with high H(Y|X) yields vague or inconsistent output.

Applying the Framework to Common Questions

Why do detailed prompts still produce unexpected results? Prompts are low‑dimensional; they expose only part of the high‑dimensional human intent. Techniques such as few‑shot examples, negative examples, and acceptance criteria increase I(X;Y) and shrink H(Y|X).

Why do new projects work better than legacy codebases? New projects have constraints that align closely with the public data the model saw ( P≈Q), so H(Q,P) is small. Legacy systems contain many hidden, private rules, making I(X;Y) near zero for critical constraints and H(Q,P) large.

Can a conversational agent deliver a whole feature from a requirements document alone? No. The data‑processing inequality guarantees that information not expressed in the document cannot be recovered later. The agent can fill the coverage layer via retrieval and testing, but the fill layer still depends on hidden business rules.

Does more memory always improve accuracy? Only if retrieved memories are highly relevant. Irrelevant or outdated memories dilute mutual information and increase H(Y|X). Effective memory design stores error records, business conventions, and architecture decisions, and discards obsolete artifacts.

Harness Engineering and Role Evolution

Beyond prompt engineering, Harness Engineering adds an outer loop: plan decomposition, state management, tool orchestration, verification gates, feedback, fallback, and human‑in‑the‑loop handoff. This loop repeatedly makes the business constraints explicit, reducing H(Q,P) over time.

The shift also blurs traditional roles. Designers and product managers can generate runnable code directly, while a single person may assume PM, design, and development duties, trading off professional depth for lower information‑transfer cost.

Conclusion

All the discussed techniques—context engineering, RAG, memory, SDD, and harness engineering—fit into the same entropy‑based coordinate system. The practical rule is simple: make the model guess less and expose more real constraints.