Complexity Must Exist Somewhere: Reflections on Software Architecture and Design

The struggle with complexity is a perpetual theme in software development, repeatedly surfacing in debates about the amount of commenting in functions, ideal abstractions, excessive framework "magic", and the proliferation of languages within organizations.

Attempting to escape or eradicate complexity is misguided because complexity must exist somewhere; it is indestructible.

Resilience Engineering introduces the control‑theory concept of "necessary diversity": only complexity can handle complexity.

When dealing with build tools, several trade‑offs become evident: simplifying the tool limits its ability to handle edge cases; handling edge cases requires deviating from any established conventions; shared default rules must be negotiated between tool and user; allowing user‑defined scripts gives users a way to specify shared rules; enforcing simplicity forces users to stay within a narrow parameter space; mismatched user cases lead to the creation of wrappers or glue code around the tool.

This complexity is inevitable and must be acknowledged; it persists whether or not developers are aware of it.

Don Norman’s distinction between "knowledge in the head" and "knowledge in the world" (also discussed in Roesler & Woods’ *Designing for Expertise*) illustrates how mental models and external artifacts interact, shaping how we understand and use software.

Focusing solely on simplicity is dangerous because complexity cannot be removed—it can only be shifted. If you move complexity out of your code, you must ask where it goes.

The Rebar3 example shows that a tool can appear simple only if users have a basic understanding of the expected Erlang/OTP project structure; the rules are externalized to the broader ecosystem, making the tool dependent on users learning those rules.

In microservice architectures, attempts to make each service simple still leave complexity elsewhere; if it isn’t in the services, it resides in the rules and knowledge required to use them.

Accepting that complexity must exist somewhere—whether in code, documentation, training, or elsewhere—and giving it a proper place allows architects to manage it effectively and even turn it into a strength for system design.