What Capabilities Will Software Need When AI Generates Apps On-Demand?

In a future where all applications (including agents) are automatically generated on demand by AI, the fundamental challenge is uncertainty. AI-generated software cannot guarantee correctness, creating a psychological barrier for users who must trust systems that may contain errors. The author notes that the real world is also uncertain, so software uncertainty is normal; however, controlling it requires several inevitable technical capabilities.

1. Automated Testing from Functional Specifications

Because of the undecidability principle of programs (Rice's theorem / halting problem), complete verification is impossible. The solution is to generate test specifications directly from a functional SPEC and only verify that required functions work correctly. This shifts the goal from proving total correctness to validating that the needed functionality operates as intended.

2. Runtime Safety Sandbox

Testing ensures functional correctness, but running software requires confinement. A sandbox acts as a cage that defines what a program cannot do — enforcing security permissions, resource isolation, and preventing dangerous or unauthorized operations. This runtime guardrail complements static testing by limiting the blast radius of any misbehavior.

3. AI-Driven UI Auto-Generation

Instead of pre-designed user experiences, a user request should naturally produce the appropriate interactive interface (e.g., displaying a price) on the fly. Future apps would not need a traditional UX layer; they would consist only of atomic capabilities and services, with the UI generated dynamically per interaction.

The author identifies four core pillars: automated testing, sandbox software, AI-friendly technology, and observability. The envisioned lifecycle is: on-demand generation, self-compilation, self-verification, self-sandboxing, self-observation, fulfillment of user needs, and finally self-destruction. The article closes by asking whether this self-contained, ephemeral software world represents an ideal form.