AI Rewrites to Rust Won't Fix Slow Software: Design Is the Real Bottleneck

The central thesis is that software slowness stems from poor design choices, not from the inherent speed of the programming language. The author demonstrates this through several lines of reasoning.

Most Programs Are I/O-Bound

In a typical CRUD web application with a database, roughly 50% of request time is spent waiting for database queries (fetching the current user, subscription records, permissions, etc.). Rewriting the application in C might at best double the speed of the remaining 50%, yielding a maximum 2× overall improvement. The same holds for any program that spends most of its time waiting for network responses, disk I/O, or third‑party APIs — making the language runtime faster has negligible impact because the bottleneck is external. Even if curl code were 100× faster, you would not notice because all time is spent waiting for network responses.

Slow Languages Already Delegate Heavy Lifting to Fast Libraries

For tasks that are genuinely CPU‑intensive, popular "slow" languages provide high‑performance libraries written in C, C++, or Rust. Examples include cmark-gfm for Markdown rendering and the entire Python data‑science stack (NumPy, PyTorch, TensorFlow) that offloads matrix multiplication to highly optimized C/assembly code running on GPUs. If a Python program spends 80% of its time in GPU matrix kernels, rewriting the surrounding Python code in C yields almost no benefit.

Rare Cases Where Language Speed Matters

There are programs that perform substantial CPU‑bound work in a slow language without using optimized libraries. These are usually low‑priority codebases that haven't received engineering attention. Actual Computer recently rewrote tiktoken in assembly and achieved a 4× speedup, but the author notes that even such a dramatic improvement only affects a tiny fraction of the total inference pipeline (tokenization vs. model inference).

Design Trumps Language: The Bryan Cantrill Case Study

Bryan Cantrill attempted to rewrite a carefully optimized C program in Rust. The idiomatic Rust rewrite was expected to be ~20% slower, but it turned out 30% faster. The reason: Rust's standard library favors BTreeMap (B‑trees) over binary search trees, and B‑trees have superior cache locality. This illustrates that even in CPU‑bound workloads, algorithmic and data‑structure choices dominate language performance characteristics.

Practical Optimization Strategies (Language‑Agnostic)

Cache slow operations (especially external API calls).

Execute slow operations asynchronously or in parallel (background jobs, parallel requests) instead of sequentially; split large operations into steps (e.g., replace N+1 SQL queries with a single query).

Replace slow SQL queries with faster ones.

Skip unnecessary work (e.g., fetch auxiliary data only for entities on the current page).

All these techniques are independent of the programming language's raw execution speed. In fact, higher‑level languages make such design flaws more visible, enabling easier optimization. If a web server were written in assembly, it would be much harder to notice N+1 queries or realize API calls could be moved to a background job.

Conclusion

Asking AI agents to rewrite entire codebases in Rust, C, or assembly is not true optimization. The range of programs that benefit significantly from such rewrites is very narrow. To genuinely accelerate software with AI, have the agent analyze the program, identify bottlenecks, and propose functional redesigns. Only when a genuine CPU‑bound hotspot cannot be eliminated by redesign should a targeted rewrite in a faster language be considered. Rewriting from scratch rarely delivers the speedups most people imagine.

Reference: https://www.seangoedecke.com/software-is-slow-because-of-bad-design-not-slow-languages