Same LRU Cache Code, GPT‑5.6 Sol, Terra, and Luna Produce wildly different results

Same code run on three model versions

Task: implement an LRU cache class LRUCache with O(1) get and put, full type annotations, docstring, and five unit tests.

Implement an LRU (least recently used) cache class LRUCache with: 1) constructor accepting capacity , 2) get(key) in O(1), 3) put(key,value) in O(1) with eviction of the least‑used entry, 4) full type annotations and docstrings, 5) five unit tests.

Token usage measured with tiktoken.

Sol generated 592 tokens in 62 lines. Added thread safety with RLock, parameter validation (raises ValueError for non‑positive capacity), __len__ and __contains__ magic methods, and a pytest fixture containing five tests covering basic read/write, eviction logic, updating existing keys, zero‑capacity error, and access‑order changes. Satisfied all five required functionality points.

Terra generated 315 tokens in 36 lines. Implemented a standard OrderedDict + move_to_end solution with type hints and docstring, and three tests. Lacked thread safety and parameter validation, covering three of the five functionality points.

Luna generated 115 tokens in 16 lines. Minimal implementation without docstring, using short variable names cap and cache, and no tests. Covered none of the required functionality points.

All three versions compiled without syntax errors and executed correctly.

Sol is most expensive but saves overall effort

OpenAI pricing (per million tokens): Sol $30, Terra $15, Luna $4. Single‑call cost: Sol ≈ $0.0178, Terra ≈ $0.0094, Luna ≈ $0.0005. Although Luna’s per‑call cost is ~30 × lower, missing tests, documentation, validation, and thread safety typically require additional conversation rounds, each adding hundreds of tokens and about 30 minutes of developer time. Sol’s one‑round delivery often results in lower total cost for production‑grade code.

Sol produced about 5.1 × the tokens of Luna; the extra lines correspond to tests, locks, checks, and docstrings required for a deployable solution.

When to use which model

Use Sol for production code that needs a complete, review‑ready implementation.

Use Luna for quick, one‑off scripts where missing pieces will be added manually.

Use Terra when the task is not core logic but still benefits from some testing and documentation.

For an LRU cache intended for production, Sol delivers a ready‑to‑merge solution in a single round. Luna’s 16‑line output would require roughly forty minutes of additional work to add tests, thread safety, and error handling. Terra fits internal‑tooling scenarios where basic functionality suffices without full robustness.

Luna’s shorter output does not guarantee faster overall development because extra conversation rounds can increase total token consumption beyond Sol’s single‑round cost.

Don’t judge solely by the price table

Choosing a model based only on per‑token price can be misleading. Sol is pricey per token but often cheaper in total effort; Luna is cheap per token but can become expensive if extensive post‑processing is needed; Terra offers a middle ground.

The primary risk is using Luna for production code and then spending days adding tests, validation, and thread safety, which can end up costing an order of magnitude more than using Sol from the start.