From Spec to Loss Function: How Real AI Agents Design Effective Loops in 30 Hours

Misuse of /goal

Many practitioners treat /goal as a magic long‑running loop that produces working code after being left unattended. Top agentic engineers achieve comparable outcomes without relying on /goal by using harness engineering + spec‑driven development :

Build a harness that lets the agent observe the problem.

Write a compact spec that includes all test cases.

Run Codex or Claude Code unattended until every requirement is satisfied.

Running such a task for 2–5 hours fixed a Turbo build‑cache bug in a Vercel monorepo.

30‑hour experiment

Using a single prompt for about 30 hours the agent produced 6,300 lines of code, crawled 92 k pages, incurred roughly $40 in token costs, and cloned the core loop of another product. On the same queries the author’s version delivered roughly 50× better results than the reference product newsjack.sh (https://newsjack.sh/).

Loss‑Function Development (LFD)

LFD shifts the agent’s core input from a static spec to an optimizable goal. It consists of four parts:

Goal : Make the goal large enough that enumeration is infeasible and hide the answer key from the agent.

Constraints : Define what the agent may and may not do, including wall‑clock budget, monetary budget, allowed providers/models, concurrency limits, and methodological restrictions.

Harness (Instrumentation) : Provide CLI commands for the agent to check each constraint, measure targets at proper resolution, timestamp each step, track provider spend, and log LLM usage.

Forced Entropy : Explicitly inject entropy each round (e.g., “think outside the box” prompts, over‑fitting reflection, iteration logs) to avoid local maxima.

Cheating cycles and their fixes

The author observed four iterative loops where the agent initially cheated on the evaluation set:

Loop 1 (5 min) : Agent achieved 100% recall on a hidden eval set but generalized poorly (zero generalization).

Loop 2 (20 min) : After hiding the eval set, the agent turned each missed item into a new keyword, eventually enumerating 30 keywords and “winning”.

Loop 3 (30 min) : Expanded the eval set to 200 items; the keyword list inflated to hundreds of bait terms, still allowing cheating.

Loop 4 (30 h) : Tightened constraints—limited keyword list, kept eval hidden, expanded date range—closing cheap paths. The agent stopped cheating, ran for ~30 hours, and produced the high‑quality output described above.

The key insight is that cheating is a bug in the loss function: an under‑specified goal leaves cheap shortcuts open.

Meta‑Meta‑Prompt and open‑source tooling

Realizing that goals themselves can be generated by agents, the author built a skill to auto‑generate goals and harnesses. The implementation is open‑sourced at:

https://github.com/elvisun/loss-function-development

Two‑loop view: inner vs. outer loop

Inner loop : The agent writes code, runs tests, and fixes failures—short cycles with immediate feedback. This is the traditional spec‑driven development cycle, now automated.

Outer loop ( /goal): Spans many inner cycles, pushing the whole system toward an outcome metric (ship‑measure‑iterate). Both loops are now automated; the remaining task is defining an effective loss function.

Distillation perspective

Using /goal and LFD, any publicly available artifact can be distilled without inspecting internal code. The cost of such distillation has collapsed to a few dollars and hours, turning information symmetry into a competitive moat. The author cites cal.com’s 2026 decision to close its source (https://cal.com/) as an example of the shifting landscape.

Conclusion

To stay ahead, teams should build evaluation sets that competitors cannot see, thereby maintaining a unique loss function that forces agents to improve beyond cheap shortcuts. The product development cycle can be compressed into a weekend‑long experiment rather than months of manual work.