Skip to content
AI Says 'Done' — Why It's Not Ready: Iterating from 0.1 to 1.0 with Human Feedback
R&D Management 12 min read

AI Says 'Done' — Why It's Not Ready: Iterating from 0.1 to 1.0 with Human Feedback

The article describes an iterative AI-assisted development workflow where the first version (0.1) is tested and refined through human feedback and regression testing to reach agreed requirements (1.0), emphasizing distinguishing missed requirements from new ones, preserving working features, and measuring effort until the product is truly usable, not just when the agent declares completion.

Data Bricklaying Diary
Data Bricklaying Diary
Data Bricklaying Diary

Goals were discussed, design reviewed. The Agent worked for a while, consuming time and tokens, then announced: "Completed." But opening the system reveals it's still not satisfactory.

This is a common situation when developing with AI. Models are increasingly powerful — give them a goal, communicate the thinking, and they can proceed independently. Yet the output often falls short of the desired result.

My current approach: get a first version out. It might be only 0.1 or maybe 0.5 of the target, then test, give feedback, and iterate step by step until reaching a 1.0 that meets the agreed requirements. These numbers are illustrative; 1.0 means meeting current agreed usage requirements; whether to release depends on production conditions.

I don't expect perfection in one shot, but sometimes wonder: after so much upfront discussion, why so many changes later?

Clear Design Doesn't Guarantee a Satisfying First Version

I still advocate designing first. Business flow, module boundaries, data sources — if these aren't clarified for complex features, rework later is even more likely.

But reading a design and actually using the feature are two different things.

Take a hypothetical batch import feature. The design specified: upload file, validate data, show results, query failure reasons. The Agent implemented all of these.

In actual use, problems surface: failure reasons buried several clicks deep; task still processing but UI appears frozen; re-uploading a corrected file raises uncertainty about duplicate imports of previously successful records.

The requirement "retries must not duplicate writes" was stated but not implemented — that's a bug to fix. The design didn't clarify how to continue after failure — that's a design gap. Some awkward interactions only become apparent after hands-on use.

Don't blame all rework on the model, nor assume every issue means requirements weren't clear. Need to revisit what was actually agreed.

First compare against original agreements, then decide whether to fix implementation, supplement design, or adjust interaction
First compare against original agreements, then decide whether to fix implementation, supplement design, or adjust interaction

The first version's value lies precisely here: finally something operable, making previously invisible problems concrete.

First Version Must Be Testable and Worth Iterating

Traditional software also requires testing and fixing; discovering issues after a first version isn't new. Now, with Agents handling much coding, humans get a testable artifact earlier. Overall project speed depends on how well the subsequent iteration goes.

Continuing the import example: page opens, file uploads — entry point exists. Next, test: invalid format, mid-process failure, resubmission behavior, permission-based result visibility.

UI shows "success" — must verify data actually wrote correctly. Checking code and backend records remains necessary.

This is why I start with 0.1: a verifiable version tells exactly what to fix next.

But the core direction must be sound. If business boundaries are reversed or key dependencies invalid, no amount of UI polish or added prompts will salvage it.

Fixing for Hours, Only to Revisit Old Problems

New issues can be addressed. More frustrating: a just-fixed area breaks again in the next round.

Example: failure prompts adjusted, then lost during layout change; retry logic validated, then a data processing change reintroduces duplicate writes. If every round requires full regression from scratch, humans tire and progress becomes unclear.

At this point, "optimize a bit more" isn't enough. Using the import example, assuming business confirmed partial success allowed and retries must not duplicate, feedback becomes specific:

First import: two succeeded, one failed. After correction and resubmit, the two previously successful records were written again. Investigate cause, fix, and add this scenario to regression tests.

This provides concrete steps, actual results, and judgment criteria. If business rules aren't settled, discuss rules first — don't let the Agent pick an easy implementation.

Once fixed, automate the checks and run them on subsequent changes. That's regression testing: verify new changes don't break previously correct behavior. Agents can write and run tests, but humans must ensure tests judge by already-confirmed rules.

For instance, if duplicate writes are forbidden but the Agent changes the test to "allow duplicates", tests pass but the bug remains. UI operations and actual data still need checking — don't trust a row of green results alone.

When business confirms partial success allowed, full-batch retry must avoid duplicate writes and this behavior becomes a regression check
When business confirms partial success allowed, full-batch retry must avoid duplicate writes and this behavior becomes a regression check

Each iteration must at least preserve what was already working. Otherwise, 0.1 and 0.5 are just feelings; the system may be spinning in place.

Human Involvement Doesn't Mean Micromanaging Every Action

Human involvement isn't about approving every code change.

Once problems and constraints are clear, a complete chunk of work can be delegated: let the Agent diagnose, propose fixes, implement, and return with verification results.

But when changes involve altering business rules, adjusting module boundaries, or touching existing data — pause and discuss. Supplement documentation, confirm with business stakeholders. Record important changes back into requirements or design to avoid reverting to old agreements.

A standalone tool can be built first, then tested thoroughly. Features involving permissions, money, or data migration need early validation of critical paths in authorized isolated environments — not on live business.

Delegating a work segment to an Agent and completely disengaging midway are not the same.

Human participates through testing, feedback, and trade-offs; re-discuss approach when direction is off; version numbers are illustrative
Human participates through testing, feedback, and trade-offs; re-discuss approach when direction is off; version numbers are illustrative

Real Effort Counts Until It's Actually Usable

Initial disappointment came from spending significant time and tokens, yet still needing manual adjustment. Another view: an early testable version that reveals unsuitable approaches or saves massive manual coding makes that investment worthwhile. True value requires counting time from initial communication to genuinely usable.

But if the Agent repeatedly misses the same requirement, each change breaks working features, or it spins on the same error for long — stop and diagnose. Adding more time and tokens won't necessarily help.

Is the task too large, needing scope reduction? Did it fail to find a valid design, or lack verification conditions? Maybe the current model simply can't handle this class of work. Find causes from the actual process; don't assume another run will fix it.

Is the human filling in previously unanswered questions, or repeatedly correcting already-clear requirements? This distinction is more useful than counting messages sent.

The former participation gradually materializes ideas; the latter is repeated firefighting that should be reduced. Communication, checking, rework, even abandoned attempts — all belong in the cost accounting.

Post-first-version testing, modification, and re-verification are part of the full investment from communication to usability
Post-first-version testing, modification, and re-verification are part of the full investment from communication to usability

I still start by having the Agent produce a version, then iterate by testing and fixing. This workflow isn't yet smooth, and doesn't always satisfy. But I know the next judgment isn't "can it write code" but "where exactly does this version fall short, and is it worth continuing to iterate?"

Models will keep improving, and my participation style will evolve. For now, I cannot just hear "completed" and hand it off to users.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

AI-assisted developmentAgentregression testingfeedback loopcost estimationiterative developmenthuman-in-the-loopsoftware development workflow
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.