Full AI vs AI-Assisted Development: Speed, Risk, and Acceptance Compared
The article compares full AI development and AI-assisted development using a bricklaying metaphor and a notification retry case study, analyzing how human intervention points affect speed, risk, and acceptance across investigation, design, implementation, review, testing, deployment, operations, and handover.
01 Requirement Investigation: Clarify What Failure Means
The case study: an order system already sends shipment notifications; the business wants timeout notifications to auto-retry and allow on-call staff to handle exceptions . Orders may ship in batches; each shipment event notifies one recipient per channel. The system has a send queue; whether it supports deduplication by identifier and send-result queries must be verified with the notification service.
Full AI development : Authorize AI to investigate send entry points, retry config, backfill entry, and test records. Require it to thread one existing test order and return file locations, responses, and confirmed facts. When interface specs and sandbox behavior diverge, keep both branches and hand differences to humans. Investigation, organization, and acceptance scenarios can proceed continuously, but external integration confirmation still consumes delivery time.
AI-assisted development : Humans first ask whether the business cares more about missed sends or duplicate sends, then let AI find evidence to verify current implementation. For example, if operations says "one order ships once," the developer immediately asks how split shipments count. Experience shortens investigation but may mistake past rules for current ones; even familiar leads must match split-shipment samples against current interface behavior.
Pre-work deliverable : A short acceptance checklist defining what counts as the same notification, when retry is allowed, what the UI shows when results are unknown, who can handle pending-verification records, and which conclusions come from code, integration, or remain unconfirmed. The critical start condition is confirming the downstream capability that auto-retry depends on . If unproven, build query and pending-verification flows first, leaving auto-retry disabled.
02 Design: Separate One Notification from Multiple Attempts
Assuming integration confirms downstream supports deduplication by client identifier within an agreed window. The system assigns a stable identifier to each shipment notification; first request and retries use it, content stays consistent. The identifier maps to the shipment event and channel; split shipments get separate identifiers. Each call also gets its own attempt record for traceability, explaining why the system sent twice but the receiver processed once.
Design must clarify boundaries: downstream "accepted" only means it took the task; confirming customer receipt depends on receipt and query capabilities. Retry window is bounded by downstream deduplication period and local business SLA, with a max attempt count. Tasks outside the window are first verified or moved to pending. Only when a status query can reliably prove the original request was not accepted and will not take effect late is it safe to resend. Query latency, expired records, or ambiguous semantics all warrant keeping as pending. (Reference: AWS documentation on request identifiers and late requests .)
Full AI efficiency and pitfalls : AI can propose state transitions, data structures, and interfaces in one go. Reusing existing notification components and conventions speeds landing. But if AI assumes "retry queue enables deduplication," local unique constraints and external side effects get conflated. The app may create one notification record yet still let the supplier process twice. Review criteria for AI must include this sequence and evidence of the supplier's deduplication window.
AI-assisted efficiency and pitfalls : Humans can decide business identity, state, and resend rules upfront, letting AI fill the rest. These decisions cascade across layers; once set, no layer needs re-discussion. Risk: humans only define the happy path, leaving backfill and historical tasks to find separate shortcuts. Run the same notification through three scenarios: worker timeout, on-call double-click, old task past deduplication window. All three must pass the same server-side rules.
Design deliverable : A state transition diagram or table showing entry evidence, allowed actions, and forbidden actions per state. The key state is "result pending verification" — it preserves "we don't know yet" so frontend display, auto-retry, and manual resend don't give conflicting answers.
03 Full-Stack Implementation: Thread One Verifiable Path First
The requirement spans shipment events, notification records, workers, supplier interfaces, and admin UI. The first slice to complete can be a thin path: generate a notification from a test order, record one call, then query the same identifier from admin. Then add timeout handling and manual entry, finally backfill legacy data compatibility. This way the first code version is both demonstrable and shows where state originates.
Full AI scope control : Give AI the current branch, available test commands, and modifiable modules; let it thread the whole path. Deliver changes split by concern for easy review: notification state and send rules as one group, admin display and action entry as another, independent acceptance cases saved with relevant changes. This keeps AI's continuous execution speed while helping reviewers locate specific state changes. If AI finds it must rewrite shared queues, change permissions, or migrate large historical data, it should list dependencies and impact first for human scope decision.
AI-assisted avoiding bottlenecks : Humans define interface field meanings, state sources, modifiable scope, and exception expectations, then let AI independently implement a verifiable segment. For example, have AI complete both query interface and admin display with server-side permission checks. If humans micromanage field-by-field, they become a relay between frontend and backend and may miss actual values in DB and queue. Reviewing at a checkpoint with a complete business result is more effective than per-file approval.
Two breakpoints to check early : (1) Between shipment transaction commit and notification task enqueue. If a reliable shipment event exists, reuse it. If current practice updates order then separately emits a message, a mid-exit may lose the send; put the pending-send record in the same transaction as the business change, then let existing forwarding deliver it. (2) Between supplier acceptance and local receipt recording; the process may exit between these steps. On recovery, reuse the original identifier and re-check the deduplication window. Before migration, ask: do historical queue tasks have this identifier? Can old processes read new state? Legacy records lacking evidence go into a separate verification flow to avoid a blanket re-send after upgrade. (Reference: AWS Transactional Outbox pattern .)
Completion evidence : One test order yields shipment event, notification identifier, attempt records, and admin state. Code diffs show entry points at each layer, plus feature flags and authorization checks in place.
04 Code Review: Follow One Order End-to-End
Looking only at the send function easily misses another message-sending entry. For instance, auto-retry reuses the old identifier while manual backfill creates a new notification. Both pass their own unit tests. The real business conflict appears only when the same order runs both paths simultaneously.
Full AI review focus : AI lists changes, call chains, assumptions, and commands run. Reviewers first check for new external calls, config default changes, and whether old paths still use original entry points. Then pick a high-risk breakpoint and trace it — e.g., worker restarts before receiving success receipt. AI explaining its own changes saves reading time, but reviewers must still verify key judgments in the diff. Large diffs that are hard to verify should be split and re-reviewed.
AI-assisted review focus : Humans participated in design, so review is faster but may only cover areas they directed. Have another developer trace from page or event entry. If solo, set aside development chats and do a second pass with acceptance criteria and code diff. Asking AI to find side paths from "where else can this same notification be sent" and checking actual entry points is more effective than generic quality evaluation.
Minimum checks : Are auto tasks and manual ops constrained by the same state condition? Which server-side mechanism prevents duplicate creation or crossing allowed states under concurrency? After hiding the admin button, does a direct API call still check permissions? Do failure logs correlate order, notification, and attempt while redacting full recipient addresses and credentials? When review finds issues, add a reproducible test case and keep the conclusion in the change record.
05 Test & Acceptance: Separate Fault Injection from Assertions
If the mock service returns an error before recording the request, the test only covers "downstream not accepted." Must also verify: receiver records first request, caller gets no response, resubmission still processes once. The oracle is the receiver's record or test inbox; app UI showing success only proves the page received some status.
Full AI : Let AI generate data, simulate faults, write regressions, and run tests. The easiest miss is implementation and tests sharing the same assumption. Acceptors first write expectations from requirements and downstream contracts, then cross-check with independent test entry points. Use a controllable receiver to verify sequencing; use supplier sandbox to confirm real deduplication contract. If sandbox only proves acceptance, record "accepted" in release notes; delivery confirmation uses a separate test channel.
AI-assisted : Humans pick key scenarios and judge outcomes; AI handles prep and repeated execution. Developers familiar with the business tend to pick the most failure-prone paths, but under pressure may only run what they just changed. Bind build and critical regressions to the same candidate version for auto execution. Reserve human time for risks unique to each change: legacy records, cross-layer concurrency, operation permissions, or real downstream behavior.
Fixed acceptance scenarios for this case (whoever implements, same results verified):
First normal acceptance → one notification, admin state matches downstream record.
Accepted but response lost → reuse original identifier, receiver processes once.
Auto-retry and manual op collide → server adjudicates by same rule, receiver not multi-sent.
Deduplication window expired, original result unknown → state pending verification, auto-retry stops.
Unauthorized caller hits operation API → server rejects, no send task created.
Pre-upgrade pending tasks → old state readable, evidence-lacking tasks stay pending verification.
Division without dedicated QA : AI runs most repeatable checks; developers set expectations for key acceptance cases, inspect run evidence, and make release decisions. With two people, swap verification on critical paths. Solo: keep acceptance as a separate step, re-verify with candidate version in test env, independent orders, and receiver records. If a key scenario can't be reproduced or results only show sender-side logs, keep that feature disabled until verification entry is ready.
06 Deploy & Rollback: Deploy First, Then Let Few Real Tasks Through
After tests pass, still verify the deployed binary, production config, existing queue tasks, and new/old instance coexistence behavior. For shipment notifications, rolling back the binary restores code, but already-sent messages, queued tasks, and newly written states each need separate handling. The deploy plan must answer: "If duplicate sends are detected, what is the first step to stop?"
Full AI : AI can organize build, config diff, migration pre-checks, and deploy steps per existing process, plus prepare verification commands. Humans verify candidate build number, production config diffs, switch defaults, and decide when to expand scope. Critically, test the switch's effect on workers: the page button may be off while background auto-retry still runs.
AI-assisted : Humans decide deploy order, initial scope, and stop conditions; AI handles check scripts, parameter diffs, and observation summaries. Manual stepwise ops risk picking wrong env/version or tweaking a param without recording. Writing deploy artifacts, config versions, and key commands into the deploy log lets another person know exactly what's running in prod.
How this case ramps : Deploy with new retry logic off. Keep new/old read-write compatible, then verify from enqueue to downstream record with controlled orders. Switches and scope must take effect before actual sends. Define observation window, baseline, and pause conditions upfront: stop if duplicate sends detected, or if pending-verification count or queue backlog deviates significantly from baseline. After pause, inventory accepted, in-flight, and pending tasks; confirm old version can read new data before deciding rollback. Resume by releasing in batches per business identity to avoid dumping backlog onto downstream. (Reference: Google SRE canary deployment practices .)
07 Production Operations: Locate First, Then Decide Resend
On-call receives "customer didn't get notification" with only an order number. Steps: query shipment events by order, then notification identifier, attempt records, and supplier receipts. If no notification record exists, check if the event arrived. If record exists, distinguish "downstream accepted," "confirmed not accepted," and "result pending verification." Accepted ones check subsequent delivery status. Only those meeting resend conditions are processed per current rules. Unknown results stay pending to avoid one alert triggering a mistaken resend.
Full AI : AI can generate runbooks, log queries, and metric explanations from implementation. Risk: it writes runbooks based on test env fields; post-deploy production logs may lack the same records, or alerts only show total failures, leaving on-call unable to distinguish request failure from pending verification. Before deploy, walk a test order through the runbook, then verify production-visible fields, query entry points, and permissions. In prod, AI can assist organizing authorized read-only evidence; on-call decides pause/resume based on evidence.
AI-assisted : Developers know where they placed logs, so initial triage may be faster. Main risk: diagnosis depends on remembering which spots they changed. Hand the diagnostic path to a rotating peer for a trial run. If solo on-call, at least keep a fixed entry that can query by the same identifier: call time, result, version, and operation log. For recurring alerts, existing queries and runbooks reduce per-incident code reading.
Useful signals : Failure rate alone mixes "accepted but slow response," "explicitly rejected," and "result unknown." Track separately: pending-verification count, oldest task wait time, attempts per notification, and receiver-confirmed duplicates. First two help on-call judge backlog; last two help judge if retry is amplifying the fault. At incident close, turn truly missed scenarios into regression tests so future changes to this module re-run them.
08 Handover & Secondary Changes: Validate with One Drill
At handover, give the incoming colleague a test order and repo access. Have them use the runbook to find which shipment the notification belongs to, how many calls were made, why it's still pending, and what the next steps are. Then change the retry limit once, run regressions in test, and revert. Completing both proves the runbook, code, and ops entry actually align.
Full AI : Have AI write key decisions, code entry points, test commands, and runbooks back to the repo, tagged to the version. Common issue: summary looks good but rationale is scattered in chat. The receiver must try it themselves — confirm runbook commands actually run, log fields are queryable, switch locations and permissions match.
AI-assisted : Humans participated in business decisions and usually know why code is shaped that way. When handing off, commit those decisions to the repo, e.g., "after deduplication window expires, move to pending verification because we cannot guarantee late-arriving requests will still be intercepted," plus the test entry that verifies it. If the team is solo, start from a clean session and fresh workspace, follow the docs end-to-end. Any location only findable by memory gets a file, command, or judgment reference added.
09 Measuring Speed: Separate First Version, Delivery, and Person-Hours
Compare using the same model tools, code baseline, environment, acceptance criteria, and observation window. Pick tasks of similar scope and risk; avoid the same developer doing a second run with memorized answers. Time from requirement entering development to deploy and handover satisfaction. Also record: time to first working version, actual human effort, and wait time blocked on integration, env, or review. Abandoned trials, handovers, or mode switches go into the record. Post-deploy, separately compare issues, rollbacks, and fix effort over the same observation window.
The diagram numbers are illustrative. When rules are clear, full AI might produce first version in 35 min, reach delivery in 65 min, with 25 min human effort. AI-assisted might take 50, 80, and 35 min respectively, assuming the delta comes from mid-process confirmations. With downstream rules unverified, full AI might produce a version in 50 min but need 180 min to deliver (85 min human), while AI-assisted does integration upfront: first version 75 min, delivery 115 min, human 65 min. Both using the same acceptance reveals where speed comes from and where extra time goes.
A student-heavy controlled experiment compared coding assistants vs autonomous agents. On correctly completed short tasks, the autonomous agent group required less human effort; total elapsed time was similar. That study compared specific tools and tasks, illustrating why human effort and wall-clock time must be separated. (Reference: Chen et al. controlled experiment .)
In practice, the same requirement can use different modes at different stages. Current-state investigation and regression script prep can run fully AI-driven. When hitting downstream contracts, legacy data, and production ramp, humans first resolve the decisions that determine delivery success. Finally, when handing the wall to the next shift, walk the same acceptance standard once more. Which brick needs rework, and at which stage, is answered by the delivery record.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
