From Vibe Coding to Controlled Delivery: Dataphin's Plan Mode for AI-Assisted Data Development

Why Vibe Coding Fails in Production Data Development

As coding agents generate thousands of lines in minutes, a new problem emerges: code generation accelerates, but human judgment is still required to verify whether requirements are correctly understood, solutions are sound, and changes are verifiable. In data development, a seemingly correct SQL may contain metric definition biases; an auto-generated task chain may miss upstream dependencies; a model-suggested refactor may introduce subtle data quality risks. Pure "vibe coding" — iterative chat-based modification — cannot support stable delivery for production systems.

Three Pain Points of Vibe Coding

Context bloat causes model drift. Vibe coding mixes requirements, constraints, historical decisions, and temporary fixes in a single growing context. As context lengthens, noise increases, making it harder for the model to distinguish current valid requirements from abandoned attempts, leading to missed constraints and gradual deviation from the original goal.

AI code generation speed far exceeds human review speed. AI marginal cost of code generation is near zero; a single request can produce thousands of lines. However, human review still requires line-by-line understanding of design intent, boundary conditions, error handling, and security risks, creating a "production speed far greater than consumption speed" imbalance.

Weak maintainability. If code is generated instantly in chat without simultaneously persisting design intent, module boundaries, key trade-offs, and constraints, future maintainers cannot understand why a design was chosen, which parts are immutable, which logic is for historical compatibility, and which was a temporary AI implementation.

Industry Research: SDD Tools and Implementation Levels

The article references Martin Fowler's analysis "Understanding Spec-Driven-Development: Kiro, spec-kit, and Tessl" (https://martinfowler.com/articles/exploring-gen-ai/sdd-3-tools.html), which defines three SDD implementation levels:

Spec-first: Write a thoughtful Spec, use it for AI-assisted task completion, but discard the Spec afterward.

Spec-anchored: Retain the Spec after task completion for ongoing feature evolution and maintenance.

Spec-as-source: The Spec becomes the primary source artifact; humans never touch code directly, only edit the Spec over time.

Core Implementations Across Kiro, GitHub Spec Kit, and Claude Code

Despite different terminology (Spec-Driven Development, Specs, Plan Mode, Tasks), all aim to transform vague human intent into reviewable, executable, verifiable, and sustainably evolvable structured context — i.e., Spec documents .

Seven Key Insights from Industry Tools

Customizable Spec templates to avoid over-design. Different teams prefer different Spec structures, acceptance criteria, and test requirements. Templates should map to task complexity and characteristics. Kiro provides separate templates for features and bug fixes; Claude Code offers four complexity-tiered templates.

Verification as a first-class capability. An unverifiable Spec is just documentation; a Spec that generates tests, runs them, and links to requirements forms an engineering loop. Kiro's Property-Based Testing (PBT) mechanism inspires converting natural language requirements into executable verification. Claude Code requires Plans to include verification commands or end-to-end validation methods. Example verification section from Claude Code:

验证方案
单元测试覆盖
1. MemoryContext 构建测试
- 测试记忆召回
...
集成测试
1. 开启白名单用户进行对话
...

User involvement at few critical points. Human bandwidth is limited; users should not review Markdown line-by-line but intervene at key nodes: requirement confirmation, solution decision, execution approval, and verification results — achieving reviewability . Kiro's three-phase delivery suits complex tasks but may be heavy for small ones; Claude Code's lightweight Plan mode is more practical currently.

SDD requires permission control. Before user approval, the agent should default to read-only exploration and plan generation, not directly modify code or cause environment changes beyond the Spec.

Allow transition from Vibe to Spec. Kiro supports initial free conversation, then "Generate spec" to enter a structured session. Real users often explore first; products should support: free chat/vibe exploration → discover stable requirements → one-click Spec generation → structured development.

When to use SDD: complex tasks needing structured planning; unclear requirements requiring exploration first; large impact scope; team collaboration needing sync; important user preferences or business constraints.

When not to use SDD: simple, low-risk, small-scope changes where Spec cost exceeds change cost; rapid exploratory prototypes; user already provides complete implementation instructions.

Dataphin's Four Design Principles for Plan Mode

Principle 1: Right-Sized Plan to Constrain Model Execution

Plan content is not about volume but about controlling agent behavior. Every document block must serve at least one purpose: constrain generation, support review, drive execution, drive verification, or support traceability. Blocks failing these criteria should be removed.

Distinguish long-term context from current request:

Long-term context: tech stack, directory structure, coding standards, quality gates.

Current request: what to do, why, acceptance criteria.

Execution plan: which files to change, in what order.

Mixing them causes redundancy and confuses the agent about what is permanently valid versus task-specific.

Avoid forcing all tasks through a full Plan. Different data warehouse demands and complexities should map to appropriately complex Plan templates.

Principle 2: Reduce User Review Burden

In the AI coding era, user bandwidth is scarce. Users may not want line-by-line code review but still need directional control. Intervention should occur at few critical nodes, reviewing: goal correctness, solution reasonableness, risk identification, and verification sufficiency.

Principle 3: Plan Must Be Model-Verifiable

An unverifiable Plan is just documentation; a Plan that maps to tests, checks, and result confirmation forms an engineering loop. Each requirement should map to verifiable outcomes: executable verification, data verification, etc. Users should see not just implementation process but completion evidence, establishing traceability between requirements, implementation, and tests.

Principle 4: Plan-to-Execution One-Way Convergence

Development flow follows "Plan → Execution" single-direction convergence. Plan phase clarifies goals, boundaries, solutions, dependencies, and verification methods, eliminating uncertainty before execution. Execution phase implements and verifies per the confirmed Plan. Plan is the sole basis for execution; execution must not drift goals or add key decisions ad-hoc.

Dataphin Plan Mode Core Design

Core Capabilities

Requirement Understanding. Extract true goals from user natural language, historical dialogue, current SQL context, and documents. Identify what problem the user wants to solve, expected outcomes, involved tables, and distinguish user-expressed information from agent-inferred information. Goal is reliable understanding via read-only analysis, not rapid template output.

Intent Clarification. When ambiguities or decisions cannot be resolved via data exploration or code reading, ask the user. Guide the model to ask high-quality questions: avoid questions answerable via data exploration; batch related questions; focus on what only the user can decide — requirements, preferences, trade-offs, edge-case priorities. Adjust questioning depth per task; vague needs may need multiple rounds.

Solution Planning. Convert "what to do" into "how to do it." Based on project context, upstream/downstream tasks, architectural constraints, and quality standards, analyze and form a recommended technical solution. For multiple implementation paths, provide trade-off rationale; for high-risk tasks, specify risks, mitigations, and rollback strategies. Core goal: present a low-cost reviewable technical approach before code changes. Data warehouse Plans should include: requirement description, data dependencies, metric analysis, solution design, requirement tests.

Task Decomposition. Break the technical solution into executable, traceable todo items. The model creates a todo list via a "todo" tool, generated from the Plan document, to drive subsequent agent execution accurately and orderly, completing design and verification per the Plan, with task status tracking.

Plan Review. Critical gate from "planning" to "execution." System maintains read-only or restricted permissions before user approval, allowing SQL exploration and Plan generation but forbidding direct business task modification or high-risk operations. Binds agent execution permission to user confirmation, preventing unaligned auto-modifications and ensuring design-phase environment safety. Agent also gets a mode-switch tool to autonomously enter Plan mode based on demand complexity.

Execution Tracking. Rich tracking in the product UI. Todo list status updates as the agent executes, until all items complete. Studio workspace shows real-time tool invocation processes and results (data exploration, asset lookup, task changes), enabling users to track and review the agent's implementation process.

Conclusion and Future Directions

Dataphin aims to structure user intent, requirement boundaries, technical solutions, task decomposition, and verification methods into a reviewable, executable, verifiable, and sustainably evolvable Plan before the agent modifies code. Core problems solved:

Prevent agents from generating code from vague prompts, reducing misinterpretation and regression risks.

Transform "what, why, how, order, verification" from implicit model thinking into explicit artifacts.

Shift user review from "inspecting massive code" to "reviewing intent, solution, tasks, verification standards."

In multi-person collaboration or long-term maintenance, turn one-off chat context into reusable, traceable knowledge assets.

Code rots, tools optimize, models upgrade. But your business understanding, architectural taste, and AI constraints persist in contracts and Spec documents — the immutable foundation.

Future Optimizations

Avoid over-design: Produce right-sized Plans on high-quality understanding, reducing review pressure and improving agent execution efficiency and output quality.

Add model review to the loop: After Plan generation, introduce a model review checkpoint as a quality gate to ensure the design meets user needs and resolves all critical issues.

Verification mechanisms: Provide more general data warehouse task verification for higher-quality engineering loops.

Evolve toward Spec-anchored: While Plans are retained, future iterations don't strictly depend on historical Plans. Explore better reuse of Plans for feature evolution and maintenance, preventing Spec drift.

Introduce Memory: Auto-condense experiences from multi-round Plan generation and execution. Model learns user preferences faster in subsequent tasks, reducing repeated communication costs.

Ultimately, Dataphin Plan mode seeks to give users higher-quality, more controllable agent execution results at lower review cost , while delivering an out-of-the-box data development experience. For the agent era of data development, Dataphin will continue exploring how to make AI more reliably understand requirements, execute tasks, and deliver verifiable results.