Skip to content
Building a Traceable AI-Native R&D Artifact Chain: From Intent to Evidence
R&D Management 41 min read

Building a Traceable AI-Native R&D Artifact Chain: From Intent to Evidence

This article details the six core artifacts of AI-Native SDLC—Intent, Spec, Plan, Task, Eval, and Evidence—explaining their design principles, versioning, lineage, and how they form a verifiable chain that replaces traditional documentation with machine-readable engineering objects to ensure Agent accountability and automated governance.

Qborfy AI
Qborfy AI
Qborfy AI

The article opens with a real incident: an AI-generated PR passed all tests but modified unrelated OAuth logic, revealing that code correctness ≠ task correctness . Traditional test suites only verify code execution, not whether the Agent did the right thing. This motivates a new artifact system for AI-Native SDLC where every intermediate state becomes a version-controlled, machine-readable engineering object.

Traditional Artifacts Are Scattered and Implicit

Requirements live in story trackers, PRDs in wikis, discussions in Git issues, decisions in meeting notes, and context in product managers' heads. Humans stitch these together via context; Agents cannot. Agents need structured, machine-readable inputs that answer four questions:

Why do this? (prevents unnecessary "optimizations")

What defines done? (tells the Agent when to stop)

What must not change? (prevents scope creep)

How to prove it's right? (enables acceptance)

Six Artifacts, One Traceable Chain

The six core artifacts form a linear but versioned chain:

Intent says why, Spec says what, Plan says how, Task says who executes, Eval says how to prove, Evidence says what was proven.

Each artifact carries an ID, version, status, provenance, and upstream/downstream links. They are engineering objects that Agents can read, generate, modify, validate, link, and consume.

Rework Returns to the Level Where the Problem Originated

Real development is not a straight line. When a Task fails, the fix level depends on the root cause:

Implementation defect, missing file, test failure → Roll back to Task. Example: Failure counter not reset after successful login. Revise current Task, re-run corresponding Eval.

Technical approach or dependency assumption invalid → Roll back to Plan. Example: Redis cluster doesn't support required atomic ops. Generate Plan v+1, mark old Plan superseded, re-split affected Tasks.

Requirement rule missing, conflicting, or unverifiable → Roll back to Spec. Example: Whether admin can unlock after lock; threshold conflicts with compliance. Generate Spec v+1, mark original Plan and Tasks stale, re-plan.

Original problem, success goal, or business boundary changed → Roll back to Intent. Example: Product decides OAuth login must also be protected. Generate Intent v+1, re-evaluate and invalidate downstream artifacts along lineage.

Key principle: retain old versions, create new versions, record why revised . For example, SPEC-001@v1 missing acceptance rule → SPEC-001@v2; referencing PLAN-001 and TASK-003 become stale. This preserves auditability.

Lightweight finding events capture discoveries during Task execution without becoming a seventh artifact. Example:

finding:
  id: FIND-001
  discovered_in: "TASK-003"
  type: "plan_assumption_invalid"
  summary: "Redis cluster mode cannot satisfy atomic counter and lock requirements"
  impacts:
    - "PLAN-001@v1"
    - "TASK-003@v1"
  action: "revise_plan"
finding

explains why rollback ; the actual revision stays on Intent, Spec, Plan, or Task. Anthropic calls this the committed artifact chain .

Intent: Why + Outcome + Boundary

Intent is the chain's start and most often written incorrectly as a Solution. Wrong example:

intent:
  solution:
    use_redis_counter: true
    add_auth_middleware: true
    modify_login_endpoint: true

This locks implementation too early. Correct Intent describes problem and expected outcome :

intent:
  problem: "Consecutive login failures create account security risk"
  outcome:
    - "Restrict anomalous login attempts"
    - "Reduce credential stuffing risk"
  constraints:
    - "No new PII fields"
    - "Reuse existing auth system"
  non_goals:
    - "Do not refactor OAuth login flow"
  success_criteria:
    - "Brute-force attempts reduced 90%"

Fields: problem – what's wrong, no solution outcome – desired business/system changes constraints – hard boundaries (security, compliance, cost) non_goals – explicit out-of-scope items (critical brake against scope creep) success_criteria – measurable value indicators

Intent = Why + Outcome + Boundary . Why aligns stakeholders; Outcome defines destination; Boundary draws the line.

Spec: Turning Intent into Verifiable Requirements

Spec answers "what" and crucially "what counts as done". Traditional vague requirements like "add login failure limit" are useless for Agents. AI-Native Spec must include Acceptance Criteria (AC) in Given-When-Then (GWT) format from BDD:

spec:
  requirements:
    - id: REQ-001
      description: "Trigger limit after consecutive login failures"
      acceptance:
        - id: AC-001
          given: "User has 4 consecutive failed logins"
          when: "5th login fails with wrong password"
          then:
            - "Account enters LOCKED state"
            - "Returns unified error code, no account existence leak"
        - id: AC-002
          given: "Account is LOCKED"
          when: "User attempts login"
          then: "Request rejected, lockout message returned"
  boundaries:
    in_scope:
      - "Password login"
    out_of_scope:
      - "OAuth login"
      - "SMS verification login"

GWT rules:

One scenario verifies one business rule or branch.

Given must be reproducible ("4 consecutive failures" not "multiple failures").

Then must be observable/verifiable (state, response, event, log, metric).

No technical implementation in scenarios (no Redis, DB tables, class names).

Multiple scenario types needed per requirement: positive, boundary, exception, negative (e.g., OAuth unaffected). GWT also serves as structured input for Agents to generate tests, run Evals, and archive Evidence.

Plan: From Requirements to Executable Implementation Design

Plan solves "how". Anthropic's Plan Mode standard: an engineer who never saw the original session should understand implementation from Plan alone . Plan is for Agents to decompose Tasks, decide file changes, and order execution.

plan:
  objective:
    spec: "spec_001"
  approach:
    summary: "Implement failure count limit via Redis Distributed Counter"
  changes:
    - id: CHANGE-001
      path: "packages/auth/src/AuthService.ts"
      operation: "modify"
      reason: "Add login failure counting logic"
    - id: CHANGE-002
      path: "packages/auth/src/LockService.ts"
      operation: "create"
      reason: "Encapsulate account lock/unlock logic"
  implementation_order:
    - CHANGE-002  # create LockService first
    - CHANGE-001  # then modify AuthService to call it
  risks:
    - risk: "Redis unavailable fallback strategy"
      mitigation: "Fail-open: no rate limiting when Redis down"

Fields: objective.spec links to Spec; approach.summary gives one-line direction; changes list file-level modifications with reasons; implementation_order captures true dependencies; risks expose and mitigate upfront.

Task: Smallest Independently Executable Unit for Agents

Task must be executable, verifiable, rollbackable . Example:

task:
  id: TASK-001
  title: "Implement account failure count"
  parent:
    plan: "plan_001"
    spec: "spec_001"
  scope:
    include:
      - "packages/auth/src/AuthService.ts"
    exclude:
      - "packages/oauth/**"  # explicit exclusion prevents Agent drift
  inputs:
    context:
      - "code://AuthService"
      - "spec://REQ-001"
  instructions:
    - "Use existing RedisClient, no new Redis library"
    - "Do not modify OAuth logic"
  acceptance:
    - "Failure count increments correctly"
    - "Count resets after successful login"
  verification:
    commands:
      - "pnpm test packages/auth"

Critical fields: scope.include/exclude defines code boundaries; inputs.context makes implicit context explicit via stable URIs; instructions captures task-specific constraints; acceptance and verification.commands enable automated validation. Tasks form a DAG for parallel execution (e.g., T1 and T2 independent, T3 depends on both).

Eval: Not Test, but Verification of Agent Behavior

Eval is the most overlooked yet innovative layer. Difference illustrated:

All unit tests pass ✅

Functional logic correct ✅

But: modified OAuth flow ❌

But: introduced new Redis client ❌

But: deleted 3 existing tests ❌

Traditional Test = PASS. Eval = FAIL. Test verifies code runs; Eval verifies Agent did the task correctly. Anthropic defines Continuous Evals as "AI-Native stage-gate QA". Real incidents should feed back into Eval Suite.

Complete Eval covers multiple dimensions:

eval:
  id: EVAL-001
  objective: "Verify login failure limit matches Spec"
  scenario:
    given:
      - "Account status ACTIVE"
      - "Failure count 4"
    when:
      - "Execute one wrong-password login"
    expected:
      - "Failure count = 5"
      - "Account status = LOCKED"
      - "OAuth login flow unmodified"  # Test never checks this
  checks:
    - id: CHECK-001
      type: "functional"
      command: "pnpm test auth"
    - id: CHECK-002
      type: "security"
      rule: "AUTH-001"
    - id: CHECK-003
      type: "architecture"
      rule: "No new Redis client introduced"
    - id: CHECK-004
      type: "agent_quality"
      checks:
        - "Modified file count <= expected"
        - "No out_of_scope files modified"
        - "No existing tests deleted"
agent_quality

is AI-Native specific: validates Agent's behavioral boundaries.

Evidence: From "Tests Pass" to Traceable Proof

Traditional release basis: "tests pass + engineer reviewed". AI-Native basis: structured Evidence artifact.

evidence:
  id: EVD-001
  result: "passed"
  subject:
    task: "TASK-003"
    spec: "SPEC-001"
  checks:
    - eval: "EVAL-001"
      status: "passed"
      score: 1.0
    - eval: "EVAL-002"
      status: "passed"
      score: 0.98
  tests:
    total: 128
    passed: 128
    failed: 0
  security:
    status: "passed"
    scanner: "semgrep"
  changed_files:
    count: 7
    unexpected_changes: 0  # no scope violations
  agent:
    provider: "anthropic"
    model: "claude-opus-4"
  ci:
    run_id: "987654"
    status: "passed"

Evidence answers: "Why allow this Agent's code into production?" with auditable checklist: Spec satisfied ✅, functional tests ✅, Eval ✅, security scan ✅, no scope violations ✅, CI ✅. This enables high automation by replacing human judgment with traceable proof.

Lineage: Tracking Upstream/Downstream Relationships

Artifacts are linked via explicit relationships ( derived_from, implements, validates, evidences). When upstream changes, downstream impact is computable. Example: Spec v2 → Plan v1 becomes STALE →

Task v1
STALE

→

Eval v1
STALE

. Without lineage, impact is unknown.

Complete Walkthrough: Login Failure Limit

Intent : prevent brute-force, reduce risk 90%, no OAuth change, no PII.

Spec : 5 failures → lock, GWT ACs, password login only.

Plan : Redis Counter + LockService, create LockService.ts then modify AuthService.ts.

Tasks : TASK-001 create LockService (Agent A), TASK-002 Redis Counter (Agent B parallel), TASK-003 modify AuthService (after T1,T2).

Eval : functional, security, architecture, agent_quality all pass.

Evidence : all Evals pass, 128 tests pass, 7 files changed, zero unexpected changes → Policy Gate → production.

Mapping to Major Vendor Approaches

Intent : Anthropic intent.md, OpenAI Engineer Intent, GitHub Spec Kit Intent-driven, AWS Kiro Prompt

Spec : Anthropic spec.md, OpenAI Design Docs, GitHub Spec Kit Spec, AWS Kiro requirements.md Plan : Anthropic plan.md, OpenAI Execution Plan, GitHub Spec Kit Plan, AWS Kiro design.md Task : Anthropic implicit in Plan, OpenAI Issue/Work Item, GitHub Spec Kit Tasks, AWS Kiro tasks.md Eval : Anthropic Continuous Evals, OpenAI Feedback Loop, GitHub Spec Kit Quality Checks, AWS Kiro Validation

Evidence : Anthropic Audit chain, OpenAI CI/Feedback, GitHub Spec Kit Checks/PR, AWS Kiro Tests

Core idea consistent: key R&D states must be artifactized, not left in heads and Slack .

Artifacts Are Engineering Objects, Not Documents

Markdown shell is just presentation; essence is machine-manageable object with identity, state, relationships, rules.

Identity : Document uses filename/link; Engineering Object uses stable ID (e.g., SPEC-001), survives moves/renames

Content : Document human-readable; Engineering Object human text + machine fields, constraints, acceptance

Version : Document overwrite or "final_v3"; Engineering Object explicit attribute ( SPEC-001@v2), knows predecessor

State : Document verbal "approved"; Engineering Object controlled states: draft, in_review, approved, stale, superseded Relationships : Document links, memory; Engineering Object explicit derived_from, implements, validates, evidences Change Impact : Document manual notification; Engineering Object upstream change auto-marks downstream stale via lineage

Automation : Document manual messages, tickets; Engineering Object state transitions trigger next steps (e.g., approved Spec → generate Plan)

Minimal artifact schema: kind, id, version, status, lineage ( derived_from, supersedes), owners ( author, approvers), content. ID and version are not Git commits; they represent artifact lifecycle. Format (Markdown+YAML front matter, YAML, JSON) secondary; object semantics primary .

State Transitions Drive the Process

Process becomes event-driven: SPEC-001@v1 draft → security approves → approved → auto-create PLAN-001@v1 → Task execution finds gap → finding → SPEC-001@v1 stale → team creates SPEC-001@v2 → approved → dependent Plan/Tasks re-planned, old versions retained for audit. Simple low-risk changes flow automatically; only uncertainty, insufficient evidence, or elevated risk pauses for human intervention.

Relationships Are Computable Lineage, Not Links

Four core relations: derived_from (upstream source), implements (design/requirement), validates (rule verification), evidences (proof for conclusion). Enables queries: "Which tests break if this requirement changes?", "Why was this PR allowed to release?", "Which requirement and Agent run caused this production incident?"

Three Common Pitfalls

Treating artifacts as formal meeting minutes – if not consumed by Task/Eval/CI, still just documents.

Allowing Agents to silently overwrite approved upstream artifacts – Agents propose revisions via new versions with review; approved versions remain immutable.

Recording only the happy path – finding, failed Evals, waivers, rollbacks are equally part of the object lifecycle. Without them, "why was this exception allowed?" reverts to chat logs.

This is Anthropic's Event-driven SDLC : next phase triggered by prior phase's state and relationship changes, not manual pushing.

Summary

Six core artifacts: Intent (why) → Spec (what) → Plan (how) → Task (who) → Eval (how to prove) → Evidence (what proven)

Intent principle: Why + Outcome + Boundary; never embed Solution

Spec requires Acceptance Criteria (GWT) with explicit in_scope/out_of_scope

Plan must be complete enough for a stranger to implement

Task: executable, verifiable, rollbackable; DAG for dependencies

Eval ≠ Test: Test checks code correctness; Eval checks Agent behavioral correctness

Evidence = traceable release basis, replacing "engineer reviewed"

Lineage enables automatic impact detection on upstream changes

Next Episode Preview

Artifact chain designed, but Agent Task execution requires deep codebase understanding . Next: L3 Context Engineering – why RAG falls short for software tasks, what Code Graph solves, and how to let Agents accurately locate and modify code in real large-scale repos.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

IntentTaskLineageEvalSpecEvidenceAgent GovernancePlanAI-Native SDLCR&D Artifacts
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.