My Design Harness Practice: Moving AI‑Generated Design from “Can Generate” to “Can Deliver”

Current Results

Reference‑image animation

Requires GPT‑5.5+ or Claude Opus 4.7+.

Free‑form text‑to‑animation

UI‑focused animation

This type of animation is constrained by a fixed style to avoid model drift.

AI‑Generated Animation Is Only the Entry Point

If you look only at the visual result, the first reaction is “AI can make animation again.” The author stresses that the real value lies beyond the static output: the generated result must retain layers, roles, color semantics, keyframes, asset sources, project state, and a rollback path so users can edit text, change style, adjust timing, swap assets, or continue the Agent’s work.

The concept the author calls “Design Harness” shifts the focus from “model can produce a pretty picture” to “model can deliver a design that fits into a real production workflow.”

What Industry Calls a Harness Means for Design

DeepSeek’s job description defines “Model + Harness = Agent.” Anthropic describes an agent harness as a system that orchestrates input, tool calls, and results. OpenAI’s Agents SDK defines an agent as a stateful planner that calls tools and collaborates across specialists. Google Cloud breaks an agent into reasoning, planning, memory, tools, and autonomy.

Applied to design, the core question becomes: How does a model turn a vague requirement into an editable, publishable design file?

Six‑Layer Design Harness Architecture

User Intent – a sentence, reference image, current canvas, and historical context.

Design Brief – a contract that both model and user can inspect.

Aesthetic Stance – tells the model what counts as “good” beyond vague adjectives.

Tool / Asset Registry – exposes real editor capabilities and existing assets.

Editable Protocol – converts the generation into a structured, editable scene.

Verification Loop – uses rules, screenshots, regression, and human polish to ensure stable improvement.

Layer 1: Show an Effect, But Don’t Be Fooled by It

Design‑focused AI tools are easily misled by a flashy first frame. A demo may attract users, but real workflow demands that the design remains editable after many iterations. The author proposes evaluating a tool by how much structure remains after the tenth edit (e.g., headline, CTA, palette, keyframes, assets, exportability).

Layer 2: Brief Is a Contract Between Model and Human

Raw prompts like “make a hello animation” are too short for both designers and models. The author introduces a typed DesignBrief object that captures goal, audience, industry, stance, palette, typography, motion, layout, editable fields, and constraints. This contract prevents the model from repeatedly guessing style, allows pre‑generation planning, and provides a basis for verification.

type DesignBrief = {
  goal: string;
  audience?: string;
  industry?: string;
  stance: "minimalist-editorial" | "glassmorphic-tech" | "kinetic-poster" | string;
  palette: {
    name: string;
    background: string;
    ink: string;
    muted: string;
    accent: string;
    surface: string;
  };
  typography: {
    display: string;
    body: string;
    hierarchy: "editorial" | "dashboard" | "poster" | "product-ui";
  };
  motion: {
    personality: "calm" | "springy" | "kinetic" | "cinematic";
    intensity: 1 | 2 | 3 | 4 | 5;
    maxDurationMs: number;
  };
  layout: {
    density: 1 | 2 | 3 | 4 | 5;
    variance: 1 | 2 | 3 | 4 | 5;
  };
  editableFields: string[];
  constraints: string[];
};

The brief solves three problems: it eliminates per‑round style guessing, persists project‑level intent, and enables a “Plan Preview” when the model detects ambiguous user intent.

Layer 3: Aesthetic Stance Makes “Beauty” Operable

Models tend to collapse to safe templates (purple‑blue gradients, rounded cards, generic copy). The author replaces vague adjectives with concrete stances that encode industry, typography, color semantics, density, layout tension, motion character, and forbidden patterns.

type AestheticStance = {
  id: string;
  useWhen: string[];
  avoidWhen: string[];
  palette: {
    background: string;
    ink: string;
    muted: string;
    accent: string;
    surface: string;
  };
  typography: {
    display: string;
    body: string;
    rule: string;
  };
  motion: {
    rhythm: string;
    easing: string[];
    antiPattern: string[];
  };
  composition: {
    density: number;
    variance: number;
    preferredLayouts: string[];
  };
};

Different stances (e.g., Minimalist Editorial vs. Glassmorphic Tech) produce dramatically different results, turning subjective taste into a programmable system.

Layer 4: Full Tool Exposure Lets the Model Actually Use the Editor

Many agents fail because the tool registry is incomplete. The author stresses a single source of truth for tools, each described with metadata such as name, category, supported modes, mutating flag, cost level, usage conditions, and return type.

type AgentToolMeta = {
  name: string;
  category: "canvas" | "asset" | "analysis" | "generation";
  modes: Array<"agent" | "animation" | "image">;
  mutating: boolean;
  needsApproval?: boolean;
  costLevel: "free" | "credit" | "expensive";
  useWhen: string;
  doNotUseWhen?: string;
  returns: "canvas-diff" | "asset-list" | "plan" | "diagnosis";
};

Assets are treated as tools as well, with a typed AssetSearchQuery that lets the model ask for existing icons, shaders, 3D models, etc., rather than always generating from scratch.

type AssetSearchQuery = {
  type: "icon" | "textfx" | "shader" | "user-image" | "model" | "material" | "all";
  query: string;
  projectId?: string;
  limit?: number;
};

Layer 5: Editable Protocol – The “Aha” Moment

The author argues that a true Design Harness must output a structured, editable design tree instead of a flat image. In KinoViz the pipeline maps HTML → SVG → Fabric.js scene, preserving semantic attributes such as data-kv-id, data-kv-role, data-kv-group, data-kv-layout, data-kv-tone, and data-kv-anim. These anchors enable stable editing, testing, and re‑theming.

<div
  data-kv-id="hero-root"
  data-kv-group="hero"
  data-kv-layout="stack"
  data-kv-tone="background"
  style="display:flex; flex-direction:column; gap:24px;"
>
  <span data-kv-id="hero-eyebrow" data-kv-role="eyebrow" data-kv-tone="accent" data-kv-anim="fadeInDown">PRIVATE BETA</span>
  <span data-kv-id="hero-headline" data-kv-role="headline" data-kv-tone="ink" data-kv-anim="fadeInUp">Ship faster with signal</span>
  <span data-kv-id="hero-cta" data-kv-role="cta" data-kv-tone="accent" data-kv-emphasis="true" data-kv-anim="scaleIn">Start free trial</span>
</div>

These data attributes form the “editable protocol” that lets the AI output be re‑imported into the design tool.

Layer 6: Motion Harness – More Complex Than Static Design

Motion adds time, keyframes, easing, staggering, loops, and export consistency concerns. The author proposes four sub‑components:

Plan Preview – a pre‑execution proposal of background, shots, elements, timing, motion personality, and open questions.

Motion Budget – separate duration and intensity budgets for feedback, state confirmation, entrance, transition, and loops.

Keyframe Sanity Check – ensure the canvas isn’t empty, keyframes exist, durations are reasonable, and no flashing exceeds three times per second.

Export Parity – verify that editor preview matches the final exported video or image.

const designChecks = [
  "canvas_is_not_empty",
  "all_visible_elements_have_semantic_id",
  "headline_uses_stance_display_font",
  "no_default_ai_indigo",
  "no_placeholder_copy",
  "animation_duration_within_budget",
  "no_flash_more_than_3_times_per_second",
  "keyframes_exist_for_animated_elements",
  "editor_preview_matches_export_snapshot"
];

Verification Loop – Don’t Hand All Evaluation to the Model

Self‑reflection alone is insufficient. The author recommends a hybrid verification pipeline: rule‑based checks for obvious errors, screenshot/pixel comparison for rendering differences, end‑to‑end tests for tool‑chain integrity, and human polish for aesthetic judgment. Model reviews are limited to explanation, suggestion, and alternative generation.

Future Extensions

Potential additions include Eval Harness (measuring generation quality, editability, tool hit‑rate, rollback success, export consistency, user iteration count, and human polish cost), Sandbox/Execution Separation (isolating stateful execution), Memory Governance (brand, system, and user‑preference memories), Sub‑agents/Specialists (brief analyst, layout designer, motion designer, asset finder, reviewer, export verifier), and Trace/Observability (recording why a stance was chosen, which tool was called, which check failed, time spent, and where the user cancelled).

Conclusion

Design Harness is not about prompting tricks; it is an external system that enables a model to participate in a real design production pipeline. It answers six questions: how the model understands the goal, inherits aesthetics, discovers tools, preserves editable structure, handles motion, and validates/rolls back/export for iterative work. When these engineering pieces are in place, AI moves from “can generate” to “can deliver.”