img2threejs: Turn One Image into Animated Three.js Code with 8-Pass AI Pipeline

Project Overview

img2threejs is an AI Agent skill compatible with Claude Code, Codex, and OpenCode that converts a single reference image into a pure-code, procedural Three.js model. Its core mechanism separates deterministic scripting from visual judgment: 16 Python scripts handle validation, state management, and code generation, while the AI vision capability only scores side-by-side comparison screenshots. The tool targets rapid prototyping, editable 3D assets, and animation-ready models.

Core Capabilities

Single-image reconstruction: One reference photo → complete Three.js Group factory code.

8-pass staged sculpting: blockout → structural → form → material → surface → lighting → interaction → optimization.

AI visual self-correction: Each pass captures a screenshot, compares it against the reference, and requires a global score ≥ 0.7 plus per-key-feature thresholds before unlocking the next pass.

Zero dependencies: Pure Python 3.10+ standard library; PNG read/write implemented with struct + zlib, no pip install required.

Animation-ready output: Generated models include pivot points, sockets, colliders, and destruction groups at runtime.

Character support (v1.2+): Human anatomy track with head-to-body ratio, facial landmarks (eye line, nose base, mouth line, hairline), and proportion-locking pass.

Quality contracts: Four complexity tiers (simple → ultra-complex) define minimum spec depth to prevent under-specified builds.

Detail inventory: 3×3 grid scans the reference image zone-by-zone to catch identity-defining features (highlights, chamfers, rivets, panel lines, weathering).

Target Users & Scenarios

Game/frontend developers needing interactive 3D props from concept art.

AI Agent users who want to invoke image-to-3D directly inside Claude Code or Codex.

3D artists seeking a procedural model skeleton for manual refinement.

Quick Start

Environment

Python 3.10+ (stdlib only).

One of: Claude Code, Codex, OpenCode.

Browser for rendering screenshots.

Installation

git clone https://github.com/hoainho/img2threejs.git ~/.codex/skills/img2threejs

Usage Example

# In Agent, attach image and invoke
› $img2threejs [Image #1]

# Or run stage scripts manually
python3 forge/stage1_intake/probe_image.py <image>
python3 forge/stage2_spec/new_pre_spec_assessment.py "Name" --image <img> --out assessment.json
python3 forge/stage2_spec/new_sculpt_spec.py "Name" --image <img> --assessment assessment.json --out spec.json
python3 forge/stage2_spec/validate_sculpt_spec.py spec.json --strict-quality
python3 forge/stage3_build/generate_threejs_factory.py spec.json --out src/createObjectModel.ts

After generation, a project is created; launch it with Codex to see the live result.

Function List (10 Core Features)

Smart image probing: Parses PNG/JPEG/GIF/WebP/BMP/TIFF metadata (dimensions, aspect ratio, color space) with zero dependencies to decide 3D-reconstruction suitability.

Quality contract generation: Auto-assesses object complexity (simple/moderate/complex/ultra-complex) and emits a qualityContract specifying target detail count, material layers, and component hierarchy depth.

Detail inventory scan: Splits reference into 3×3 grid or component regions; enumerates identity-defining details (highlights, chamfers, rivets, panel lines, stains) and maps each to a component or material field.

Character anatomy extraction: For character/hybrid domains, extracts head-to-body ratio, facial landmarks, and body proportions as structured data for the proportion-locking pass.

Sculpt spec generation: Produces ObjectSculptSpec JSON (schema 2.0) containing full component tree, material definitions, 8-pass build config, and self-correction loop parameters — the pipeline's blueprint.

Strict spec validation: Checks field completeness, score ranges, material references, component ID chains; --strict-quality upgrades quality warnings to hard errors, blocking shallow specs from code generation.

8-pass orchestration: Manages sequential unlocking of the eight passes; each must pass screenshot + score + feature triple-gate.

Three.js code generation: Emits TypeScript Three.js Group factory skeleton based on spec and currently unlocked pass; supports 13 primitives (box, sphere, cylinder, lathe, extrude, etc.) and only generates code for the current pass's hierarchy level.

AI visual review log: Appends Agent's visual scores (global + per-feature), render screenshot paths, comparison sheet paths, and action decision (continue/refine-code/stop) to reviewHistory for traceable quality audit.

Auto comparison sheet: Stitches current pass render side-by-side with original reference for AI "spot-the-difference" review; script only handles compositing, scoring fully delegated to Agent vision.

Overall Architecture

Conclusion first: img2threejs uses a "four-stage pipeline + AI visual loop" architecture. Scripts own deterministic gating; AI owns visual judgment. Responsibilities are strictly separated.

Four Stages (under forge/ )

stage1_intake (intake layer): image probing, 3×3 detail inventory, character landmarks, PBR evidence extraction, camera pose solving.

stage2_spec (spec layer): complexity grading, quality contract, ObjectSculptSpec generation and validation.

stage3_build (build layer): 8-pass orchestration, pass-gated code generation, projection texture baking.

stage4_review (review layer): comparison sheet compositing, AI visual score logging.

Shared module _shared/feature_acceptance_policy.py provides per-feature scoring gate strategy used by both stage3 and stage4.

Module Responsibilities

stage1_intake: Image metadata probe, 3×3 detail inventory, character landmarks, PBR evidence, camera pose.

stage2_spec: Complexity grading, quality contract, ObjectSculptSpec generation, strict-quality validation.

stage3_build: 8-pass state management, pass-gated code generation, projection textures.

stage4_review: Comparison sheet compositing, reviewHistory append, self-correction decision log.

_shared: Per-feature scoring gate policy ( feature_gate_failures).

grimoire/: Detailed rule docs for each gate (validation, materials, attachments, self-correction, etc.).

Architecture diagram included in source (not reproduced here).

Core Call Chain

The pipeline's core design is "scripts gate, AI judges": Python handles all deterministic logic (validation, state, codegen); AI vision only looks at a comparison image and scores. Every pass must clear screenshot + score + feature triple-gate before the next pass unlocks. A data-flow diagram is provided in the source.

Comparison with Similar Projects

Core positioning: img2threejs = single image → procedural code model; Meshy AI / Tripo3D = image/text → mesh file.

Output format: img2threejs = TypeScript + JSON (diffable); Meshy AI / Tripo3D = GLB/FBX/OBJ (binary).

Language: img2threejs = Python (pure stdlib); Meshy AI / Tripo3D = closed-source SaaS.

Deployment: img2threejs = local Agent skill; Meshy AI / Tripo3D = cloud API.

Animation support: img2threejs = native (pivot/socket/collider); Meshy AI / Tripo3D = manual rigging required.

Extensibility: img2threejs = open source, custom passes/gates; Meshy AI / Tripo3D = not extensible.

License: img2threejs = MIT; Meshy AI / Tripo3D = commercial.

Documentation quality: img2threejs = detailed (SKILL.md + grimoire/); Meshy AI / Tripo3D = average.

No reproducible public benchmarks found. img2threejs's advantage lies in editable code output rather than binary mesh, suiting version-controlled and animation-heavy workflows.

Common Pitfalls & Avoidance

Low image resolution: probe_image.py warns when width or height < 512 px ( forge/stage1_intake/probe_image.py:139). Geometry/material details become unreliable; use ≥ 1024 px reference images.

Extreme aspect ratios: Aspect ratio > 3.0 or < 0.33 flagged as conditional ( probe_image.py:141); object may be cropped or surrounded by large blank areas.

strict-quality interception: Complex objects with only a root component, no repetition systems, and no local material overrides are blocked by --strict-quality even if JSON is valid ( SKILL.md:75-76). This is by design, not a bug.

Non-PNG images: build_detail_inventory.py and make_comparison_sheet.py natively decode only PNG (hand-written struct + zlib). Other formats attempt macOS sips conversion ( build_detail_inventory.py:147-151); on Linux, pre-convert to PNG.

Single-view limitation: One image cannot reveal hidden surfaces; output is explicitly labeled approximate/stylized ( SKILL.md:33-34), not a 100% faithful reconstruction.

Summary

img2threejs is an AI Agent skill that rebuilds a single reference image into a pure-code, animatable, quality-gated Three.js procedural model. Its core strength is the "scripts gate, AI judges" division of labor, delivering diffable TypeScript instead of binary meshes.

One-line pitch: One image → animated 3D code.

Best fit:

Game/frontend developers rapidly generating interactive 3D props.

Teams needing version-controllable, editable 3D assets.

AI Agent users wanting in-chat image-to-3D.

References

- Repository: https://github.com/hoainho/img2threejs
- Live demo gallery: https://hoainho.github.io/img2threejs-showcase/
- Technical spec: docs/UPGRADE_PLAN.md
- Token cost analysis: docs/TOKEN_COST.md