How AI Turns UI Screenshots into Ready‑to‑Edit Front‑End Code
This article explains the Screenshot‑to‑Code project, detailing how AI‑driven image recognition converts UI screenshots into editable HTML, CSS, and JavaScript, describes the front‑end (React + Vite + Radix‑UI) and back‑end (Python + Poetry) architecture, showcases prompt engineering, and provides step‑by‑step setup instructions.
Project Introduction
"Screenshot-to-code" is an AI‑powered technique that automatically converts UI screenshots into editable front‑end code (HTML, CSS, JavaScript). It identifies UI elements such as buttons, text fields, and lists, then generates corresponding code, offering automation, efficiency, real‑time feedback, and customization.
Core Technology – Frontend
The front‑end uses React, Vite, and Radix‑UI. The main logic resides in App.tsx , which communicates with the backend via WebSocket to receive generated code. Key libraries include Codemirror (code editor), react‑hot‑toast (toast notifications), and Zustand (state management).
The UI flow is illustrated below:
Code preview is rendered in an iframe using the srcdoc attribute, with throttling to improve performance. For fine‑tuning generated pages, click events on the body are captured, passed to an EditPopup component, and the target DOM element is used to retrieve and modify its HTML.
Core Technology – Backend
The backend is written in Python and managed with Poetry. Important files include:
routes/generate_code.py : main endpoint for code generation.
llm.py : interface to large language models (e.g., OpenAI GPT).
The generation flow calls the GPT model via AsyncOpenAI . Prompts guide the model to produce Tailwind, Bootstrap, React, Ionic, Vue, or SVG code that matches the screenshot exactly.
<code>HTML_TAILWIND_SYSTEM_PROMPT = """
You are an expert Tailwind developer...
"""
BOOTSTRAP_SYSTEM_PROMPT = """
You are an expert Bootstrap developer...
"""
REACT_TAILWIND_SYSTEM_PROMPT = """
You are an expert React/Tailwind developer...
"""
... (other prompts omitted for brevity) ...
</code>These prompts enforce exact visual matching, use placeholder images from https://placehold.co , and require returning complete <html> code without comments.
Setup Instructions
1. Prepare Python 3.10+, install Poetry and Pyright.
2. Run the following commands in the backend directory:
<code>cd backend
echo "OPENAI_API_KEY=sk-your-key" > .env
poetry install
poetry shell
poetry run uvicorn main:app --reload --port 7001</code>Configure .env with your OpenAI API key. To run without GPT, set MOCK=true and start the server similarly. The project supports hot‑reloading.
Conclusion
The Screenshot‑to‑Code tool demonstrates a practical AI‑driven workflow for rapid UI prototyping, allowing developers to focus on complex features while the AI handles boilerplate code. It can be extended to support additional component libraries and styling frameworks.
Code Mala Tang
Read source code together, write articles together, and enjoy spicy hot pot together.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.