Master Stable Diffusion: From Prompt Basics to Advanced ControlNet Techniques

1. Diffusion: How a picture is formed

Diffusion models generate images by first adding noise to a picture (like closing your eyes) and then gradually denoising it (opening your eyes) to produce the final result, allowing the AI to reinterpret the content.

2. AI painting = Prompt + Parameters + Model

The Stable Diffusion WebUI provides a graphical interface where you input prompts, adjust parameters, and select models to generate images.

2.1 Prompt: the spell for AI

Prompts are textual instructions (or images) that guide the AI. They can include positive and negative terms to specify desired and undesired elements. Effective prompts describe subject, style, color, and texture.

2.2 Parameters: controlling the spell

Sampler – algorithm that iteratively denoises the image (e.g., DPM++ 2M Karras, DDIM, Euler).

Steps – number of denoising iterations; typical values 20‑30.

Face restoration – improves facial details.

Width & Height – image resolution; higher values give more detail but require more VRAM.

Tiling – generates seamless textures; enable only when needed.

CFG Scale – strength of prompt adherence (1‑30, usually 7‑12).

Seed – random number that makes results reproducible; -1 generates a new seed each run.

Batch size – number of images generated per request; lower values reduce memory usage.

2.3 Model: the magic source

Models provide the learned knowledge. The base model (e.g., sd‑v1.5.ckpt) contains general image concepts. VAE enhances color and lighting. LoRA, Embeddings, and Hypernetwork are lightweight add‑ons that specialize the style or subject.

3. High‑resolution upscaling

Upscaling scripts and the “High‑Resolution Fix” re‑run the diffusion process on a low‑resolution image to produce a larger, sharper result. Tile overlap (e.g., 64 px) smooths seams between generated patches.

4. Local repaint (inpaint)

Inpainting lets you mask a specific region and ask the AI to redraw only that part, preserving the rest of the image. The mask acts as a “blackout” area that the model fills based on the new prompt and parameters.

5. ControlNet: industrial revolution

ControlNet adds extra conditioning to diffusion models, allowing precise control over pose, edges, depth, etc. The Canny pre‑processor extracts image edges for line‑art generation. Multi‑ControlNet enables stacking several conditioning models (e.g., lineart + depth) for complex control.

6. AI will not replace you, but those who master it will

AI painting has moved from a novelty to a production‑grade tool adopted by major tech companies. Understanding diffusion, prompt engineering, and extensions like ControlNet empowers creators to harness AI effectively.