How to Create Your Own AI Digital Avatar with Stable Diffusion and ControlNet

This article presents an efficient, easy‑to‑use, low‑cost, and secure way to create personal “digital avatars” using AI image generation tools. It compares the popular AI photo mini‑programs (which charge per use and require cloud processing) with a self‑hosted solution that runs entirely on a local machine, preserving data privacy.

Three Core Questions

The workflow is built around answering three questions:

What to draw? – Define the desired content with textual prompts.

What to use? – Choose the source image(s) and the region to be re‑drawn.

What to reference? – Provide auxiliary information (e.g., pose) to guide the generation.

Three Corresponding Steps

What to draw: Write positive and negative prompts in English. Example positive prompt:

1girl, wearing song hanfu, wearing delicate traditional chinese hairpins, chinese palace background, masterpiece, best quality, ultra-detailed

. Negative prompt: blurry, ugly, bad quality, cartoon, anime, NSFW, nude.

What to use: Upload a clear half‑body portrait, select the region to keep (the face) and mark the area to be re‑drawn using the inpaint mask ("not masked" mode). The original photo serves as the material for img2img inpainting.

What to reference: Optionally enable the ControlNet plugin with the openpose pre‑processor to feed the model with pose information extracted from the same portrait, improving the realism of generated limbs and posture.

Step‑by‑Step Guide

1. Install Stable Diffusion WebUI (installation details omitted for brevity).

2. Prepare prompts as described above.

3. Upload the source image and use the black brush to mask the facial region you want to preserve (see

). Choose the option “inpaint not masked” so that everything except the masked face will be regenerated.

4. Enable ControlNet (optional) : Activate the plugin, select the openpose model, and load the same portrait as the reference image. This lets the AI learn the pose and produce more natural results (see

).

5. Generate the image . The result is a set of stylized avatars (e.g., ancient‑style, modern school uniform, futuristic) that can be further refined by repeating the mask‑and‑prompt cycle.

Customising Styles

To change the appearance, simply edit the positive prompt. For a school‑uniform look, replace the clothing description with

1girl, wearing school uniform, ponytail, campus background, masterpiece, best quality, ultra-detailed

while keeping the negative prompt unchanged. Click “Generate” to obtain the new avatar (see

).

Comparison with Commercial AI Photo Apps

Both approaches take a portrait as input and output stylized images. The commercial mini‑programs often deliver higher visual fidelity but require per‑use payment and upload photos to external servers, raising cost and privacy concerns. The self‑hosted Stable Diffusion workflow incurs no per‑image fee, offers unlimited customization, and keeps all data on the local machine, though it demands more technical setup.

Further Resources

For readers interested in deploying Stable Diffusion and fine‑tuning parameters, see the Tencent Cloud developer article “Stable Diffusion Beginners’ Handbook” (link retained as a technical reference).

By following the steps above, users can iteratively create a diverse collection of digital avatars spanning ancient costumes, modern school outfits, futuristic designs, and more, all while maintaining control over data security and cost.