AIGC Poster Generation Project: Methods and Optimizations

The Inspiration Artist project leverages AIGC drawing capabilities to create a low‑threshold, high‑fun poster design competition for new product promotion.

Background: generate product posters (e.g., Chanel No.5) using large language models (Tongyi Qianwen) for prompts and focus on image generation.

Four visual styles are supported: product poster, Pixar, anime, realistic. The team selected Stable Diffusion (SD) as the core algorithm due to its open‑source nature and extensibility, implementing VAE, ControlNet, LoRA, embeddings, warm‑up and auto‑predict features via the diffusers library.

Four solution prototypes were explored:

SD + Outpainting – fixes background while keeping product fixed.

SD Inpainting + Reference Only – injects product image into UNet attention.

Reference‑based Diffusion (PBE, IP‑Adapter, AnyDoor) – generates similar products from a reference.

SD + LoRA/DreamBooth – fine‑tunes the model to embed product appearance.

Optimization directions include:

Enhancing VAE to preserve fine details.

Applying super‑resolution to improve text clarity.

Copy‑and‑paste of extracted text regions onto generated images.

The final offline‑online pipeline combines a background library, multi‑angle product images, ControlNet‑guided generation, SAM/LAMA masking, and light‑shadow projection to produce high‑quality posters.

Results: detail restoration above 90 %, generation success rate >95 %, and inference speed of 3–5 seconds per image on an A10 GPU.

Future work explores better occlusion handling, proportion harmony (e.g., GLIGEN), and VAE‑free consistency decoders.