Researchers have introduced CommerceGen-Studio, a new end-to-end framework designed to automate the generation of main images for e-commerce products. This system aims to streamline the process of creating high-quality, commercially viable product visuals.
The framework operates through a three-stage pipeline. The initial stage involves information extraction, where heterogeneous metadata for Stock Keeping Units (SKUs) is distilled into concise lists of selling points. This process utilizes an iterative loop involving Vision-Language Models (VLMs) and Large Language Models (LLMs), guided by a reference-free rubric for evaluation. Following this, a semantic rewriting stage transforms the extracted information into compelling text. The final stage is multi-condition generation, where the actual images are produced.
A key component of the framework is the Prompt-Eval-Refine-Loop Agent (PERL-Agent). This agent is responsible for distilling SKU metadata into effective selling points through a closed loop of VLM and LLM interactions. The quality of these selling points is assessed using a rubric that does not require a reference. The Layout Thinker Optimization (LTO) component then aligns the rewritten text with both the generated caption and the final image quality. This alignment is achieved through alternating policy optimization specific to the domain.
The image generation itself is handled by SKU-DiT, a diffusion transformer model. This model is trained using a progressive training method called Reinforce-Distill-Reinforce (RDR). SKU-DiT is conditioned on multiple inputs, including the white-background product image, a trademark crop of the product, and the rewritten text descriptions. This multi-condition approach ensures that the generated images are relevant and adhere to specific product details.
Previous efforts in automated e-commerce image generation have focused on various aspects, such as generating product image sequences or creating textual listings from images. For instance, a framework developed for automatic generation of product-image sequences in e-commerce aimed to reduce manual effort and costs associated with image selection and organization, achieving a 13.6% reject rate by December 2021. Another system proposed a multimodal, multitask approach for generating e-commerce text listings from images, significantly reducing factual hallucination rates by using a hierarchical generation process. Systems have also been developed to automate the generation of marketing imagery, focusing on brand safety and scalable workflows through agentic pipelines.
The CommerceGen-Studio framework builds upon these advancements by providing an integrated solution for main image generation. Its structured pipeline, combining information extraction, semantic rewriting, and multi-condition generation, addresses the need for efficient and high-quality visual content in e-commerce. The use of advanced agents and diffusion models, along with specific training methodologies, aims to produce commercially appropriate images.