Paint-Anything: Unified Any-Color Control for Image Generation and Editing

Problem

The paper addresses a significant gap in the capability of existing image generation and editing models, specifically the ability to specify an object's target color using any 24-bit hex value. This limitation restricts the flexibility and precision in color manipulation tasks, which are crucial for various applications in computer graphics and design. The work is presented as a preprint and has not undergone peer review.

Method

The authors introduce the Paint-Anything architecture, which is designed to facilitate unified any-color control for image generation and editing. The training dataset, Paint-500K, is constructed from real images through a combination of object grounding, perceptual color labeling, and editing-pair synthesis. The training mechanism employs a two-phase approach:

• High-noise timesteps utilize pure-color anchors, ensuring that the pixels correspond to the specified hex values.
• Low-noise training is conducted on natural images to enhance the model's ability to generalize across diverse visual contexts.

The model is evaluated on the Any Color Benchmark (ACBench), which includes two components: ACBench-T2I for text-to-image generation and ACBench-Edit for image editing tasks.

Results

The results demonstrate substantial improvements over the baseline model:

• ACBench-T2I Score: 85.3% improvement compared to the base model.
• ACBench-Edit Score: 28.3% improvement compared to the base model. Additionally, Paint-Anything achieves the highest average score on the CompColor benchmark among the compared methods, indicating its superior performance in color manipulation tasks.

Limitations

The authors acknowledge that the real-image labels used in the training dataset are only approximate due to the presence of shadows, which can affect the accuracy of color representation. This limitation may impact the model's performance in scenarios where precise color matching is critical. Furthermore, the paper does not discuss potential issues related to the generalization of the model to unseen objects or colors outside the training distribution.

Why it matters

The implications of this work are significant for downstream applications in image generation and editing, particularly in fields requiring high fidelity in color representation, such as digital art, product design, and virtual reality. By enabling users to specify any color with precision, Paint-Anything could enhance creative workflows and expand the capabilities of existing image synthesis tools.