skillfed
RESEARCH

Image editing models improve more from diverse edit concepts than from more images

on: Unlocking the Potential of Image Editing via Concept Scaling and Dense Supervision

The central argument here is that image editing models have been scaling the wrong thing. Most existing work treats editing as a variant of text-to-image generation and scales up source image diversity accordingly. This paper argues that the actual bottleneck is edit concept diversity — the range and granularity of modifications a model has been trained to perform — and that fixing this matters more than adding more source images.

The evidence for distribution collapse is concrete. When existing synthesis approaches let a vision-language model stochastically sample from coarse categories, the top five styles in a "style transfer" category end up dominating nearly three-quarters of generated instructions, leaving dozens of others below one percent each. The proposed remedy is a structured concept library: an LLM-expanded, human-refined taxonomy of over 1,000 fine-grained edit categories, built iteratively and then used to control sampling weights during data generation. The resulting dataset, ConceptEdit-12M, contains 12 million verified editing pairs — tied for the largest in the field by the authors' own accounting.

The second contribution addresses a different inefficiency. In a typical single-edit training pair, most pixels are static background; the model is mostly learning identity mapping rather than transformation. The fix is compositional: pack multiple spatially non-interfering edits into a single image pair, forcing the model to learn from denser supervision per forward pass. A 1:1 mix of single-concept and composite samples improves ImgEdit-Bench scores by around 0.15 points across training scales, with larger gains in action, replacement, and adjustment categories specifically. Convergence curves show the composite strategy reaching equivalent performance with fewer samples.

Quality filtering gets its own mechanism. Rather than applying generic VLM prompts to assess whether an edit succeeded, the framework generates instance-specific question-answer pairs at instruction time, then uses those to probe localized regions during verification. Compared to generic validation, this approach reduces false negatives substantially and improves F1 by around 21 percentage points, while adding only a marginal per-sample latency overhead — the image generation step dominates runtime by a wide margin.

The ablation on concept granularity is the most instructive part. Training with 10 coarse categories, 500, and 1,000-plus shows monotonic improvement: at 5M samples, the 1,000-category variant outperforms the 10-category baseline by 0.33 points on ImgEdit-Bench. The gains are not uniform — style, adjustment, and replacement tasks benefit most, which makes sense given how much those categories fragment under fine-grained labeling.

ConceptEdit-Bench, the accompanying evaluation suite, mirrors the taxonomy: 1,000 distinct categories, each representing a unique operation. Existing benchmarks top out around 50 generic types, which the authors argue masks failures on long-tail cases. Benchmark results show that even strong closed-source models drop noticeably on portrait and composition categories — precisely the ones requiring detailed world knowledge or spatial reasoning.

The core insight — that concept distribution, not image count, drives generalization — is both simple and underexplored. The execution is thorough enough that the ablations actually support the claim.

Scaling edit concept diversity, not source image count, is what actually drives generalization in instruction-based image editing.

Sources & links