arxiv:2607.18227

FlowMimic: Mask-free Visual Editing and Generation with Pixel-pair Warped Flow Field for Online Video Editing Data Generation and Modality Mimicry

Published on Jul 20

· Submitted by

yichen he on Jul 21

ByteDance Seed

Upvote

Authors:

Dingyun Zhang ,

Abstract

In line with the prevailing direction of vision research, we explore the integration of both generation and editing capabilities for video and image modalities within a single model. Current approaches to collecting video editing data typically depend on labour-intensive, time-consuming curated procedures--involving object mask annotation, the use of error-introducing pair synthesis via I2V model and ControlNet-like guidance, and VLM-based quality filtering or refinement--and demonstrate limited task scalability. As a result, the diversity of editing tasks remains substantially narrower than that available for image editing models. We develop a pixel-pair temporal warped flow field that can directly generate corresponding video editing samples in real time from image editing samples, and we demonstrate across multiple levels of video editing tasks that a model can learn video editing using only such data. We regard the image modality as a particular form of the video modality. Accordingly, we design a modality mimic generation loss and a modality mimic editing loss to relatively align the capabilities--and thereby the output distributions--of the two modalities through mutual imitation. Moreover, language-based visual editing entails the comprehension of the editing instruction and the reference visual content, the localization of the region corresponding to that instruction within the reference visual contents, and the modification of that region alone. Existing approaches predominantly rely on external aids, such as fine-tuning an additional MLLM or explicitly supplying a mask sequence as auxiliary input during inference. In contrast, we aspire for the model to internalize this capability. To that end, we introduce sense-related tasks--for instance, referring expression segmentation--along with corresponding editing-region-aware latent-level loss and attention-level loss.

View arXiv page View PDF Add to collection

Community

hyc2026

Paper submitter 4 days ago

This comment has been hidden

librarian-bot

3 days ago

This comment has been hidden

FlowMimic

about 16 hours ago

•

edited about 16 hours ago

🦋 FlowMimic: Mask-free Visual Editing and Generation with Pixel-pair Warped Flow Field for Online Video Editing Data Generation and Modality Mimicry

✨ Information

☘️ 2026/07/21: main.pdf in the dataset repository is the uncompressed file of the corresponding arXiv file. Thank you for reading, and have a nice day 🌷

🍃 Brief Introduction

Please refer to the paper for details.

We abstract the essence of video editing learning as the temporal maintaining of pixel-level editing effect, propose and model a temporal point-flow warped field to scalably generate video editing data in real time from image editing samples alone, demonstrate the effectiveness of this pixel-aligned paradigm towards learning video editing across diverse-level editing tasks, and reveal its promising extensibility to high-level video tasks such as segmentation-related tasks, low-level video tasks such as video deblurring, and controllable video generation such as condition maps to video.
We posit that a model trained concurrently on both image and video modalities for generation and editing tasks should yield relatively aligned output distributions across the two modalities, as the image modality is essentially a special case of a video modality. Accordingly, we design a modality mimic generation loss and a modality mimic editing loss.
The capacity for language-based visual editing fundamentally requires a model to comprehend the instruction and the reference visual inputs, localize the corresponding region in the reference visual input, and modify it exclusively. Rather than relying on fine-tuning additional modules such as VLMs with connector adaptation or assisted mask sequence inputs, we endow the model with this internalized ability through introducing sense-related tasks and corresponding editing-region-aware latent-level and attention-level losses.
Viewed from the perspective of on-policy self-distillation (OPSD), the mimicry procedure underlying the modality mimicry paradigm, which we explored last April, essentially constitutes a form of OPSD for flow matching models. The editing and generation results of modality mimicry demonstrate that OPSD of flow matching models is indeed feasible. The mimicry methods described in this research—for instance, the modality mimic generation feature score loss or feature loss—can also be applied to the OPSD of image or video flow matching models, such as Z-image and Stable Diffusion 3.5 Medium.

🎞️ Video Editing Results

More results of various video editing tasks are demonstrated in the paper—all achieved by the same 1.3B model.

Changing Word

Non-rigid Element Replacement and Adding Visual Effect

Trajectory-guided Conditional Input Editing

121 Inference Frames: 3× above the Training Average - Instruction: "Put pale purple crystalline magical hats on the two men."

🌿 Citation

If you find this research useful for your work, or you would like to build upon the findings or contents described herein, the following BibTeX could be used for citation:

@article{zhang2026flowmimic,
  title   = {FlowMimic: Mask-free Visual Editing and Generation with Pixel-pair Warped Flow Field for Online Video Editing Data Generation and Modality Mimicry},
  author  = {Zhang, Dingyun and Gong, Lixue and Liu, Wei},
  journal = {arXiv preprint arXiv:2607.18227},
  year    = {2026}
}