Consistency Flow Matching: Defining Straight Flows with Velocity Consistency

Consistency Flow Matching: Defining Straight Flows with Velocity Consistency

2 Jul 2024 | Ling Yang, Zixiang Zhang, Zhilong Zhang, Xingchao Liu, Minkai Xu, Wentao Zhang, Chenlin Meng, Stefano Ermon, Bin Cui
Consistency Flow Matching (Consistency-FM) is a novel method for defining straight flows via velocity consistency in flow matching (FM). Unlike previous approaches that rely on iterative rectification or optimal transport, Consistency-FM enforces self-consistency in the velocity field to directly define straight flows from different times to the same endpoint. This approach enhances model expressiveness through multi-segment training, enabling better trade-offs between sampling quality and speed. Experiments show that Consistency-FM converges 4.4 times faster than consistency models and 1.7 times faster than rectified flow models while achieving superior generation quality. The method also facilitates distillation from pre-trained FM models, improving the balance between sampling speed and quality. Theoretical analysis supports the effectiveness of velocity consistency in learning straight flows, and the method demonstrates strong performance on CIFAR-10, CelebA-HQ, and AFHQ-Cat datasets, outperforming existing flow and diffusion models in both efficiency and quality. Future work includes extending the method to text-to-image generation and exploring distillation with pre-trained diffusion models.Consistency Flow Matching (Consistency-FM) is a novel method for defining straight flows via velocity consistency in flow matching (FM). Unlike previous approaches that rely on iterative rectification or optimal transport, Consistency-FM enforces self-consistency in the velocity field to directly define straight flows from different times to the same endpoint. This approach enhances model expressiveness through multi-segment training, enabling better trade-offs between sampling quality and speed. Experiments show that Consistency-FM converges 4.4 times faster than consistency models and 1.7 times faster than rectified flow models while achieving superior generation quality. The method also facilitates distillation from pre-trained FM models, improving the balance between sampling speed and quality. Theoretical analysis supports the effectiveness of velocity consistency in learning straight flows, and the method demonstrates strong performance on CIFAR-10, CelebA-HQ, and AFHQ-Cat datasets, outperforming existing flow and diffusion models in both efficiency and quality. Future work includes extending the method to text-to-image generation and exploring distillation with pre-trained diffusion models.
Reach us at info@study.space
Understanding Consistency Flow Matching%3A Defining Straight Flows with Velocity Consistency