diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..938fec3 --- /dev/null +++ b/.gitignore @@ -0,0 +1,2 @@ +.DS_Store +*.pdf diff --git a/index.html b/index.html index 385fd1a..7e55686 100644 --- a/index.html +++ b/index.html @@ -52,28 +52,44 @@
-

Academic Project Page

+

TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models

- First Author*, + Yushi Huang1,2*, - Second Author*, + Ruihao Gong1,2*, - Third Author + Jing Liu2,3, + + Tianlong Chen4, + + Xianglong Liu1📧
- Institution Name
Conferance name and year -
*Indicates Equal Contribution + Beihang University1, + SenseTime Research2, + Monash University3 + UT Austin4 +
+ +
+ CVPR 2024 Highlight (Acceptence Rate 2.8%)
+ *Equal Contribution, 📧Corresponding Author
+ huangyushi@sensetime.com, + gongruihao@sensetime.com, + liujing5@sensetime.com,
+ tianlong.chen@utexas.edu, + xlliu@buaa.edu.cn
- @@ -83,7 +99,7 @@

Academic Project Page

 - + - @@ -106,7 +122,7 @@

Academic Project Page

- @@ -123,22 +139,26 @@

Academic Project Page

-
- -

- Aliquam vitae elit ullamcorper tellus egestas pellentesque. Ut lacus tellus, maximus vel lectus at, placerat pretium mi. Maecenas dignissim tincidunt vestibulum. Sed consequat hendrerit nisl ut maximus. -

+ + +

+ In TFMQ-DM, our contributions are as follows: + + 1) we discover that existing quantization methods suffer from temporal feature disturbance, disrupting the denoising trajectory of diffusion models and significantly affecting the quality of generated images. + + 2) We reveal that the disturbance comes from two aspects: inappropriate reconstruction target and unaware of finite activations. Both inducements ignore the special characteristics of time information-related modules。 + + 3) An advanced framework (TFMQ-DM) is proposed, consisting of temporal information aware reconstruction (TIAR) for weight quantization and finite set calibration (FSC) for activation quantization. Both are based on a Temporal Information Block specially devised for diffusion models. + + 4) Extensive experiments on various datasets show that our novel framework achieves a new state-of-the-art result in PTQ of diffusion models, especially under 4-bit weight quantization, and significantly accelerates quantization time. For some hard tasks, e.g., CelebA-HQ 256 × 256, our method reduces the FID score by 6.71 +

+
-
@@ -148,8 +168,27 @@

Abstract

- Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin ullamcorper tellus sed ante aliquam tempus. Etiam porttitor urna feugiat nibh elementum, et tempor dolor mattis. Donec accumsan enim augue, a vulputate nisi sodales sit amet. Proin bibendum ex eget mauris cursus euismod nec et nibh. Maecenas ac gravida ante, nec cursus dui. Vivamus purus nibh, placerat ac purus eget, sagittis vestibulum metus. Sed vestibulum bibendum lectus gravida commodo. Pellentesque auctor leo vitae sagittis suscipit. -

+ The Diffusion model, a prevalent framework for image + generation, encounters significant challenges in terms of + broad applicability due to its extended inference times and + substantial memory requirements. Efficient Post-training + Quantization (PTQ) is pivotal for addressing these issues in + traditional models. Different from traditional models, diffusion models heavily depend on the time-step t to achieve + satisfactory multi-round denoising. Usually, t from the finite set {1, . . . , T} is encoded to a temporal feature by a + few modules totally irrespective of the sampling data. However, existing PTQ methods do not optimize these modules + separately. They adopt inappropriate reconstruction targets + and complex calibration methods, resulting in a severe disturbance of the temporal feature and denoising trajectory, + as well as a low compression efficiency. To solve these, + we propose a Temporal Feature Maintenance Quantization + (TFMQ) framework building upon a Temporal Information + Block which is just related to the time-step t and unrelated + to the sampling data. Powered by the pioneering block design, we devise temporal information aware reconstruction + (TIAR) and finite set calibration (FSC) to align the fullprecision temporal features in a limited time. Equipped with + the framework, we can maintain the most temporal information and ensure the end-to-end generation quality. Extensive experiments on various datasets and diffusion models + prove our state-of-the-art results. Remarkably, our quantization approach, for the first time, achieves model performance nearly on par with the full-precision model under + 4-bit weight quantization. Additionally, our method incurs + almost no extra computational cost and accelerates quantization time by 2.0Ă— on LSUN-Bedrooms 256 Ă— 256 compared to previous works. +

@@ -157,128 +196,151 @@

Abstract

-
-
- - - - -
+
- -

Video Presentation

-
- -
- - -
-
+

Acceleration Results

-
-
-
- - - - -
-
-
-

Another Carousel

- -
-
- - - - - - + +

BibTeX

- 
BibTex Code Here
+ 
@inproceedings{huang2024tfmqdm,
+  title={TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models}, 
+  author={Yushi Huang and Ruihao Gong and Jing Liu and Tianlong Chen and Xianglong Liu},
+  booktitle={The IEEE/CVF Conference on Computer Vision and Pattern Recognition},
+  year={2024}
+}
+
diff --git a/static/images/acceleration.png b/static/images/acceleration.png new file mode 100644 index 0000000..fb9c947 Binary files /dev/null and b/static/images/acceleration.png differ diff --git a/static/images/overview.png b/static/images/overview.png new file mode 100644 index 0000000..13be1b8 Binary files /dev/null and b/static/images/overview.png differ diff --git a/static/images/vis-1_00.png b/static/images/vis-1_00.png new file mode 100644 index 0000000..6d0c626 Binary files /dev/null and b/static/images/vis-1_00.png differ diff --git a/static/images/vis-2_00.png b/static/images/vis-2_00.png new file mode 100644 index 0000000..df87210 Binary files /dev/null and b/static/images/vis-2_00.png differ diff --git a/static/images/vis-3_00.png b/static/images/vis-3_00.png new file mode 100644 index 0000000..47c3ac5 Binary files /dev/null and b/static/images/vis-3_00.png differ diff --git a/static/images/vis-4_00.png b/static/images/vis-4_00.png new file mode 100644 index 0000000..b6c099c Binary files /dev/null and b/static/images/vis-4_00.png differ diff --git a/static/images/vis-5_00.png b/static/images/vis-5_00.png new file mode 100644 index 0000000..9fe81ab Binary files /dev/null and b/static/images/vis-5_00.png differ diff --git a/static/images/vis-6_00.png b/static/images/vis-6_00.png new file mode 100644 index 0000000..adb8b02 Binary files /dev/null and b/static/images/vis-6_00.png differ diff --git a/static/images/vis-7_00.png b/static/images/vis-7_00.png new file mode 100644 index 0000000..26c95e6 Binary files /dev/null and b/static/images/vis-7_00.png differ