Wan2.2-I2V

[Tinyyuumi]

我把wan的代码迁移到wan22图生视频任务上，81帧外推2倍，生成161帧的视频，同样是修改自注意力层attn1。但是出来的效果非常差，视频全是噪点。wan22也是从wan21上训练而来的，模型结构应该基本相同，按理来说不应该有那么大的差异，又或者是该方法不支持图生视频的原因吗？

以下是视频的前几s的其中一帧

[zhuhz22]

Hi @Tinyyuumi , thank you for your attention to our work!

UltraViCo should be able to support Wan2.2 I2V. I think the issue is likely related to the code migration. Could you try the following:

• 1. Generate slightly more than 81 frames, e.g., 97 frames — does it still have issues?
• 2. For 161 frames, set alpha = 1 (i.e., no modification) — does it have issues?
• 3. For 161 frames, set alpha = 0.98 (i.e., a very small modification) — does it have issues?

Also:

• For I2V we need to ensure the attention on the conditional image part does not decay. Have you taken this into account?
• Could you provide a video (rather than a single frame as it is now) so we can better observe the issue?

I’m happy to discuss this further with you. Since I’m currently rushing an ICML submission, I don’t have time right now to implement official support for more models. After January 28, we’ll further improve support for other models, including the Wan2.2 series.

[Tinyyuumi]

Hi @Tinyyuumi , thank you for your attention to our work!

UltraViCo should be able to support Wan2.2 I2V. I think the issue is likely related to the code migration. Could you try the following:

• 1. Generate slightly more than 81 frames, e.g., 97 frames — does it still have issues?
• 2. For 161 frames, set alpha = 1 (i.e., no modification) — does it have issues?
• 3. For 161 frames, set alpha = 0.98 (i.e., a very small modification) — does it have issues?

Also:

• For I2V we need to ensure the attention on the conditional image part does not decay. Have you taken this into account?
• Could you provide a video (rather than a single frame as it is now) so we can better observe the issue?

I’m happy to discuss this further with you. Since I’m currently rushing an ICML submission, I don’t have time right now to implement official support for more models. After January 28, we’ll further improve support for other models, including the Wan2.2 series.

Thank you for your reply. After our testing, we confirmed that the issue was related to code migration, and now it is running well. We are using the code provided in the Wan22-Lightning repository and have modified the attention layer directly according to the paper.

Our tests revealed that Wan22 + 4 steps lora + UltraViCo significantly reduces the problem of periodic repetition when generating a 161 frames video. This is truly excellent work.