Does LMS Lora have strict requirements for the size of the input video?

#9
by xuantianliliang - opened

Does LMS Lora have strict requirements for the size of the input video? When entering a 528 * 768 video, an error occurs: ERROR lora diffusion_model.blocks.49.adaln_proj.linear.weight shape '[96768, 2688]' is invalid for input of size 774144

Does LMS Lora have strict requirements for the size of the input video? When entering a 528 * 768 video, an error occurs: ERROR lora diffusion_model.blocks.49.adaln_proj.linear.weight shape '[96768, 2688]' is invalid for input of size 774144

There's no problem with video sizes; this issue seems to have more to do with the base model you're using. The LMS was trained on the ref2va pruned base, and that's probably why you're having this problem. Are you using the pruned base?

Does LMS Lora have strict requirements for the size of the input video? When entering a 528 * 768 video, an error occurs: ERROR lora diffusion_model.blocks.49.adaln_proj.linear.weight shape '[96768, 2688]' is invalid for input of size 774144

There's no problem with video sizes; this issue seems to have more to do with the base model you're using. The LMS was trained on the ref2va pruned base, and that's probably why you're having this problem. Are you using the pruned base?

image
I am using minimax_h3_ref2va_bf16.safetensors, not the pruning version

Does LMS Lora have strict requirements for the size of the input video? When entering a 528 * 768 video, an error occurs: ERROR lora diffusion_model.blocks.49.adaln_proj.linear.weight shape '[96768, 2688]' is invalid for input of size 774144

There's no problem with video sizes; this issue seems to have more to do with the base model you're using. The LMS was trained on the ref2va pruned base, and that's probably why you're having this problem. Are you using the pruned base?

image
I am using minimax_h3_ref2va_bf16.safetensors, not the pruning version

That's why you're having these layer mismatch error problems.

Does LMS Lora have strict requirements for the size of the input video? When entering a 528 * 768 video, an error occurs: ERROR lora diffusion_model.blocks.49.adaln_proj.linear.weight shape '[96768, 2688]' is invalid for input of size 774144

There's no problem with video sizes; this issue seems to have more to do with the base model you're using. The LMS was trained on the ref2va pruned base, and that's probably why you're having this problem. Are you using the pruned base?

image
I am using minimax_h3_ref2va_bf16.safetensors, not the pruning version

That's why you're having these layer mismatch error problems.

Thanks!Do you have a plan to train a Lora based on the minimax_h3_ref2va_bf16. safetensors model?

Thanks!Do you have a plan to train a Lora based on the minimax_h3_ref2va_bf16. safetensors model?

No, because most people will use the pruned version and not BF16 no pruned version, so it doesn't make sense for me to train on that if almost nobody uses it.

Thanks!Do you have a plan to train a Lora based on the minimax_h3_ref2va_bf16. safetensors model?

No, because most people will use the pruned version and not BF16 no pruned version, so it doesn't make sense for me to train on that if almost nobody uses it.

ok, thanks for you reply

xuantianliliang changed discussion status to closed

Sign up or log in to comment