# Deepspeed zero3 does not work with Diffusion Models. Does anyone know how to fix this?

**URL:** <https://discuss.huggingface.co/t/deepspeed-zero3-does-not-work-with-diffusion-models-does-anyone-know-how-to-fix-this/36293>\
**Category:** DeepSpeed\
**Created:** [April 12, 2023, 3:19am UTC](https://discuss.huggingface.co/t/deepspeed-zero3-does-not-work-with-diffusion-models-does-anyone-know-how-to-fix-this/36293 "2023-04-12T03:19:55Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![uygnef](https://avatars.discourse-cdn.com/v4/letter/u/49beb7/32.png) [@uygnef](https://discuss.huggingface.co/u/uygnef)\
**Post date:** [April 12, 2023, 3:19am UTC](https://discuss.huggingface.co/t/deepspeed-zero3-does-not-work-with-diffusion-models-does-anyone-know-how-to-fix-this/36293/1 "2023-04-12T03:19:55Z")

</div>

here is detail: [using Deepspeed zero stage3 finetune sd2, dimension error occurs · Issue #22705 · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/issues/22705)  
It appears that there may be a problem with partitioning the parameters.

train(args)  
File “/workdir/fengyu05/501587/033c0dc6fc985d5dee8904327c53b497/src/diffuser/train\_txt2img.py”, line 529, in train  
encoder\_hidden\_states = text\_encoder(batch[“input\_ids”].to(accelerator.device))[0]  
File “/usr/local/conda/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1194, in \_call\_impl  
return forward\_call(\*input, \*\*kwargs)  
File “/usr/local/conda/lib/python3.9/site-packages/transformers/models/clip/modeling\_clip.py”, line 816, in forward  
return self.text\_model(  
File “/usr/local/conda/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1194, in \_call\_impl  
return forward\_call(\*input, \*\*kwargs)  
File “/usr/local/conda/lib/python3.9/site-packages/transformers/models/clip/modeling\_clip.py”, line 712, in forward  
hidden\_states = self.embeddings(input\_ids=input\_ids, position\_ids=position\_ids)  
File “/usr/local/conda/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1194, in \_call\_impl  
return forward\_call(\*input, \*\*kwargs)  
File “/usr/local/conda/lib/python3.9/site-packages/transformers/models/clip/modeling\_clip.py”, line 227, in forward  
inputs\_embeds = self.token\_embedding(input\_ids)  
File “/usr/local/conda/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1194, in \_call\_impl  
return forward\_call(\*input, \*\*kwargs)  
File “/usr/local/conda/lib/python3.9/site-packages/torch/nn/modules/sparse.py”, line 160, in forward  
return F.embedding(  
File “/usr/local/conda/lib/python3.9/site-packages/torch/nn/functional.py”, line 2210, in embedding  
return torch.embedding(weight, input, padding\_idx, scale\_grad\_by\_freq, sparse)  
RuntimeError: ‘weight’ must be 2-D  
Steps: 0%|

---

<div class="post-metadata">

**Author:** ![zhanwenchen](https://avatars.discourse-cdn.com/v4/letter/z/a6a055/32.png) [@zhanwenchen](https://discuss.huggingface.co/u/zhanwenchen)\
**Post date:** [April 18, 2025, 6:22pm UTC](https://discuss.huggingface.co/t/deepspeed-zero3-does-not-work-with-diffusion-models-does-anyone-know-how-to-fix-this/36293/2 "2025-04-18T18:22:34Z")

</div>

You may need to prepare all your model, trainer, dataloaders, optimizers, etc. If you do them separately, there may be an issue with proper partitioning.