# Issues saving and loading wav2vec2 models fine tuned using Deepspeed **URL:** \ **Category:** DeepSpeed\ **Created:** [October 22, 2021, 10:43pm UTC](https://discuss.huggingface.co/t/issues-saving-and-loading-wav2vec2-models-fine-tuned-using-deepspeed/11007 "2021-10-22T22:43:10Z")\ **Posts on this page:** 2\ **Page:** 1

**Author:** ![alexerdmann](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/alexerdmann/32/2872_2.png) [@alexerdmann](https://discuss.huggingface.co/u/alexerdmann)\ **Post date:** [October 22, 2021, 10:43pm UTC](https://discuss.huggingface.co/t/issues-saving-and-loading-wav2vec2-models-fine-tuned-using-deepspeed/11007/1 "2021-10-22T22:43:10Z")

Hi, First off, thanks for all your hard work and publicly released code! I’ve implemented the Deepspeed integration for fine tuning a wav2vec2 model [here](https://github.com/huggingface/transformers/tree/master/examples/research_projects/wav2vec2). I had to make some small changes for it to run in my environment, but essentially, my code is identical to the example. After training some toy models, I realized that I couldn’t load from the checkpoints or save and reload the model in the same way that other wav2vec2 fine tuned models can be saved and loaded via the \*.from\_pretrained() commands. To reproduce my struggles, simply add the following at the end of the run\_asr.py script in the example repo I mentioned above and run with 2 GPUs, using the repo-provided Zero3 config and relevant command: ``` tuned_model_dir = './my_tuned_model' print('--> Saving fine tuned model with processor in', tuned_model_dir) model.save_pretrained(tuned_model_dir) processor.save_pretrained(tuned_model_dir) print('--> Checking if model and processor were saved properly..') model = Wav2Vec2ForCTC.from_pretrained(tuned_model_dir).to('cuda') print('--> Worker {} Got Model!'.format(training_args.local_rank)) processor = Wav2Vec2Processor.from_pretrained(tuned_model_dir) print('--> Worker {} Got Processor!'.format(training_args.local_rank)) ``` I’m including the log below (starting after the training finishes successfully). Interestingly, the print statements reveal in the log that one of the workers seems to load the model successfully but the other does not. How can this be? I’ve also tried several other methods of saving and loading the fine tuned model that all failed. These include saving the model at the end of training, then attempting to load the saved model offline (which again complains about a size mismatch saying the model layers are all curiously of size [1]); as well as all the more complicated save and load procedures outlined [here](https://huggingface.co/transformers/main_classes/deepspeed.html?highlight=how%20save%20model%20trained%20deepspeed) and [here](https://deepspeed.readthedocs.io/en/latest/model-checkpointing.html) (which mostly also all fail for the same reason). Any help would be greatly appreciated! The model trains, I’m most of the way there… I just can’t use it haha. Thanks in advance! Log: … Training completed. Do not forget to share your model on [huggingface.co/models](http://huggingface.co/models) =) → Saving fine tuned model with processor in ./my\_tuned\_model {‘train\_runtime’: 97.5809, ‘train\_samples\_per\_second’: 5.124, ‘train\_steps\_per\_second’: 1.281, ‘train\_loss’: 304.0380625, ‘epoch’: 1.0} 100%|██████████| 125/125 [01:37\<00:00, 1.28it/s] → Saving fine tuned model with processor in ./my\_tuned\_model Configuration saved in ./my\_tuned\_model/config.json Model weights saved in ./my\_tuned\_model/pytorch\_model.bin Configuration saved in ./my\_tuned\_model/preprocessor\_config.json tokenizer config file saved in ./my\_tuned\_model/tokenizer\_config.json Special tokens file saved in ./my\_tuned\_model/special\_tokens\_map.json → Checking if model and processor were saved properly… loading configuration file ./my\_tuned\_model/config.json Model config Wav2Vec2Config { “\_name\_or\_path”: “./”, “activation\_dropout”: 0.0, “apply\_spec\_augment”: true, “architectures”: [ “Wav2Vec2ForCTC” ], “attention\_dropout”: 0.1, “bos\_token\_id”: 1, “classifier\_proj\_size”: 256, “codevector\_dim”: 256, “contrastive\_logits\_temperature”: 0.1, “conv\_bias”: false, “conv\_dim”: [ 512, 512, 512, 512, 512, 512, 512 ], “conv\_kernel”: [ 10, 3, 3, 3, 3, 2, 2 ], “conv\_stride”: [ 5, 2, 2, 2, 2, 2, 2 ], “ctc\_loss\_reduction”: “sum”, “ctc\_zero\_infinity”: false, “diversity\_loss\_weight”: 0.1, “do\_stable\_layer\_norm”: false, “eos\_token\_id”: 2, “feat\_extract\_activation”: “gelu”, “feat\_extract\_norm”: “group”, “feat\_proj\_dropout”: 0.1, “feat\_quantizer\_dropout”: 0.0, “final\_dropout”: 0.0, “freeze\_feat\_extract\_train”: true, “gradient\_checkpointing”: false, “hidden\_act”: “gelu”, “hidden\_dropout”: 0.1, “hidden\_size”: 768, “initializer\_range”: 0.02, “intermediate\_size”: 3072, “layer\_norm\_eps”: 1e-05, “layerdrop”: 0.05, “mask\_channel\_length”: 10, “mask\_channel\_min\_space”: 1, “mask\_channel\_other”: 0.0, “mask\_channel\_prob”: 0.0, “mask\_channel\_selection”: “static”, “mask\_feature\_length”: 10, “mask\_feature\_prob”: 0.0, “mask\_time\_length”: 10, “mask\_time\_min\_space”: 1, “mask\_time\_other”: 0.0, “mask\_time\_prob”: 0.05, “mask\_time\_selection”: “static”, “model\_type”: “wav2vec2”, “no\_mask\_channel\_overlap”: false, “no\_mask\_time\_overlap”: false, “num\_attention\_heads”: 12, “num\_codevector\_groups”: 2, “num\_codevectors\_per\_group”: 320, “num\_conv\_pos\_embedding\_groups”: 16, “num\_conv\_pos\_embeddings”: 128, “num\_feat\_extract\_layers”: 7, “num\_hidden\_layers”: 12, “num\_negatives”: 100, “pad\_token\_id”: 0, “proj\_codevector\_dim”: 256, “torch\_dtype”: “float32”, “transformers\_version”: “4.12.0.dev0”, “use\_weighted\_layer\_sum”: false, “vocab\_size”: 32 } loading weights file ./my\_tuned\_model/pytorch\_model.bin → Checking if model and processor were saved properly… Detected DeepSpeed ZeRO-3: activating zero.init() for this model Traceback (most recent call last): File “run\_asr.py”, line 576, in main() File “run\_asr.py”, line 566, in main model = Wav2Vec2ForCTC.from\_pretrained(tuned\_model\_dir).to(‘cuda’) File “/home/users/aerdmann/repos/others/transformers/src/transformers/modeling\_utils.py”, line 1442, in from\_pretrained \n Worker 1 Got Model! ``` model, missing_keys, unexpected_keys, mismatched_keys, error_msgs = cls._load_state_dict_into_model( ``` File “/home/users/aerdmann/repos/others/transformers/src/transformers/modeling\_utils.py”, line 1594, in \_load\_state\_dict\_into\_model raise RuntimeError(f"Error(s) in loading state\_dict for {model. **class**. **name** }:\n\t{error\_msg}") RuntimeError: Error(s) in loading state\_dict for Wav2Vec2ForCTC: size mismatch for wav2vec2.feature\_extractor.conv\_layers.1.conv.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([512, 512, 3]). size mismatch for wav2vec2.feature\_extractor.conv\_layers.2.conv.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([512, 512, 3]). size mismatch for wav2vec2.feature\_extractor.conv\_layers.3.conv.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([512, 512, 3]). size mismatch for wav2vec2.feature\_extractor.conv\_layers.4.conv.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([512, 512, 3]). size mismatch for wav2vec2.feature\_extractor.conv\_layers.5.conv.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([512, 512, 2]). size mismatch for wav2vec2.feature\_extractor.conv\_layers.6.conv.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([512, 512, 2]). size mismatch for wav2vec2.feature\_projection.projection.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 512]). size mismatch for wav2vec2.encoder.pos\_conv\_embed.conv.weight\_v: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 48, 128]). size mismatch for wav2vec2.encoder.layers.0.attention.k\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.0.attention.v\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.0.attention.q\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.0.attention.out\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.0.feed\_forward.intermediate\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([3072, 768]). size mismatch for wav2vec2.encoder.layers.0.feed\_forward.output\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 3072]). size mismatch for wav2vec2.encoder.layers.1.attention.k\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.1.attention.v\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.1.attention.q\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.1.attention.out\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.1.feed\_forward.intermediate\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([3072, 768]). size mismatch for wav2vec2.encoder.layers.1.feed\_forward.output\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 3072]). size mismatch for wav2vec2.encoder.layers.2.attention.k\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.2.attention.v\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.2.attention.q\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.2.attention.out\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.2.feed\_forward.intermediate\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([3072, 768]). size mismatch for wav2vec2.encoder.layers.2.feed\_forward.output\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 3072]). size mismatch for wav2vec2.encoder.layers.3.attention.k\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.3.attention.v\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.3.attention.q\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.3.attention.out\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.3.feed\_forward.intermediate\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([3072, 768]). size mismatch for wav2vec2.encoder.layers.3.feed\_forward.output\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 3072]). size mismatch for wav2vec2.encoder.layers.4.attention.k\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.4.attention.v\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.4.attention.q\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.4.attention.out\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.4.feed\_forward.intermediate\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([3072, 768]). size mismatch for wav2vec2.encoder.layers.4.feed\_forward.output\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 3072]). size mismatch for wav2vec2.encoder.layers.5.attention.k\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.5.attention.v\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.5.attention.q\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.5.attention.out\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.5.feed\_forward.intermediate\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([3072, 768]). size mismatch for wav2vec2.encoder.layers.5.feed\_forward.output\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 3072]). size mismatch for wav2vec2.encoder.layers.6.attention.k\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.6.attention.v\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.6.attention.q\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.6.attention.out\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.6.feed\_forward.intermediate\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([3072, 768]). size mismatch for wav2vec2.encoder.layers.6.feed\_forward.output\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 3072]). size mismatch for wav2vec2.encoder.layers.7.attention.k\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.7.attention.v\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.7.attention.q\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.7.attention.out\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.7.feed\_forward.intermediate\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([3072, 768]). size mismatch for wav2vec2.encoder.layers.7.feed\_forward.output\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 3072]). size mismatch for wav2vec2.encoder.layers.8.attention.k\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.8.attention.v\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.8.attention.q\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.8.attention.out\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.8.feed\_forward.intermediate\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([3072, 768]). size mismatch for wav2vec2.encoder.layers.8.feed\_forward.output\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 3072]). size mismatch for wav2vec2.encoder.layers.9.attention.k\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.9.attention.v\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.9.attention.q\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.9.attention.out\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.9.feed\_forward.intermediate\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([3072, 768]). size mismatch for wav2vec2.encoder.layers.9.feed\_forward.output\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 3072]). size mismatch for wav2vec2.encoder.layers.10.attention.k\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.10.attention.v\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.10.attention.q\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.10.attention.out\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.10.feed\_forward.intermediate\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([3072, 768]). size mismatch for wav2vec2.encoder.layers.10.feed\_forward.output\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 3072]). size mismatch for wav2vec2.encoder.layers.11.attention.k\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.11.attention.v\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.11.attention.q\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.11.attention.out\_proj.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 768]). size mismatch for wav2vec2.encoder.layers.11.feed\_forward.intermediate\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([3072, 768]). size mismatch for wav2vec2.encoder.layers.11.feed\_forward.output\_dense.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([768, 3072]). size mismatch for lm\_head.weight: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([32, 768]). \n Worker 1 Got Processor! Killing subprocess 2105751 Killing subprocess 2105752 Traceback (most recent call last): File “/home/software/kdd/Anaconda3.8-2021.05/lib/python3.8/runpy.py”, line 194, in \_run\_module\_as\_main return \_run\_code(code, main\_globals, None, File “/home/software/kdd/Anaconda3.8-2021.05/lib/python3.8/runpy.py”, line 87, in \_run\_code exec(code, run\_globals) File “/home/software/kdd/Anaconda3.8-2021.05/lib/python3.8/site-packages/deepspeed/launcher/launch.py”, line 171, in main() File “/home/software/kdd/Anaconda3.8-2021.05/lib/python3.8/site-packages/deepspeed/launcher/launch.py”, line 161, in main sigkill\_handler(signal.SIGTERM, None) # not coming back File “/home/software/kdd/Anaconda3.8-2021.05/lib/python3.8/site-packages/deepspeed/launcher/launch.py”, line 139, in sigkill\_handler raise subprocess.CalledProcessError(returncode=last\_return\_code, cmd=cmd) subprocess.CalledProcessError: Command ‘[’/home/software/kdd/Anaconda3.8-2021.05/bin/python’, ‘-u’, ‘run\_asr.py’, ‘–local\_rank=1’, ‘–output\_dir=./wav2vec2-base-timit-asr’, ‘–num\_train\_epochs=1’, ‘–per\_device\_train\_batch\_size=2’, ‘–per\_device\_eval\_batch\_size=2’, ‘–evaluation\_strategy=steps’, ‘–save\_steps=500’, ‘–eval\_steps=100’, ‘–logging\_steps=25’, ‘–learning\_rate=5e-4’, ‘–warmup\_steps=3000’, ‘–model\_name\_or\_path=facebook/wav2vec2-base’, ‘–dataset\_name=timit\_asr’, ‘–dataset\_config\_name=clean’, ‘–train\_split\_name=train’, ‘–validation\_split\_name=test’, ‘–orthography=timit’, ‘–preprocessing\_num\_workers=1’, ‘–group\_by\_length’, ‘–freeze\_feature\_extractor’, ‘–deepspeed’, ‘ds\_config\_wav2vec2\_zero3.json’]’ returned non-zero exit status 1. ---

**Author:** ![hughbzhang](https://avatars.discourse-cdn.com/v4/letter/h/bcef8e/32.png) [@hughbzhang](https://discuss.huggingface.co/u/hughbzhang)\ **Post date:** [March 3, 2023, 6:30pm UTC](https://discuss.huggingface.co/t/issues-saving-and-loading-wav2vec2-models-fine-tuned-using-deepspeed/11007/2 "2023-03-03T18:30:01Z")

By any chance, did you ever figure out the solution to this issue?