# Label Smoothing in NLLB gives ValueError: You have to specify either decoder\_input\_ids or decoder\_inputs\_embeds

**URL:** <https://discuss.huggingface.co/t/label-smoothing-in-nllb-gives-valueerror-you-have-to-specify-either-decoder-input-ids-or-decoder-inputs-embeds/25320>\
**Category:** Beginners\
**Created:** [November 1, 2022, 10:27am UTC](https://discuss.huggingface.co/t/label-smoothing-in-nllb-gives-valueerror-you-have-to-specify-either-decoder-input-ids-or-decoder-inputs-embeds/25320 "2022-11-01T10:27:47Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Shamus](https://avatars.discourse-cdn.com/v4/letter/s/858c86/32.png) [@Shamus](https://discuss.huggingface.co/u/Shamus)\
**Post date:** [November 1, 2022, 10:27am UTC](https://discuss.huggingface.co/t/label-smoothing-in-nllb-gives-valueerror-you-have-to-specify-either-decoder-input-ids-or-decoder-inputs-embeds/25320/1 "2022-11-01T10:27:48Z")

</div>

Hi. I am fine-tuning a NLLB model with the Trainer API. When I am am specifying the parameter **label\_smoothing\_factor** to some value in hf.json, I get an error like this. If the error shown is correct, how do I specify decoder\_input\_ids or decoder\_inputs\_embeds?

```auto
  File "run_translation.py", line 849, in <module>
    main()
  File "run_translation.py", line 753, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/root/.clearml/venvs-builds/3.8/task_repository/hf-translation.git/.venv/lib/python3.8/site-packages/transformers/trainer.py", line 1498, in train
    return inner_training_loop(
  File "/root/.clearml/venvs-builds/3.8/task_repository/hf-translation.git/.venv/lib/python3.8/site-packages/transformers/trainer.py", line 1740, in _inner_training_loop
    tr_loss_step = self.training_step(model, inputs)
  File "/root/.clearml/venvs-builds/3.8/task_repository/hf-translation.git/.venv/lib/python3.8/site-packages/transformers/trainer.py", line 2470, in training_step
    loss = self.compute_loss(model, inputs)
  File "/root/.clearml/venvs-builds/3.8/task_repository/hf-translation.git/.venv/lib/python3.8/site-packages/transformers/trainer.py", line 2502, in compute_loss
    outputs = model(**inputs)
  File "/root/.clearml/venvs-builds/3.8/task_repository/hf-translation.git/.venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/.clearml/venvs-builds/3.8/task_repository/hf-translation.git/.venv/lib/python3.8/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1315, in forward
    outputs = self.model(
  File "/root/.clearml/venvs-builds/3.8/task_repository/hf-translation.git/.venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/.clearml/venvs-builds/3.8/task_repository/hf-translation.git/.venv/lib/python3.8/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1206, in forward
    decoder_outputs = self.decoder(
  File "/root/.clearml/venvs-builds/3.8/task_repository/hf-translation.git/.venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/.clearml/venvs-builds/3.8/task_repository/hf-translation.git/.venv/lib/python3.8/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 985, in forward
    raise ValueError("You have to specify either decoder_input_ids or decoder_inputs_embeds")
ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds

```

---

<div class="post-metadata">

**Author:** ![binwang](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/binwang/32/16303_2.png) [@binwang](https://discuss.huggingface.co/u/binwang)\
**Post date:** [May 30, 2023, 5:02am UTC](https://discuss.huggingface.co/t/label-smoothing-in-nllb-gives-valueerror-you-have-to-specify-either-decoder-input-ids-or-decoder-inputs-embeds/25320/2 "2023-05-30T05:02:40Z")

</div>

I met the same issue. Have you resolved it?

---

<div class="post-metadata">

**Author:** ![Bendang](https://avatars.discourse-cdn.com/v4/letter/b/a4c791/32.png) [@Bendang](https://discuss.huggingface.co/u/Bendang)\
**Post date:** [July 22, 2025, 10:31am UTC](https://discuss.huggingface.co/t/label-smoothing-in-nllb-gives-valueerror-you-have-to-specify-either-decoder-input-ids-or-decoder-inputs-embeds/25320/3 "2025-07-22T10:31:26Z")

</div>

I couldnt get it running too. So i did a workaround with this.

```python
class SmoothingSeq2SeqTrainer(Seq2SeqTrainer):
    def __init__ (self, *args, smoothing=0.1, **kwargs):
        super(). __init__ (*args, **kwargs)
        self.smoothing = smoothing

    def compute_loss(self, model, inputs, return_outputs=False, num_items_in_batch=None):
        labels = inputs["labels"]
        outputs = model(**inputs)
        logits = outputs.logits

        # Standard shift for sequence-to-sequence models
        shift_logits = logits[..., :-1, :].contiguous()
        shift_labels = labels[..., 1:].contiguous()

        loss_fct = CrossEntropyLoss(
            label_smoothing=self.smoothing,
            ignore_index=self.processing_class.pad_token_id
        )
        loss = loss_fct(
            shift_logits.view(-1, shift_logits.size(-1)),
            shift_labels.view(-1)
        )
        return (loss, outputs) if return_outputs else loss

trainer = SmoothingSeq2SeqTrainer(
    smoothing=0.1,
    ...
)

```

---

<div class="post-metadata">

**Author:** ![Bendang](https://avatars.discourse-cdn.com/v4/letter/b/a4c791/32.png) [@Bendang](https://discuss.huggingface.co/u/Bendang)\
**Post date:** [July 23, 2025, 2:40pm UTC](https://discuss.huggingface.co/t/label-smoothing-in-nllb-gives-valueerror-you-have-to-specify-either-decoder-input-ids-or-decoder-inputs-embeds/25320/4 "2025-07-23T14:40:45Z")

</div>

I believe I found the root cause of the issue: the NLLB model doesn’t automatically generate `decoder_input_ids` during data collation. Without them, training won’t work as expected with `label_smoothing_factor `.

To fix this, I created a custom data collator that manually generates `decoder_input_ids` by shifting the `labels` to the right — as is standard in seq2seq training. Here’s how you can implement it:

```python
from transformers.data.data_collator import DataCollatorForSeq2Seq
class ManualShiftDataCollator(DataCollatorForSeq2Seq):
    def __call__ (self, features, return_tensors=None):
        batch = super(). __call__ (features, return_tensors)

        if "labels" in batch:
            labels = batch["labels"]
            
            decoder_start_token_id = self.model.config.decoder_start_token_id
            decoder_input_ids = labels.clone()
            
            # Shift the entire tensor to the right.
            decoder_input_ids[:, 1:] = labels[:, :-1]
            
            # Set the first token of each sequence to the decoder_start_token_id.
            decoder_input_ids[:, 0] = decoder_start_token_id
            
            # Replace any -100 that got shifted with the actual padding token ID.
            decoder_input_ids.masked_fill_(decoder_input_ids == -100, self.tokenizer.pad_token_id)

            batch["decoder_input_ids"] = decoder_input_ids
        return batch

# then replace your collator with this
data_collator = ManualShiftDataCollator(
    tokenizer=tokenizer,
    model=model,
    label_pad_token_id=tokenizer.pad_token_id
    pad_to_multiple_of=8,
    return_tensors="pt"
)

```

Using this collator fixed the root problem for me — the model now trains properly and shows good convergence.

**Note** : Please disregard the custom label-smoothing loss I mentioned above (`SmoothingSeq2SeqTrainer`). In my experience, it caused the model to converge very slowly and plateau at a high loss. The proper fix is to use this custom data collator instead.

Hope this helps