# Is there a way to backpropagate through multiple steps while using Trainer API

**URL:** <https://discuss.huggingface.co/t/is-there-a-way-to-backpropagate-through-multiple-steps-while-using-trainer-api/8215>\
**Category:** 🤗Transformers\
**Created:** [July 9, 2021, 2:53pm UTC](https://discuss.huggingface.co/t/is-there-a-way-to-backpropagate-through-multiple-steps-while-using-trainer-api/8215 "2021-07-09T14:53:22Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![rlian](https://avatars.discourse-cdn.com/v4/letter/r/a88e57/32.png) [@rlian](https://discuss.huggingface.co/u/rlian)\
**Post date:** [July 9, 2021, 2:53pm UTC](https://discuss.huggingface.co/t/is-there-a-way-to-backpropagate-through-multiple-steps-while-using-trainer-api/8215/1 "2021-07-09T14:53:22Z")

</div>

I was wondering if there’s a way to backpropagate the accumulation of loss through multiple optimizer steps while using Trainer API? Since it’s easy to get cuda out of memory and would like to avoid it.

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [July 9, 2021, 3:06pm UTC](https://discuss.huggingface.co/t/is-there-a-way-to-backpropagate-through-multiple-steps-while-using-trainer-api/8215/2 "2021-07-09T15:06:11Z")

</div>

If you are talking about gradient accumulation, you can set it with `gradient_accumulation_steps=xxx` in your `TrainingArguments`.