# Question about training\_step Function in Class Trainer

**URL:** <https://discuss.huggingface.co/t/question-about-training-step-function-in-class-trainer/108140>\
**Category:** 🤗Transformers\
**Created:** [September 23, 2024, 1:52pm UTC](https://discuss.huggingface.co/t/question-about-training-step-function-in-class-trainer/108140 "2024-09-23T13:52:46Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![satwoman](https://avatars.discourse-cdn.com/v4/letter/s/a8b319/32.png) [@satwoman](https://discuss.huggingface.co/u/satwoman)\
**Post date:** [September 23, 2024, 1:52pm UTC](https://discuss.huggingface.co/t/question-about-training-step-function-in-class-trainer/108140/1 "2024-09-23T13:52:46Z")

</div>

I’m analyzing the Class Trainer source code and have a question regarding the training\_step function. In this function, it appears that the backward pass is performed before the function returns the loss. Specifically, after calculating the loss, the code calls scaled\_loss.backward() (or self.accelerator.backward(loss)) before returning loss.detach() / self.args.gradient\_accumulation\_steps.

Does this mean that the function first performs backpropagation on the computed loss and then divides the loss by self.args.gradient\_accumulation\_steps before returning it? How does this fit into the gradient accumulation strategy?

Thanks for your help!  
[Link to the source code of the training\_step function.](https://github.com/huggingface/transformers/blob/v4.44.2/src/transformers/trainer.py#L3293)