# Bigger batch size, the lower throughput and GPU usage？

**URL:** <https://discuss.huggingface.co/t/bigger-batch-size-the-lower-throughput-and-gpu-usage/12449>\
**Category:** 🤗Transformers\
**Created:** [December 2, 2021, 9:45am UTC](https://discuss.huggingface.co/t/bigger-batch-size-the-lower-throughput-and-gpu-usage/12449 "2021-12-02T09:45:34Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![pipi](https://avatars.discourse-cdn.com/v4/letter/p/e9bcb4/32.png) [@pipi](https://discuss.huggingface.co/u/pipi)\
**Post date:** [December 2, 2021, 9:45am UTC](https://discuss.huggingface.co/t/bigger-batch-size-the-lower-throughput-and-gpu-usage/12449/1 "2021-12-02T09:45:34Z")

</div>

I’m training **bert-base** on single node with 8xA100 GPU, using **run\_mlm.py** script.

when the batch size set to **256** , the throughput is 8000 sample/s, and GPU usage is 80%.  
when the batch size set to **384** , the throughput is 4000 sample/s, and GPU usage is 50%.

What is the reason for this phenomenon？Data IO became bottleneck?

---

<div class="post-metadata">

**Author:** ![andrewvk](https://avatars.discourse-cdn.com/v4/letter/a/7ba0ec/32.png) [@andrewvk](https://discuss.huggingface.co/u/andrewvk)\
**Post date:** [July 16, 2022, 12:10am UTC](https://discuss.huggingface.co/t/bigger-batch-size-the-lower-throughput-and-gpu-usage/12449/2 "2022-07-16T00:10:03Z")

</div>

Hi, I’m having a similar issue with ViT but I am getting very inconsistent and low GPU usage. Did you figure this out? I think it is IO as well