# Training PEGASUS on unlabeled data

**URL:** <https://discuss.huggingface.co/t/training-pegasus-on-unlabeled-data/5565>\
**Category:** 🤗Transformers\
**Created:** [April 16, 2021, 8:17pm UTC](https://discuss.huggingface.co/t/training-pegasus-on-unlabeled-data/5565 "2021-04-16T20:17:06Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Skylixia](https://avatars.discourse-cdn.com/v4/letter/s/439d5e/32.png) [@Skylixia](https://discuss.huggingface.co/u/Skylixia)\
**Post date:** [April 16, 2021, 8:17pm UTC](https://discuss.huggingface.co/t/training-pegasus-on-unlabeled-data/5565/1 "2021-04-16T20:17:06Z")

</div>

Hi,

I was wondering if there is any way to load a PEGASUS model and further train it on unlabeled data with its training objective rather than supervised finetuning with labeled data ?
