# Generating model embeddings from Conditional Generation models

**URL:** <https://discuss.huggingface.co/t/generating-model-embeddings-from-conditional-generation-models/56057>\
**Category:** Research\
**Created:** [September 23, 2023, 12:35am UTC](https://discuss.huggingface.co/t/generating-model-embeddings-from-conditional-generation-models/56057 "2023-09-23T00:35:17Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![gchhablani](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/gchhablani/32/19723_2.png) [@gchhablani](https://discuss.huggingface.co/u/gchhablani)\
**Post date:** [September 23, 2023, 12:35am UTC](https://discuss.huggingface.co/t/generating-model-embeddings-from-conditional-generation-models/56057/1 "2023-09-23T00:35:17Z")

</div>

I’m trying to break apart BLIP2 from LAVIS ([https://github.com/salesforce/LAVIS/blob/main/lavis/models/blip2\_models/modeling\_t5.py](https://github.com/salesforce/LAVIS/blob/main/lavis/models/blip2_models/modeling_t5.py)) which uses HuggingFace PreTrained Model to generate sentence embeddings from T5 model.  
So, questions:

- What do I replace this with to get the embeddings: [https://github.com/salesforce/LAVIS/blob/e4040b13d6120062829ee9625f016f3cd3dd16e6/lavis/models/blip2\_models/blip2\_t5.py#L296-L304](https://github.com/salesforce/LAVIS/blob/e4040b13d6120062829ee9625f016f3cd3dd16e6/lavis/models/blip2_models/blip2_t5.py#L296-L304) ?
- What is passed to decoder\_input\_ids in Conditional Generation models during generation when there is no input\_ids but input\_embeds?