# Using RAG with local documents

**URL:** <https://discuss.huggingface.co/t/using-rag-with-local-documents/5326>\
**Category:** Models\
**Created:** [April 6, 2021, 12:02pm UTC](https://discuss.huggingface.co/t/using-rag-with-local-documents/5326 "2021-04-06T12:02:26Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![saichandra](https://avatars.discourse-cdn.com/v4/letter/s/f07891/32.png) [@saichandra](https://discuss.huggingface.co/u/saichandra)\
**Post date:** [April 6, 2021, 12:02pm UTC](https://discuss.huggingface.co/t/using-rag-with-local-documents/5326/1 "2021-04-06T12:02:26Z")

</div>

Hi,  
I have a requirement that model should search for relevant documents to answer the query and I found [RAG](https://arxiv.org/pdf/2005.11401.pdf) from Facebook AI which perfectly fits my usecase. I also found [this](https://huggingface.co/blog/ray-rag) post in which HuggingFace explains RAG and came to know that HF implemented RAG which is awesome!  
My doubt is whether I could extend this functionality so that the model should do retrieval from local documents rather than from HF’s wikipedia corpus.  
Are there any notebooks to refer to?

---

<div class="post-metadata">

**Author:** ![lewtun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lewtun/32/30995_2.png) [@lewtun](https://discuss.huggingface.co/u/lewtun)\
**Post date:** [April 6, 2021, 9:19pm UTC](https://discuss.huggingface.co/t/using-rag-with-local-documents/5326/2 "2021-04-06T21:19:23Z")

</div>

hey @saichandra, one possibility would be to use [Haystack’s](https://haystack.deepset.ai/) implementation of RAG (which is based on HF transformers), e.g. see here: [https://haystack.deepset.ai/docs/latest/tutorial7md](https://haystack.deepset.ai/docs/latest/tutorial7md)

one advantage of using Haystack is that they provide a nice API for FAISS (and other document stores) so you can store the embeddings locally with just a few lines of code.

i’ve had mixed results from using RAG with the Natural Questions checkpoints (the answers are often gibberish). if you’re doing QA on a specialised corpus, you might be better off using the classic Retriever-Reader architecture or fine-tuning RAG on your domain

---

<div class="post-metadata">

**Author:** ![shamanez](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shamanez/32/17994_2.png) [@shamanez](https://discuss.huggingface.co/u/shamanez)\
**Post date:** [April 21, 2021, 1:00pm UTC](https://discuss.huggingface.co/t/using-rag-with-local-documents/5326/3 "2021-04-21T13:00:39Z")

</div>

You can easily do this by encoding your local KL. Please follow the [use\_own\_knowledge\_dataset.py](https://github.com/huggingface/transformers/blob/master/examples/research_projects/rag/use_own_knowledge_dataset.py)

---

<div class="post-metadata">

**Author:** ![lewtun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lewtun/32/30995_2.png) [@lewtun](https://discuss.huggingface.co/u/lewtun)\
**Post date:** [April 21, 2021, 1:50pm UTC](https://discuss.huggingface.co/t/using-rag-with-local-documents/5326/4 "2021-04-21T13:50:56Z")

</div>

to add to what @shamanez said, i also recently learnt that you can add faiss indices directly in the `datasets` library: [Adding a FAISS or Elastic Search index to a Dataset — datasets 1.6.0 documentation](https://huggingface.co/docs/datasets/faiss_and_ea.html?highlight=faiss#adding-a-faiss-or-elastic-search-index-to-a-dataset)