v16

README.md

@@ -6,17 +6,20 @@ language:
 
 ***<p style="font-size: 24px">Feel free to try out our [OpenChatKit feedback app](https://huggingface.co/spaces/togethercomputer/OpenChatKit)!</p>***
 
-# GPT-NeoXT-Chat-Base-20B
+# GPT-NeoXT-Chat-Base-20B-v0.16
 
 > TLDR: As part of OpenChatKit (codebase available [here](https://github.com/togethercomputer/OpenChaT)),
-> GPT-NeoXT-Chat-Base-20B is a 20B parameter language model, fine-tuned from EleutherAI’s GPT-NeoX with over 40 million instructions on 100% carbon negative compute.
+> GPT-NeoXT-Chat-Base-20B-v0.16 is a 20B parameter language model, fine-tuned from EleutherAI’s GPT-NeoX with over 40 million instructions on 100% carbon negative compute.
 
-GPT-NeoXT-Chat-Base-20B is based on ElutherAI’s GPT-NeoX model, and is fine-tuned with data focusing on dialog-style interactions. 
+GPT-NeoXT-Chat-Base-20B-v0.16 is based on ElutherAI’s GPT-NeoX model, and is fine-tuned with data focusing on dialog-style interactions. 
 We focused the tuning on several tasks such as question answering, classification, extraction, and summarization. 
-We’ve fine-tuned the model with a collection of 43 million high-quality instructions. 
+We’ve fine-tuned the model with a collection of 43 million high-quality instructions.
 Together partnered with LAION and Ontocord.ai, who both helped curate the dataset the model is based on.
 You can read more about this process and the availability of this dataset in LAION’s blog post [here](https://laion.ai/blog/oig-dataset/). 
 
+In addition to the aforementioned fine-tuning, GPT-NeoXT-Chat-Base-20B-v0.16 has also undergone further fine-tuning via a small amount of feedback data. 
+This allows the model to better adapt to human preferences in the conversations.
+
 ## Model Details
 - **Developed by**: Together Computer.
 - **Model type**: Language Model
@@ -27,18 +30,53 @@ You can read more about this process and the availability of this dataset in LAI
 
 # Quick Start
 
+## GPU Inference
+
+This requires a GPU with 48GB memory.
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+# init
+tokenizer = AutoTokenizer.from_pretrained("togethercomputer/GPT-NeoXT-Chat-Base-20B")
+model = AutoModelForCausalLM.from_pretrained("togethercomputer/GPT-NeoXT-Chat-Base-20B", torch_dtype=torch.float16)
+model = model.to('cuda:0')
+# infer
+inputs = tokenizer("<human>: Hello!\n<bot>:", return_tensors='pt').to(model.device)
+outputs = model.generate(**inputs, max_new_tokens=10, do_sample=True, temperature=0.8)
+output_str = tokenizer.decode(outputs[0])
+print(output_str)
+```
+
+## GPU Inference in Int8
+
+This requires a GPU with 24GB memory.
+
 ```python
-from transformers import pipeline
-pipe = pipeline(model='togethercomputer/GPT-NeoXT-Chat-Base-20B')
-pipe('''<human>: Hello!\n<bot>:''')
+from transformers import AutoTokenizer, AutoModelForCausalLM
+# init
+tokenizer = AutoTokenizer.from_pretrained("togethercomputer/GPT-NeoXT-Chat-Base-20B")
+model = AutoModelForCausalLM.from_pretrained("togethercomputer/GPT-NeoXT-Chat-Base-20B", device_map="auto", load_in_8bit=True)
+# infer
+inputs = tokenizer("<human>: Hello!\n<bot>:", return_tensors='pt').to(model.device)
+outputs = model.generate(**inputs, max_new_tokens=10, do_sample=True, temperature=0.8)
+output_str = tokenizer.decode(outputs[0])
+print(output_str)
 ```
-or
+
+## CPU Inference
+
 ```python
 from transformers import AutoTokenizer, AutoModelForCausalLM
+# init
 tokenizer = AutoTokenizer.from_pretrained("togethercomputer/GPT-NeoXT-Chat-Base-20B")
-model = AutoModelForCausalLM.from_pretrained("togethercomputer/GPT-NeoXT-Chat-Base-20B")
+model = AutoModelForCausalLM.from_pretrained("togethercomputer/GPT-NeoXT-Chat-Base-20B", torch_dtype=torch.bfloat16)
+# infer
+inputs = tokenizer("<human>: Hello!\n<bot>:", return_tensors='pt').to(model.device)
+outputs = model.generate(**inputs, max_new_tokens=10, do_sample=True, temperature=0.8)
+output_str = tokenizer.decode(outputs[0])
+print(output_str)
 ```
 
+
 ## Strengths of the model
 
 There are several tasks that OpenChatKit excels at out of the box. This includes: 
@@ -140,19 +178,19 @@ Excluded uses are described below.
 
 ### Misuse, Malicious Use, and Out-of-Scope Use
 
-The OpenChatKit community provides GPT-NeoXT-Chat-Base-20B as an open source tool for building chatbots. 
+The OpenChatKit community provides GPT-NeoXT-Chat-Base-20B-v0.16 as an open source tool for building chatbots. 
 The community is not responsible for any misuse, malicious use, or out-of-scope use of the model. 
 It is the responsibility of the end user to ensure that the model is used in a responsible and ethical manner.
 
 #### Out-of-Scope Use
 
-GPT-NeoXT-Chat-Base-20B is designed for use in chatbot applications and may not perform well for other use cases outside of its intended scope. 
+GPT-NeoXT-Chat-Base-20B-v0.16 is designed for use in chatbot applications and may not perform well for other use cases outside of its intended scope. 
 For example, it may not be suitable for use in safety-critical applications or for making decisions that have a significant impact on individuals or society. 
 It is important to consider the limitations of the model and to only use it for its intended purpose.
 
 #### Misuse and Malicious Use
 
-GPT-NeoXT-Chat-Base-20B is designed for use in chatbot applications and should not be used for any other purpose.
+GPT-NeoXT-Chat-Base-20B-v0.16 is designed for use in chatbot applications and should not be used for any other purpose.
 Misuse of the model, such as using it to engage in illegal or unethical activities, is strictly prohibited and goes against the principles of the OpenChatKit community project.
 
 Using the model to generate content that is cruel to individuals is a misuse of this model. This includes, but is not limited to:
@@ -169,7 +207,7 @@ Using the model to generate content that is cruel to individuals is a misuse of
 
 ## Limitations
 
-GPT-NeoXT-Chat-Base-20B, like other language model-based chatbots, has limitations that should be taken into consideration. 
+GPT-NeoXT-Chat-Base-20B-v0.16, like other language model-based chatbots, has limitations that should be taken into consideration. 
 For example, the model may not always provide accurate or relevant answers, particularly for questions that are complex, ambiguous, or outside of its training data. 
 We therefore welcome contributions from individuals and organizations, and encourage collaboration towards creating a more robust and inclusive chatbot.
 
@@ -189,4 +227,4 @@ Please refer to [togethercomputer/OpenDataHub](https://github.com/togethercomput
 
 ## Community
 
-Join us on [Together Discord](https://discord.gg/6ZVDU8tTD4)
+Join us on [Together Discord](https://discord.gg/6ZVDU8tTD4)

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "togethercomputer/OpenChaT",
+  "_name_or_path": "togethercomputer/GPT-NeoXT-Chat-Base-20B",
   "architectures": [
     "GPTNeoXForCausalLM"
   ],

pytorch_model-00001-of-00005.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3b56b667fea48813a8b8b9d7d860b5d3860e23b08888908179c30948a406e78b
+oid sha256:1cfd3a71c56d95e80f3d964e8be957f71bea4f1073788ac56d28a7815294ff5e
 size 9953774091

pytorch_model-00002-of-00005.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ebad6672b8d33a16a1d55a1d8ba926066c73ea6b14a88dfb8031430368ccf971
+oid sha256:c895eabbe65d8c9be65fd02a124436b2154aff20e2f9acd573496bd367d8ad1d
 size 9787088144

pytorch_model-00003-of-00005.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cdab89888e02c97ebea0e3b8165c8f51c29f453dcc25e331c10b7e2a25edfadc
+oid sha256:71e7644ee43a87b0cb150fd0543d2f0f4f2a8b97c10015549303b2bca33ae4f2
 size 9707369423

pytorch_model-00004-of-00005.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:48a9332605718b53c0b42eba1246100f7278ed35699b9ad20ded6132abccbe28
+oid sha256:26add9b5cdf3454dbba9eaf8a1255734a9b684fa058b3032c6111782cf9d0f92
 size 9711578808

pytorch_model-00005-of-00005.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b547bbd094f3b040ee5a33b6216c5d6b64ce023bdcc1ab621e1e10aa27871990
+oid sha256:c902504815cc65f0349a5180d72159fb4250de1c6d22d7320bbf0e2772ffe0b4
 size 2134105435