models : change convert-pt-to-ggml to use .tiktoken tokenizer files (#725) a7879d3 unverified Ivan Gorin commited on Apr 14, 2023
models : add usage comments to the HF convert script (#157) 8528ec5 unverified ggerganov commited on Nov 23, 2022
minor : fix double float32 conversion in python script e6c2e3e unverified ggerganov commited on Nov 11, 2022
Add enconding parameter to vocab.json opening to fix errors bf50d62 Joonas Pihlajamaa commited on Oct 23, 2022