Commit History
-
add timestamp tokens (#33)
- add timestamp tokens (e12501d8fd1f9cca96b62761331e6c106f1eeacb)e8ff9f9
-
change hebrew code from iw (deprecated) to he (#6)
- change hebrew code from iw (deprecated) to he (2f637251702bb9799886af508a2863d3584bbb1a) Co-authored-by: Altryne <altryne@users.noreply.huggingface.co>42189de
-
Changes for fast tokenizer (#5)
- Add tokenizer.json (dcca07232bfb1028e499333730f868b87fd3d043) - Update unknown token (8852c40b30c9b7b981faf4fa77167fd862fd5fdb) - Move <|endoftext|> from added_tokens.json to vocab.json (3e9581879a6134abfb58f5788096027dd1756a63) Co-authored-by: Jonatan Kłosko <jonatanklosko@users.noreply.huggingface.co>a179016
-
Upload tokenizer
c659245
-
Upload processor
25da2ff