tiny-stack-tokenizer / config.json
pweb002's picture
Upload 4 files
a148a53 verified
raw
history blame contribute delete
279 Bytes
{
"model_type": "gpt2",
"vocab_size": 52000,
"min_frequency": 2,
"special_tokens": [
"<s>",
"<pad>",
"</s>",
"<unk>",
"<mask>",
"<code>",
"</code>",
"<error_message>",
"</error_message>",
"<description>",
"</description>"
]
}