HuggingFaceFW/dclm_100BT
Viewer ā¢ Updated ā¢ 89.3M ā¢ 1.82k ā¢ 1
š¤ Smol-Data
Tried and tested mixes for strong pretraining. Inspired by https://huggingface.co/blog/codelion/optimal-dataset-mixing
-
-
HuggingFaceFW/dclm_100BT-shuffled
Viewer ā¢ Updated ā¢ 89.3M ā¢ 1.13k ā¢ 3 -
HuggingFaceFW/finepdfs_100BT
Viewer ā¢ Updated ā¢ 29.9M ā¢ 1.25k ā¢ 1 -
HuggingFaceFW/finepdfs_100BT-shuffled
Viewer ā¢ Updated ā¢ 14.6M ā¢ 623 -
HuggingFaceFW/finepdfs_edu_100BT
Viewer ā¢ Updated ā¢ 17.8M ā¢ 1.19k ā¢ 1 -
HuggingFaceFW/finepdfs_edu_100BT-shuffled
Viewer ā¢ Updated ā¢ 17.8M ā¢ 848 -
HuggingFaceFW/fineweb_100BT
Viewer ā¢ Updated ā¢ 161M ā¢ 494 ā¢ 4 -
HuggingFaceFW/fineweb_100BT-shuffled
Viewer ā¢ Updated ā¢ 161M ā¢ 750 -
HuggingFaceFW/fineweb_edu_100BT
Preview ā¢ Updated ā¢ 2.7k ā¢ 4 -
HuggingFaceFW/fineweb_edu_100BT-shuffled
Viewer ā¢ Updated ā¢ 102M ā¢ 3.13k ā¢ 6 -
HuggingFaceFW/finepdfs_edu_50BT-dclm_30BT-fineweb_edu_20BT
Viewer ā¢ Updated ā¢ 56.1M ā¢ 2.63k -
HuggingFaceFW/finepdfs_edu_50BT-dclm_30BT-fineweb_edu_20BT-shuffled
Viewer ā¢ Updated ā¢ 56.1M ā¢ 1.23k ā¢ 2 -
HuggingFaceFW/finepdfs_50BT-dclm_30BT-fineweb_edu_20BT
Viewer ā¢ Updated ā¢ 62.1M ā¢ 2.71k ā¢ 3 -
HuggingFaceFW/finepdfs_50BT-dclm_30BT-fineweb_edu_20BT-shuffled
Viewer ā¢ Updated ā¢ 62.1M ā¢ 1.23k ā¢ 6