train_multirc_456_1767437476

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the multirc dataset. It achieves the following results on the evaluation set:

• Loss: 0.1345
• Num Input Tokens Seen: 264580656

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2259	1.0	6130	0.1345	13210560
0.0277	2.0	12260	0.1371	26427632
0.0134	3.0	18390	0.1444	39656608
0.0069	4.0	24520	0.1434	52911264
0.0022	5.0	30650	0.2154	66151456
0.2126	6.0	36780	0.2699	79368416
0.0025	7.0	42910	0.3077	92601264
0.0	8.0	49040	0.3135	105825424
0.0051	9.0	55170	0.3736	119051808
0.1258	10.0	61300	0.3859	132282512
0.0002	11.0	67430	0.3017	145503424
0.0	12.0	73560	0.4880	158718688
0.0	13.0	79690	0.3862	171968272
0.0	14.0	85820	0.4405	185192912
0.0	15.0	91950	0.4823	198406496
0.0	16.0	98080	0.5583	211654512
0.0	17.0	104210	0.5550	224875552
0.0	18.0	110340	0.5691	238097088
0.0	19.0	116470	0.5758	251336240
0.0	20.0	122600	0.5760	264580656

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1