CoPE is a drop-in enhancement of RoPE that delivers consistent gains within the training context and during long-context extrapoaltion.
Haoran Li PRO
haoranli-ml
·
AI & ML interests
ML, RL, Foundation Models
Recent Activity
updated a model about 3 hours ago
haoranli-ml/lcft_gemma-2b_prolong-gemma-parts_ProLong64KMix_bsz256_steps1250_lr1e-5_warmup0.1_rope200000rope updated a model about 4 hours ago
haoranli-ml/lcft_gemma-2b_prolong-gemma-parts_ProLong64KMix_bsz256_steps1250_lr1e-5_warmup0.1_rope200000cope published a model about 5 hours ago
haoranli-ml/lcft_gemma-2b_prolong-gemma-parts_ProLong64KMix_bsz256_steps1250_lr1e-5_warmup0.1_rope200000ropeOrganizations
models 52
haoranli-ml/lcft_gemma-2b_prolong-gemma-parts_ProLong64KMix_bsz256_steps1250_lr1e-5_warmup0.1_rope200000cope
3B • Updated
haoranli-ml/lcft_gemma-2b_prolong-gemma-parts_ProLong64KMix_bsz256_steps1250_lr1e-5_warmup0.1_rope200000rope
3B • Updated
haoranli-ml/lcft_gemma-2b_prolong-gemma-parts_ProLong64KMix_bsz256_steps1250_lr1e-5_warmup0.1_rope500000cope
Updated
haoranli-ml/lcft_gemma-2b_prolong-gemma-parts_ProLong64KMix_bsz256_steps50_lr1e-5_warmup0.1_rope200000
3B • Updated
haoranli-ml/Gemma-7B-CoPE-sft
9B • Updated • 46
haoranli-ml/Gemma-7B-RoPE-sft
9B • Updated • 30
haoranli-ml/genvf-v8-tcs-4B-v2
Updated • 3
haoranli-ml/genvf-v8-tcs-4B
Updated
haoranli-ml/genvf-v8-8B
Updated
haoranli-ml/genvf-v9_1-nextN-proof-only-8B
Updated
datasets 148
haoranli-ml/prolong-data-64K-gemma-test
Updated • 9
haoranli-ml/genvf-data-generator-100prefix-v1
Viewer • Updated • 110 • 12
haoranli-ml/genvf-filtered-proof-graded_score7_only_with_summaries-full
Viewer • Updated • 1.11k • 18
haoranli-ml/genvf-tcs-full-prl-train-prompt_v2
Viewer • Updated • 2.78k • 23
haoranli-ml/genvf-tcs-full-prl-train
Viewer • Updated • 2.78k • 25
haoranli-ml/genvf-tcs-full
Viewer • Updated • 1.42k • 32
haoranli-ml/tcs-stage1-suffixes-cross
Viewer • Updated • 2.32k • 40
haoranli-ml/genvf-tcs-prefix
Viewer • Updated • 2.32k • 27
haoranli-ml/genvf-tcs
Viewer • Updated • 435 • 62
haoranli-ml/VB
Viewer • Updated • 45 • 38