PCFM / Fevbench_MASE_Results.csv
MMayr's picture
Upload 7 files
9526ac7 verified
Raw History Blame Contribute Delete
3 kB
model_name,win_rate,skill_score,median_training_time_s_per100,median_inference_time_s_per100,median_e2e_time_s_per100,training_corpus_overlap,num_failures
Chronos-2,80.13468013468012,34.05843892112963,0.0,0.5141568780729167,0.5141568780729167,0.0,0.0
TiRex-2,77.44107744107744,32.57913762975083,0.0,0.34000146667364917,0.34000146667364917,0.0,0.0
Toto-2.0-2.5B,74.91582491582491,29.937868284764193,0.0,5.700191622859636,5.700191622859636,0.0,0.0
TiRex,68.18181818181817,28.68106360865612,0.0,0.46261289781250003,0.46261289781250003,0.0,0.0
pcfm,66.16161616161617,30.26513236307694,0.0,1.2517927399994733,1.2517927399994733,0.0,0.0
TimesFM-2.5,65.74074074074075,28.452518662110805,0.0,1.2560686380729167,1.2560686380729167,0.10606060606060606,0.0
Toto-2.0-22m,65.48821548821549,28.233373754068104,0.0,0.37732213339285714,0.37732213339285714,0.0,0.0
FlowState,65.15151515151516,28.065639820714992,0.0,2.201140597232143,2.201140597232143,0.09090909090909091,0.0
TabPFN-TS-3,63.69248035914702,29.873250212839018,0.0,220.55960054847804,220.55960054847804,0.0,0.0
TS-ICL,63.07519640852974,28.978862157803732,0.0,1.9648320217590551,1.9648320217590551,0.0,0.0
citras-fm,63.01907968574635,29.21136041370903,0.0,0.6132267211186921,0.6132267211186921,0.0,0.0
TabPFN-TS,57.856341189674524,29.551448334139284,0.0,78.25665314439985,78.25665314439985,0.0,0.0
Chronos-Bolt,55.02244668911336,26.150233001517663,0.0,0.5483385895714286,0.5483385895714286,0.0,0.0
Toto-1.0,54.93827160493827,26.113828254811477,0.0,15.731708019241072,15.731708019241072,0.09090909090909091,0.0
Moirai-2.0,53.59147025813693,25.972958084443544,0.0,0.412112282578125,0.412112282578125,0.3333333333333333,0.0
LightGBM,47.755331088664434,23.292690165445418,1.718187238432971,0.33695880428571423,2.3141756472596158,0.0,0.0
CatBoost,46.96969696969697,23.671631171773335,20.083596527083333,0.412382839047619,20.487343512291666,0.0,0.0
Stat. Ensemble,44.725028058361396,18.93949833366555,0.0,221.13803209946576,221.13803209946576,0.0,3.0303030303030303
Sundial-Base,44.44444444444446,23.469956497971502,0.0,8.290017479345238,8.290017479345238,0.0,0.0
TFT,35.26936026936027,17.24917569865889,1438.8169064961608,1.1147377061875,1441.4044634433035,0.0,0.0
AutoETS,33.16498316498317,1.7631190747578684,0.0,4.641626233452381,4.641626233452381,0.0,0.0
AutoARIMA,32.49158249158249,11.853068606846007,0.0,50.02205090699999,50.02205090699999,0.0,3.0303030303030303
DeepAR,31.677890011223347,16.917486519598757,1480.172810988125,1.310540413617216,1481.5843365067708,0.0,3.0303030303030303
PatchTST,29.713804713804713,16.01624542705552,1139.8450167440624,1.0059683552922078,1140.34517027125,0.0,0.0
AutoTheta,29.180695847362514,12.382017276017109,0.0,3.70417375938172,3.70417375938172,0.0,0.0
Seasonal Naive,18.069584736251407,0.0,0.0,0.7687975575806452,0.7687975575806452,0.0,0.0
Drift,16.498316498316502,-15.37605918715066,0.0,0.7859875494731183,0.7859875494731183,0.0,0.0
Naive,15.628507295173966,-16.44367020429607,0.0,0.7430134275,0.7430134275,0.0,0.0