ozyman commited on
Commit
e1d0cc6
·
verified ·
1 Parent(s): 51e1876

Label the public index as Decision Index 0.4. Scores unchanged.

Browse files
Files changed (7) hide show
  1. README.md +6 -6
  2. areas.png +0 -0
  3. areas.svg +1 -1
  4. decision_index.png +0 -0
  5. decision_index.svg +2 -2
  6. vision.png +0 -0
  7. vision.svg +1 -1
README.md CHANGED
@@ -16,7 +16,7 @@ tags:
16
 
17
  # Drex Max
18
 
19
- **67.52 on the public Decision Index 0.3, and 73.92 on the public vision index.**
20
 
21
  Drex Max, from [Nace.AI](https://www.nace.ai/), answers typed questions about a given context, in text, JSON and images. Pass the context as `state` ([request format](https://github.com/nace-ai/drex-decision-models/blob/main/docs/request-format.md)) with one or more named questions, and get a probability for every option back in one forward pass. Nothing is generated. Drex Max is built on [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B), a dense 27B model, with a 255-way readout head, and is released under the Nace.AI Open RAIL-M license.
22
 
@@ -33,9 +33,9 @@ The backbone has 64 layers: 48 Gated DeltaNet linear-attention layers that stay
33
 
34
  ## Performance
35
 
36
- As of 19 Oct 2026. Text scores are from the [Decision Index 0.3 results](https://huggingface.co/datasets/nace-ai/drex-max-v1-decision-index-results) (`decision_index score --edition 0.3`). The kit calls this edition 0.3.
37
 
38
- ### Decision Index 0.3 (public text)
39
 
40
  | Model | Public index | Raw | Knowledge & Reasoning | Language | Retrieval | Tools | Arts |
41
  |---|---|---|---|---|---|---|---|
@@ -45,9 +45,9 @@ As of 19 Oct 2026. Text scores are from the [Decision Index 0.3 results](https:/
45
 
46
  The public index averages 37 benchmarks in five areas. All 140,178 scoreable requests were answered (140,620 requests in the run). Unanswered requests count as wrong. Area columns are chance-corrected skill × 100, from `index.json`. The Drex v2.0 row is from its [results](https://huggingface.co/datasets/nace-ai/drex-v2.0-decision-index-results). The Jev row is the public-board row cited on the [Drex v2.0 page](https://huggingface.co/nace-ai/drex-v2.0#performance), from the [leaderboard](https://huggingface.co/spaces/multimodalart/jev-decision-index).
47
 
48
- <p><img src="decision_index.png" alt="Decision Index 0.3 public index" width="720"></p>
49
 
50
- <p><img src="areas.png" alt="Area skill on Decision Index 0.3" width="720"></p>
51
 
52
  ### JevBench
53
 
@@ -59,7 +59,7 @@ Drex Max scores 94.30% (1010/1071).
59
 
60
  ### Vision
61
 
62
- Public vision index **73.92** on Decision Index 0.3, from [`vision-scores.json`](https://huggingface.co/datasets/nace-ai/drex-max-v1-decision-index-results/blob/main/runs/drex-max-v1-vision/vision-scores.json). All 12,210 rows were answered. The column is chance-corrected skill × 100. This run capped images at 1,638,400 pixels. R-Bench-M is weighted 0.5 and MMMU-Pro vision 0.25. The other rows are weighted 1.
63
 
64
  | Benchmark | Rows | Skill |
65
  |---|---|---|
 
16
 
17
  # Drex Max
18
 
19
+ **67.52 on the public Decision Index 0.4, and 73.92 on the public vision index.**
20
 
21
  Drex Max, from [Nace.AI](https://www.nace.ai/), answers typed questions about a given context, in text, JSON and images. Pass the context as `state` ([request format](https://github.com/nace-ai/drex-decision-models/blob/main/docs/request-format.md)) with one or more named questions, and get a probability for every option back in one forward pass. Nothing is generated. Drex Max is built on [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B), a dense 27B model, with a 255-way readout head, and is released under the Nace.AI Open RAIL-M license.
22
 
 
33
 
34
  ## Performance
35
 
36
+ As of 19 Oct 2026. Text scores are from the [results](https://huggingface.co/datasets/nace-ai/drex-max-v1-decision-index-results). Decision Index 0.4 uses this same public suite, so the scores are unchanged.
37
 
38
+ ### Decision Index 0.4 (public text)
39
 
40
  | Model | Public index | Raw | Knowledge & Reasoning | Language | Retrieval | Tools | Arts |
41
  |---|---|---|---|---|---|---|---|
 
45
 
46
  The public index averages 37 benchmarks in five areas. All 140,178 scoreable requests were answered (140,620 requests in the run). Unanswered requests count as wrong. Area columns are chance-corrected skill × 100, from `index.json`. The Drex v2.0 row is from its [results](https://huggingface.co/datasets/nace-ai/drex-v2.0-decision-index-results). The Jev row is the public-board row cited on the [Drex v2.0 page](https://huggingface.co/nace-ai/drex-v2.0#performance), from the [leaderboard](https://huggingface.co/spaces/multimodalart/jev-decision-index).
47
 
48
+ <p><img src="decision_index.png" alt="Decision Index 0.4 public index" width="720"></p>
49
 
50
+ <p><img src="areas.png" alt="Area skill on Decision Index 0.4" width="720"></p>
51
 
52
  ### JevBench
53
 
 
59
 
60
  ### Vision
61
 
62
+ Public vision index **73.92** on Decision Index 0.4, from [`vision-scores.json`](https://huggingface.co/datasets/nace-ai/drex-max-v1-decision-index-results/blob/main/runs/drex-max-v1-vision/vision-scores.json). All 12,210 rows were answered. The column is chance-corrected skill × 100. This run capped images at 1,638,400 pixels. R-Bench-M is weighted 0.5 and MMMU-Pro vision 0.25. The other rows are weighted 1.
63
 
64
  | Benchmark | Rows | Skill |
65
  |---|---|---|
areas.png CHANGED
areas.svg CHANGED
decision_index.png CHANGED
decision_index.svg CHANGED
vision.png CHANGED
vision.svg CHANGED