poolside

/

Laguna-S-2.1

like 868

Follow

Poolside 1.31k

I am testing this, I been sick on the couch and running and improving my benchmark webui on it with claude and codex.
I never tried the --dry* params you used. How did you come up with those?

They were buried in the readme...

确实是解决循环问题，至少Q4不抽筋了！

Okay I cannot tell if it got better. Reasoning is triggered more but they just updated the gguf and that works better

It got better in some area but now is quite bad in tool calling. Did you changed something there?

Okay I cannot tell if it got better. Reasoning is triggered more but they just updated the gguf and that works better

Yeah I'm going to wait for unsloth to update their quants.

It got better in some area but now is quite bad in tool calling. Did you changed something there?

Hey! if you dont mind, can you check new dry settings and yarn settings in my repo? (from readme of the template), and the template, one by one, so we know what is causing this degradation? preferably, 3 passes + avg/mean error

Not possible as I'm using the NVFP4 weights in vllm. There is no dry or yarn settings. I just took your jinja template. But I guess you are using some other quant where you have a lot of problems, which is why you need those changes. For me the NVFP4 works fine (as can be seen in the screenshots).

Not possible as I'm using the NVFP4 weights in vllm. There is no dry or yarn settings. I just took your jinja template. But I guess you are using some other quant where you have a lot of problems, which is why you need those changes. For me the NVFP4 works fine (as can be seen in the screenshots).

Yeah, i use apex-i-compact which weights -20gb from nvfp4
0.00.559.878 I llama_model_loader: - type f32: 287 tensors
0.00.559.879 I llama_model_loader: - type q3_K: 114 tensors
0.00.559.879 I llama_model_loader: - type q4_K: 267 tensors
0.00.559.879 I llama_model_loader: - type q6_K: 146 tensors
0.00.559.880 I print_info: file format = GGUF V3 (latest)
0.00.559.881 I print_info: file type = Q6_K
0.00.559.883 I print_info: file size = 50.63 GiB (3.70 BPW)

For me all of those settings in combination fix insane overthinking per turn, instead of going "but wait../Actually" for 10k+ tokens, it just works, thinks more as turns get more complicated, less if its just a simple follow up toolcall.
i mean, it does also work with overthinking, but at 10t/s tg and 500pp its not what i would call a snappy experience.
what did hurt your tool calling is probably "optimizations" for claude code/harness, not sure what exactly though. ive been using this for last 2 days vibecoding and never saw a failed toolcall in CC. all edits\searches\mcp\agents calls are spot on, it even loads right skills by itself, which qwen 3.6 35b a3b never did (at nvfp4-fast by unsloth or apex-i-balanced)

anyways, if it works, dont break it

I tested the new q4km without any params it loops, with the recommended:
Temp: 0.7
TopK: 20
TopP: 0.95
MinP: 0
It is pretty bad but doesn’t loop.
But it makes mistakes left and right or refuses to do much of my prompts at all.

Temp: 0.3 made it more focused but worse.
Temp: 0.6 was better

But funnily TopK does not make much of a difference for me.

I will experiment with minP and topP.
But it takes a long time to test. At 15-20tps it takes 12h per Variation to test for me

I tested the new q4km without any params it loops, with the recommended:
Temp: 0.7
TopK: 20
TopP: 0.95
MinP: 0
It is pretty bad but doesn’t loop.
But it makes mistakes left and right or refuses to do much of my prompts at all.

Temp: 0.3 made it more focused but worse.
Temp: 0.6 was better

But funnily TopK does not make much of a difference for me.

I will experiment with minP and topP.
But it takes a long time to test. At 15-20tps it takes 12h per Variation to test for me

if you are using llama.cpp, try to add dry-sampling. also "refuses to do much of my prompts at all." for me was a "non-moe-aware" quant (IQ3_M in my case) issue. Apex-i-compact steers very well.

Left it overnight to cook a plan with /goal, over 1000 toolcalls, 10+ agents, no issues at all, and kimi k3 rated plan as "good to implement"

That solves something mysterious I found.
With iq4nl from unsloth it performed the best, but only sometimes.
Like 10 being shit and then my sims neighborhood prompt, it outperformed qwen, gemma and deepseekv4 flash.
But also only once and I am comfused.
Q4km never achieved that, but appears to be more stable.
Maybe its the params…
Also iq4nl seems to perform best with kv cache quant q8 instead of f16 or bf16.
The others I tested fail at q8:
Unsloth ud Mxfp4, unsloth ud q4kxl.
Apex one from Myril I think was their name was the fastest but worse, although stable throughout all kv quant sizes.

I will try the dry sampling thing with the no param and the recommended temp topk etc.

Thank you

there is also a matter of this as far as KV cache issues go. apparently, you gotta set yarn-attn-factor = 1.0 instead of stock 1.43xxxx since its wrong, or smth. i did and Myrics apex-i-compact works wonders for me, with dry as follows:
cmd = [
r".\llama-server.exe",
"-m", r"D:\Laguna-S-2.1-APEX-i-compact.gguf",
"--reasoning-preserve",
"--reasoning-budget", "16000",
"--reasoning-budget-message", "I've thought enough. Answering now with what I have.",
"-ngl", "999",
"--n-cpu-moe", "47",
"--rope-scaling", "yarn",
"--rope-scale", "32",
"--yarn-orig-ctx", "8192",
"--yarn-attn-factor", "1.0",
"--load-mode", "dio",
"-c", "170000",
"--cache-type-k", "q8_0",
"--cache-type-v", "q8_0",
"-np", "1",
"-fa", "on",
"-t", "8",
"-tb", "8",
"-b", "3072",
"-ub", "3072",
"--jinja",
"-kvu",
"--temp", "0.7",
"--top-p", "0.95",
"--top-k", "20",
"--min-p", "0.0",
"--dry-multiplier", "0.6",
"--dry-base", "1.75",
"--dry-allowed-length", "4",
"--dry-penalty-last-n", "-1",
"--dry-sequence-breaker", r'\n,:,",*,;,{,}',
"--samplers", "top_k;top_p;min_p;temperature;dry",
"--alias", "laguna-s-2.1",
"--cache-reuse", "256",
"--cache-ram", "1024",
"--host", "127.0.0.1",
"--port", "8080",
"--verbosity", "4",
"--chat-template-file", r"G:\xlam3\laguna\chat_template_laguna.jinja",
]

@sanjxz
First — thank you. Your agentic template + recipe are what made Laguna-S-2.1 genuinely usable for me: it's now driving both a local agent (Hermes) and a coding lane on an all-Mac fleet, where before it wandered. Real, load-bearing work — appreciated.

I wanted to share a controlled result, because it refines which part of the recipe does the work — and it's not the part I expected.

Setup. Laguna-S-2.1 Q4_K_M (poolside-official GGUF) on the poolside fork (b10008), Apple Metal, 256k ctx, q8_0 KV, your agentic template, --top-k 20 --min-p 0, your DRY params (--dry-multiplier 0.6 --dry-base 1.75 --dry-allowed-length 4 --dry-sequence-breaker "\n,:,",*,;,{,}"). Task: a real multi-turn code-nav + edit job on an existing repo, N=5, vs template-only baseline.

Result. Open-ended commits went 0/N on every prior config → 4/5 with the full recipe. One run completed fully and caught a latent bug our other tooling (antirez/ds4 deepseekv4flash and Claude Opus) + human review had missed.

The refinement — the reasoning-budget never engaged. Trace-checked across 183 turns: --reasoning-budget 16000 + halt-message fired zero times (max reasoning ~7k tokens; the template already suppresses thinking below the 16k cap on tool-heavy turns). The active ingredient is DRY-with-sequence-breakers: the repetition penalty discourages "read the same file again" and pushes novel actions → edits. (Side effect: ~1/5 runs the same pressure pushed it into filesystem exploration when in-project reads got penalized — contained, but worth knowing.)

Two things for others: (1) the --dry-sequence-breaker set is essential — naked DRY corrupted tool-call paths for us; with your breakers, zero corruption across all runs. (2) On agentic coding the budget is inert, but it likely still matters for pure long-reasoning tasks that exceed 16k tokens — a different regime.

Net: your recipe works; on agentic coding the mechanism is DRY-with-breakers, not the budget. Thanks again — this moved Laguna from "spec-executor only" to a viable open-ended coder for me. When it works, it is almost at par with Antirez/ds4 on agentic-coding.

there is also a matter of this as far as KV cache issues go. apparently, you gotta set yarn-attn-factor = 1.0 instead of stock 1.43xxxx since its wrong, or smth. i did and Myrics apex-i-compact works wonders for me, with dry as follows:
cmd = [
r".\llama-server.exe",
"-m", r"D:\Laguna-S-2.1-APEX-i-compact.gguf",
"--reasoning-preserve",
"--reasoning-budget", "16000",
"--reasoning-budget-message", "I've thought enough. Answering now with what I have.",
"-ngl", "999",
"--n-cpu-moe", "47",
"--rope-scaling", "yarn",
"--rope-scale", "32",
"--yarn-orig-ctx", "8192",
"--yarn-attn-factor", "1.0",
"--load-mode", "dio",
"-c", "170000",
"--cache-type-k", "q8_0",
"--cache-type-v", "q8_0",
"-np", "1",
"-fa", "on",
"-t", "8",
"-tb", "8",
"-b", "3072",
"-ub", "3072",
"--jinja",
"-kvu",
"--temp", "0.7",
"--top-p", "0.95",
"--top-k", "20",
"--min-p", "0.0",
"--dry-multiplier", "0.6",
"--dry-base", "1.75",
"--dry-allowed-length", "4",
"--dry-penalty-last-n", "-1",
"--dry-sequence-breaker", r'\n,:,",*,;,{,}',
"--samplers", "top_k;top_p;min_p;temperature;dry",
"--alias", "laguna-s-2.1",
"--cache-reuse", "256",
"--cache-ram", "1024",
"--host", "127.0.0.1",
"--port", "8080",
"--verbosity", "4",
"--chat-template-file", r"G:\xlam3\laguna\chat_template_laguna.jinja",
]

This made mine better for the one-shot html+js coding benchmarks, but also introduced looping for Iq4nl, I am scared to try it with UD Q5xl though as I only have 5-8tps there.
At least I implemented a looping check and it cuts it off then.
Would you recommend to set the yarn params even lower for just 64k context?
Also I think I see no max tokens here, what numbers do you get?
I often get like 8k-20k reasoning tokens and when reasoning budget is set, it just ignores it.

With all of this + just compiled mainline llama with load-mode mlock and apex-i-compact (~3.7 BPW) i get 500pp 11tg on my system with full MoE weight offloading to cpu, and no loops\broken toolcalls\whatever in pi code and claude code. never saw reasoning go above 16k tokens even on hardest tasks i gave it, and it averages way lower, some turns are 100 tokens max, some more complex up to 1000, finalizing a plan can go to 12k.
when i had reasoning budget set lower to 1k, it worked as expected too. but its not a right way to fix overthinking, "ideally" you should also inject a message halfway through & at like 85%, which is unsupported by llama as of now. (well ideally it should work as reasoning effort works in CC\Codex\etc )

asked K3 to rate plans and it rated all of them as good & ready to implement, as well as code after implementation, and it worked perfectly in one shot (python with submodules + adb/ffmpeg-like encoders/DEE mostly).

as for rope scale, im not sure it will yield any results , but for 65k you can try to set it to 65000 / 8192 = 7.93... ~= 8

Hey @Neiko2002 i updated the template, if you dont mind testing it with kwarg enable_toolcall_instructions false & true, can you do it? false is native laguna template style for toolcalls, true is qwen fork style.

Hi @sanjxz , here are the test results. At least the NVFP4 weights in vLLM work better with tool call instructions. The new template with tool call instructions disabled work better, than the old template (called loopguard), but is in both cases "tool call instructions = true or false" slower by about 3–6 minutes and produce more tokens . These numbers represent the best run of three (not the average).

I think it is important to move this discussion to the correct repository. All of the problems mentioned in this thread belong to "poolside/Laguna-S-2.1-GGUF". Both "poolside/Laguna-S-2.1" and "poolside/Laguna-S-2.1-NVFP4" are working fine in vLLM.

@Wladastic Please keep in mind, this model does not support vision, is not trained on web design, and is therefore not suited for it.

@all In general, this is an "agentic coding and long-horizon work" model that requires an agentic harness loop. It does a lot of self-correction, and without it, it fails to provide a good answer. It should therefore not be used in a simple chat application like the ones provided by LM Studio or Unsloth Studio.

I see, thanks! I made them default off for anyone who wants to try. i cant see the difference in pi\CC either way on the quant i run. only issue i noticed so far is that it sometimes convinces itself that HEVC (x265) is HEBC or something like that, then fails and self corrects\i steer it. fully stock template is noticably worse at self correcting for me, but that is maybe due to extreme sub 4 bit quantization. YMMV.

Hi @sanjxz , here are the test results. At least the NVFP4 weights in vLLM work better with tool call instructions. The new template with tool call instructions disabled work better, than the old template (called loopguard), but is in both cases "tool call instructions = true or false" slower by about 3–6 minutes and produce more tokens . These numbers represent the best run of three (not the average).

I think it is important to move this discussion to the correct repository. All of the problems mentioned in this thread belong to "poolside/Laguna-S-2.1-GGUF". Both "poolside/Laguna-S-2.1" and "poolside/Laguna-S-2.1-NVFP4" are working fine in vLLM.

@Wladastic Please keep in mind, this model does not support vision, is not trained on web design, and is therefore not suited for it.

@all In general, this is an "agentic coding and long-horizon work" model that requires an agentic harness loop. It does a lot of self-correction, and without it, it fails to provide a good answer. It should therefore not be used in a simple chat application like the ones provided by LM Studio or Unsloth Studio.

I know it has no Vision, I let the models one-shot an html, not agentic.
So far for me the best fix was to set Rope Scaling and yarn down.
Once that was done the model (Apex compact GGUF in llamacpp) outperformed Gemma4 26b as well as Ornith 35b and Qwen3.6 35b.
I can't tell if it is any better than the Dense variants Gemma4 31b and Qwen3.6 27b though.
I am downloading the new variants and nvfp4, as I have high hopes now.
Will test in the evening though

I have not tested it as agentic yet, as I am still tweaking the prompt processing and I get weird results from 1-2 seconds for 800 token prompts or it takes 6 Minutes.