# DenseMem-Folded Ram Tech

**URL:** <https://discuss.huggingface.co/t/densemem-folded-ram-tech/175488>\
**Category:** Models\
**Created:** [April 23, 2026, 3:49pm UTC](https://discuss.huggingface.co/t/densemem-folded-ram-tech/175488 "2026-04-23T15:49:25Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Zoberzzz](https://avatars.discourse-cdn.com/v4/letter/z/a88e57/32.png) [@Zoberzzz](https://discuss.huggingface.co/u/Zoberzzz)\
**Post date:** [April 23, 2026, 3:49pm UTC](https://discuss.huggingface.co/t/densemem-folded-ram-tech/175488/1 "2026-04-23T15:49:25Z")

</div>

Title: I built a KV cache compression protocol — 256x ratio, 0.9994 fidelity, running live on an RTX 4090

* * *

Hey r/LocalLLaMA,

I’ve been running a 72B model’s full KV cache in 640MB of DDR5 RAM on my RTX 4090 + Core i9. Wanted to share what I built.

**DenseMem v0.2.0 — FoldedMemory Protocol**

The problem: a 72B model at 32K context needs ~160GB of KV cache. That’s H100 territory. Most of us can’t touch it.

The insight: KV cache activations aren’t random. They’re highly structured and correlated. SVD at rank=64 exploits that geometry. The compression is lossy in theory but in practice the fidelity holds at 0.9994 cosine similarity — because real transformer activations live in a low-dimensional subspace.

**Live benchmark (RTX 4090 + Core i9 + DDR5):**

- Compression: 256x
- Fidelity: 0.9994 cosine similarity
- Negative control (random noise): 0.12 — confirms it’s exploiting structure, not luck
- Avg fetch latency: 1.95ms
- Max fetch latency under load: 3.96ms
- Evictions: 2,944 clean
- 16,384 MB → 63.9 MB live test

**Architecture:**

Two-tier hierarchy — VRAM hot, DDR5 warm. Attention-weighted eviction (0.5 attn + 0.3 recency + 0.2 freq). Prefetcher using layer lookahead + sequential token prediction. Two-method API: store() and fetch().

**Current limitation:**

Hit rate is 25% — my i9’s 2-channel DDR5 is the bottleneck (~38 GB/s). On Threadripper PRO 8-channel DDR5 (~224 GB/s) I’m projecting 65-75% hit rate with sub-2ms latency.

**Running live:**

Qwen2.5-7B at 32K context on a single 4090. Every tick compressed INT8 via PCA into DDR5. Context went from 4K to 32K — 8x expansion via DenseMem.

**Cost:**

Uncompressed 72B KV cache at 32K ctx: $32,000 in HBM3e.  
FoldedMemory: $1.88 in DDR5.

GitHub: [GitHub - thorshammerztp-arch/densemem-protocol: DenseMem protocol package · GitHub](https://github.com/thorshammerztp-arch/densemem-protocol) Patent pending (US 64/045,595).

Happy to answer questions on the compression math, architecture, or benchmark methodology.

* * *

_Built by a solo developer / Navy veteran on personal hardware. No funding._