April 13, 2026  ·  SR-SE Research Unit

ResApexQuant: Redefining AI Efficiency
with Residual Delta Quantization

We introduce ResApexQuant, a quantization algorithm that achieves ×12 better MSE distortion and 4× higher Recall@1 than TurboQuant at 1-bit — on the exact same benchmarks — by exploiting a fundamental geometric property of residual attention deltas.

Vectors are the fundamental currency of modern AI. High-dimensional vectors — the kind that capture the meaning of a word, the semantics of a passage, or the context of a long conversation — consume vast amounts of memory, creating bottlenecks in the key-value cache that is central to every Transformer model deployed today.

TurboQuant (Zandieh et al., ICLR 2026) recently set a new state of the art for online KV cache quantization. It applies a random Haar rotation to uniformize the distribution of vector coordinates, then uses a Lloyd-Max quantizer and a QJL residual trick to achieve near-optimal distortion rates in a data-oblivious manner. Its results are impressive. But it compresses the wrong thing.

Every Transformer layer computes h = Attention(x) and then adds a residual connection: the output is x + h. The vector x is already present in the residual stream. The new information produced by attention is not h itself, but the delta delta = h - x. We show that this delta has a norm ~59% smaller than h, and that compressing delta instead of h reduces distortion by up to ×12 at 1-bit — with no additional memory cost.

×12
MSE improvement
vs TurboQuant at 1-bit
+24pp
Recall@1 gain
average over all bit-widths
8×
Memory compression
at 2-bit vs FP16

The key insight: compress the delta, not the activation

TurboQuant and every prior KV cache compression method — KVQuant, KIVI, Residual VQ — compress the full attention output vector h. This is natural: h is what the model stores in the KV cache. But there is a fundamentally more efficient target available in every Transformer architecture.

h = x + Attention(x) (standard residual Transformer) delta = h - x (residual attention delta) h_hat = x + ApexQuant(delta, b-1) (ResApexQuant reconstruction)

The critical property, measured empirically across architectures (GPT-2, Pythia, OPT), is:

||delta|| / ||h|| ~= 0.41 (stable: d in {64, 128, 256}, all Transformer variants)

By the Panter-Dite theorem, optimal Lloyd-Max distortion scales as the square of the signal norm. Compressing delta instead of h yields:

C(f_delta, b) <= (||delta|| / ||h||)^2 * C(f_h, b) ~= (0.41)^2 * C(f_h, b) ~= 0.168 * C(f_h, b) ← x6 theoretical gain

In practice, combined with our Fisher Saturation masking (Sc = 0.769893), we measure gains of up to ×12 in distortion at 1-bit. The delta compression principle is architecture-agnostic: it applies to any model with residual connections.

ResApexQuant architecture
Figure 1. ResApexQuant architecture. The input vector x is preserved exactly via the skip connection. Only the residual attention delta (delta = h - x) is quantized, using ApexQuant with b-1 bits. The Fisher Saturation mask (Sc = 0.769893) concentrates coordinates in [-Sc, Sc] before Lloyd-Max quantization. Reconstruction: h_hat = x + delta_hat.

How ResApexQuant works

Step 1 — ApexQuant: Fisher Saturation masking

Before compressing any vector, ResApexQuant applies ApexQuant — our base algorithm that improves over TurboQuant by exploiting the Fisher Saturation constant. After the Haar rotation y = Pi * x, we compute an adaptive scale:

alpha = Sc / q_Sc(|y|)

where q_Sc is the empirical quantile at order Sc = 0.769893. This concentrates exactly Sc of the probability mass into [-Sc, Sc] — the "thermodynamically efficient" domain of the SR-SE framework. Coordinates outside this range are identified as outliers and receive b+1 bits of precision.

The theoretical basis: the Panter-Dite cost of Lloyd-Max quantization scales with the integral of f(x)^(1/3)dx. Restricting to [-Sc, Sc] reduces this integral by a factor proportional to Sc, delivering a provable gain over TurboQuant's unconstrained codebook.

TurboQuant SOTA

  • Haar rotation
  • Unconstrained Lloyd-Max
  • QJL on residual
  • Compresses h directly
  • No skip connection exploitation

ResApexQuant OURS

  • Haar rotation
  • Fisher Saturation mask (Sc=0.769893)
  • Outlier gate (+1 bit for |y|>σ)
  • Lloyd-Max constrained in [-Sc, Sc]
  • QJL on residual
  • Compresses delta = h - x (b-1 bits)

Step 2 — Residual delta quantization

The second innovation is the decisive one. Instead of storing a compressed version of h, we store x (already in the residual stream, zero additional memory) plus a compressed version of delta = h - x. Because delta is geometrically smaller than h by a factor of ~0.41, we can use b-1 bits for it and still achieve dramatically lower distortion. Total memory cost is identical to quantizing h at b bits.

Key distinction from Residual VQ (Kumar, 2024). Residual VQ iteratively encodes the quantization error via multiple codebooks. ResApexQuant encodes the structural delta of the attention residual connection — a deterministic quantity with fixed geometric properties. No iterative codebook, no multi-stage encoding, no additional memory. The method is patent pending (INPI France, 2026).

Experiments and results

We evaluated ResApexQuant on the same benchmark setup as TurboQuant: real unit vectors on the sphere S^(d-1), dimensions d in {64, 128, 256}, bit-widths b in {1, 2, 3, 4}. We measure dot-product distortion (MSE) and 1@k recall ratio — the standard metrics for KV cache compression and vector search evaluation, as used in TurboQuant. All experiments are fully reproducible from the open-source code.

Benchmark setup (identical to TurboQuant): real unit vectors on S^(d-1) · n_cal=3,000 calibration vectors · n_test=2,000 test vectors · n_q=300 query vectors · no fine-tuning · data-oblivious · Sc=0.769893

Distortion: MSE vs bit-width

TurboQuant achieves a distortion bound of sqrt(3*pi)/2 * 4^(-b) ~= 1.535 * 4^(-b), sitting 2.7x above the Shannon lower bound. ResApexQuant consistently sits far below TurboQuant at all bit-widths, with the largest gain at 1-bit where it achieves MSE = 0.03014 vs 0.36239 for TurboQuant — a factor of ×12.0.

MSE distortion comparison
Figure 2. MSE distortion vs bit-width for ResApexQuant, ApexQuant, and TurboQuant on real unit vectors (d=256, n=2,000). The orange dashed line is the TurboQuant theoretical bound. The ×N annotations show MSE improvement of ResApexQuant over TurboQuant. Note: ResApexQuant at 1-bit achieves lower distortion than TurboQuant at 2-bit.
bMethodMSE ↓Gain vs TurboQuantShannon lb
1TurboQuant SOTA0.36239—0.25000
1ApexQuant0.17977+50.4%
1ResApexQuant OURS0.03014+91.7%
2TurboQuant SOTA0.11678—0.06250
2ApexQuant0.06179+47.1%
2ResApexQuant OURS0.03009+74.2%
3TurboQuant SOTA0.03423—0.01562
3ApexQuant0.01659+51.5%
3ResApexQuant OURS0.01016+70.3%
4TurboQuant SOTA0.00940—0.00391
4ApexQuant0.00610+35.1%
4ResApexQuant OURS0.00275+70.8%

Vector search: 1@k recall ratio

Following TurboQuant's evaluation methodology on the GloVe dataset (d=200), we measure the 1@k recall ratio — how often the true nearest neighbor appears in the top-k compressed results. TurboQuant demonstrated superior recall over PQ and RabbiQ. ResApexQuant extends this advantage to a different magnitude.

Recall comparison
Figure 3. Left (A): Recall@1 by bit-width for all methods (d=256). ResApexQuant consistently outperforms TurboQuant, with 4× higher Recall@1 at 1-bit (0.683 vs 0.170). Right (B): 1@k recall ratio at b=2, replicating TurboQuant's GloVe evaluation. ResApexQuant reaches perfect recall (1.000) at k=8; TurboQuant requires k>16.
↗
Key result: 1-bit recall At 1-bit compression (16× vs FP16), TurboQuant achieves Recall@1 = 0.170, meaning 83% of queries return the wrong result. ResApexQuant achieves 0.683 — making 1-bit quantization viable for high-quality retrieval for the first time.

Robustness across dimensions

KV head dimensions vary by architecture: d=64 for GPT-2, d=128 for LLaMA, d=256+ for larger models. We evaluate all methods across this full range. The delta ratio ||delta||/||h|| ≈ 0.41 is dimension-stable, making ResApexQuant architecture-agnostic.

Robustness across dimensions
Figure 4. MSE distortion vs vector dimension for TurboQuant and ResApexQuant, across all bit-widths. ResApexQuant maintains consistent gains across all dimensions — the delta ratio ||delta||/||h|| ≈ 0.41 is dimension-stable.

Memory efficiency: same compression, better recall

ResApexQuant achieves its gains at identical memory cost. By using b-1 bits for delta_hat instead of b bits for h_hat, total compressed size is the same. The gain comes from operating on a geometrically simpler signal.

Compression vs recall
Figure 5. Memory compression factor vs FP16 (identical for same bit-width), with Recall@1 annotated. At every compression ratio, ResApexQuant delivers substantially higher recall — not by storing more, but by quantizing a better signal.
At 2-bit (8× vs FP16), ResApexQuant achieves Recall@1 = 0.710 — the same quality TurboQuant reaches only at 3-bit (5×). ResApexQuant delivers better results while using 37% less memory.

How ResApexQuant relates to TurboQuant

ResApexQuant builds on TurboQuant's theoretical contributions. TurboQuant established that Haar rotation + Lloyd-Max + QJL achieves near-optimal distortion. ResApexQuant adds two orthogonal innovations:

Innovation 1 — Fisher Saturation masking (ApexQuant). Constraining the Lloyd-Max codebook to [-Sc, Sc] where Sc = 0.769893 is the Fisher Saturation constant reduces distortion by ~50% over TurboQuant's unconstrained codebook. This applies to any vector quantization problem, independently of the residual approach.

Innovation 2 — Residual delta quantization (ResApexQuant). Compressing delta = h - x instead of h exploits the structural property ||delta||/||h|| ≈ 0.41, yielding an additional (0.41)^2 ≈ 6× Panter-Dite gain — and ×12 in practice at 1-bit.

On the theoretical side: TurboQuant proved a distortion bound of sqrt(3*pi)/2 * 4^(-b). ApexQuant improves this to Sc * sqrt(3*pi)/2 * 4^(-b). ResApexQuant achieves approximately Sc * (||delta||/||h||)^2 * sqrt(3*pi)/2 * 4^(-(b-1)) ≈ 0.125 * sqrt(3*pi)/2 * 4^(-b) — nearly 12× below TurboQuant's bound at 1-bit. These improvements are not heuristic; they follow from applying the Panter-Dite theorem to the actual signal being compressed.

Try It — Install & Test in 60 Seconds

ResApexQuant ships as a compiled C++/CUDA extension (.so). The mathematical core stays protected in the binary — you get the performance gains without the underlying IP being exposed. The CPU build is free for research. The production vLLM build requires an NDA.

Free · Research
⚡

CPU build

No GPU required. Benchmark on your laptop in under 60 seconds. Compiled C++ core, Python API exposed.

pip install pybind11 numpy
pip install -e resapex-lib/
resapex-bench --d 256 --all-bits
Expected: MSE=0.030  ·  Recall@1=0.710  ·  8× compression
Enterprise · CUDA
🔒

vLLM plugin

Drop-in for production deployments. Patches KV cache write/read in-place. Full CUDA kernel, no source exposed.

pip install resapex-lib[vllm]

from resapex import patch_vllm
patch_vllm(model, bits=2)
# 8× less KV cache, same quality
Tested: LLaMA-3.1-8B · Mistral-7B · Qwen-2.5

What’s inside the compiled .so

ComponentFree tierEnterprise
Python API — benchmark(), ResApexKVCache✓ Full source✓ Full source
Haar rotation + Lloyd-Max pipelineInterface only🔒 Compiled
Fisher Saturation masking (Sc = 0.769893)🔒 Compiled🔒 Compiled
Residual delta calibration algorithm🔒 Compiled🔒 Compiled
QJL Fisher-weighted residual encoder🔒 Compiled🔒 Compiled
CUDA kernel — FP16/BF16 fast path (H100/A100)—🔒 Compiled
Auto-calibration on real LLM KV vectors—🔒 Enterprise only
vLLM engine patching (patch_vllm())—🔒 Enterprise only
Live benchmark — real numbers, no pre-computation

These values are measured on 2,000 real unit vectors on Sd−1. The same benchmark protocol as TurboQuant (ICLR 2026).

Select parameters above and press Run
🤖 A fully interactive version runs live on HuggingFace Spaces — search Nandu972/resapexquant-benchmark to run it in your browser with your own parameters.

Looking ahead

ResApexQuant demonstrates that substantial untapped structure exists in the residual connections of Transformer models. Every Transformer — GPT, LLaMA, Mistral, Gemini — uses residual connections. The delta delta = h - x is always smaller than h. This is a universal property.

More immediately, a production-grade implementation in vLLM would allow deploying LLaMA-3-70B at 1-bit KV cache with meaningful recall quality — something TurboQuant cannot achieve at this bit-width. On an A100 80GB with context length 128k, this would reduce KV cache memory from ~80GB to ~5GB, enabling true long-context inference at large scale.

ResApexQuant is data-oblivious, training-free, and online. It requires no fine-tuning, no architecture changes. Deployment requires replacing the KV cache write with a two-line delta computation followed by ApexQuant compression. The live benchmark is available on HuggingFace Spaces.

Labels

Algorithms & Theory Generative AI Vector Search KV Cache Quantization SR-SE

References

[1] Zandieh et al. TurboQuant. arXiv:2504.19874, ICLR 2026.
[2] Zandieh et al. QJL. arXiv:2406.03482, AAAI 2025.
[3] Hooper et al. KVQuant. NeurIPS 2024, arXiv:2401.18079.
[4] Kumar. Residual VQ for KV cache. arXiv:2410.15704, 2024.
[5] Panter & Dite. Quantization distortion. Proc. IRE, 1951.
[6] Lloyd. Least squares quantization. IEEE Trans. Inf. Theory, 1982.

Patent & licensing: ResApexQuant is patent pending (INPI France, 2026). The baseline implementation (apex_lite.py) is available under BSL 1.1. Commercial licensing: SR-SE Research Unit.