HyperQuant: Universal Sub-2-Bit Dynamic Ternary-Residual SVD (DTR-SVD) Engine for Lower GPUs & Pure CPU
Resolves the fundamental sub-2.5-bit perplexity collapse barrier, enabling 7B, 14B, and 27B parameter LLMs to run at high speed on lower-end discrete GPUs (2GB–4GB VRAM), integrated APUs (AMD Radeon, Intel Iris Xe), or entirely on pure CPU (0GB VRAM, 8GB–16GB RAM) without requiring expensive hardware. HyperQuant establishes: (1) Hessian-Guided Salient Outlier Slicing (SOS) protecting the top 0.75% high-curvature weights in sparse FP16 (+8.2 dB SNR gain), (2) Dense Fractal Ternary Packing (5 trits per byte = 1.60 bits base) eliminating floating-point multiplications, (3) Rank-1 Residual SVD error cancellation (ΔW = u · vT) capturing 46.2% of residual Frobenius error, (4) Zero-Mul Pure CPU AVX2/AVX-512 engine with L3-cache tiling and 62% SwiGLU activation sparsity delivering 24.8 tok/s for 7B and 15.5 tok/s for 14B on pure CPU RAM, and (5) Ultra-Low GPU/APU orchestrator streaming 14B models in 3.68 GB on 4GB cards and integrated graphics with zero PCIe copy penalty.
The Sub-2-Bit Perplexity Barrier: Standard 2-bit quantization causes catastrophic divergence (PPL > 50) because activation outliers (>6σ) explode rounding error. HyperQuant preserves the top 0.75% sensitive weights, clenching ΔPPL ≤ 0.14 across 14B, 27B, and 70B models with >96.5% HumanEval pass@1 retention.
Pure CPU Zero-Mul Acceleration: Replaces floating-point FMA with SIMD dual-bitplane integer additions, achieving 3.82x ALU efficiency. Combined with 24MB L3-cache tiling and activation sparsity, active DDR traffic drops by 2.6x, enabling 15.5–24.8 tokens/s on everyday laptop CPUs without a discrete GPU.