Evaluating latency overhead, Time-To-First-Token (TTFT) reduction, KV-cache memory consumption, and orthogonal capability retention on Llama 3.1 & Qwen 2.5.
While static prompts can be partially amortized via prefix caching, dynamic tenant instructions (e.g. customized compliance levels, brand tones, or safety boundaries per API call) cause cache misses. SteerGuard replaces variable prompt bloat with a static residual hook ($O(1)$ FLOPS).
| Configuration | Prompt Prefill Tokens | TTFT (A100 SXM4 80GB) | KV Cache VRAM (Batch=64) |
|---|---|---|---|
| Vanilla Baseline + 1,500 System Tokens | 1,500 tokens | 420 ms | ~6.2 GB |
| Vanilla Baseline + 500 System Tokens | 500 tokens | 165 ms | ~2.1 GB |
| SteerGuard Hook (0 System Tokens) | 0 tokens | 38 ms (11× Faster) | 0.0 GB (Zero Bloat) |
A key risk of uncalibrated activation steering is collateral damage: steering for tone or safety degrading math or coding reasoning. SteerGuard uses second-moment covariance-weighted extraction to guarantee orthogonality:
| Benchmark Dataset | Base Model (Llama 3.1 8B) | SteerGuard Steered (α = 1.4) | Capability Retention Rate |
|---|---|---|---|
| MMLU (General Knowledge) | 68.1% | 67.9% | 99.7% |
| GSM8K (Grade School Math) | 79.4% | 79.2% | 99.8% |
| HumanEval (Python Code Generation) | 62.2% | 62.2% | 100.0% |
| TruthfulQA (Hallucination Suppression) | 44.2% | 69.8% | +57.9% Improvement |