A while back I made a post about my 4x3090 rig in a Silverstone RV-02 . Check it out if you're a conoissuer of OG PC cases. With the incredible Qwen 3.8 27B release I ran benchmarks. So in case you are rocking 3090s you might be interested in this. Everything below is entirely LLM generated based on the benchmarks (which were also LLM driven, gotta love agentic harnesses): Setup 4x RTX 3090 24GB, NVLink between pairs (GPU0↔2, GPU1↔3 = NV4), plain PCIe between pairs vLLM 0.20.1, --kv-cache-dtype fp8 , gpu-mem-util 0.90 cyankiwi/Qwen3.8-27B-AWQ-INT4 vllm bench serve , random dataset, ignore-eos. Generation runs: 1k in / 512 out. PP runs: 16 out, single wave per concurrency level. 1. TP=2 on one NVLink pair beats TP=4 on all four cards Not a typo. From concurrency 12 up, two GPUs outrun four by 33–41% , because TP=4 pays a cross-pair PCIe all-reduce on every layer of every step, and that tax scales with batch: Concurrency 1 2 4 8 12 16 24 32 TP=4 gen tok/s 77.6 127.6 208.8 266.1 319.9 371.9 399.4 417.9 TP=2 gen tok/s 66.0 111.7 201.6 252.9 428.0 495.1 532.0 587.4 TP=4 TPOT ms 11.3 13.5 15.6 24.4 31.0 35.5 50.8 64.7 TP=2 TPOT ms 13.9 16.3 17.4 25.6 22.6 25.8 36.7 44.2 TP=4 only wins single-stream (77.6 vs 66 tok/s) and KV pool size. Run 2x TP=2 instances (one per NVLink pair) and you're at ~1,175 tok/s aggregate on hardware where a single TP=4 does 418. Same lesson likely applies to any mixed NVLink/PCIe topology — check nvidia-smi topo -m before you pick your TP. 2. MTP speculative decoding: slower at every concurrency level The model ships an MTP head and the acceptance rate was genuinely fine: 64% avg, 2.6–2.9 tokens accepted per step (spec tokens = 3). Still lost: Concurrency 1 2 4 8 12 16 24 32 baseline 77.6 127.6 208.8 266.1 319.9 371.9 399.4 417.9 + MTP 56.3 90.6 121.1 285.9 288.8 308.3 308.0 348.3 delta −27% −29% −42% +7% −10% −17% −23% −17% The problem is step economics, not the draft head: baseline decode step is 11.3ms, spec step is 39ms ( 3.5x ). On Ampere…

Read original ↗ Content from Reddit r/LocalLLaMA（Community）