RTX 4090 vs Two RTX 3090s for Local LLMs

Used 3090s sit around $700-900. New 4090s are $1500-1800. Two 3090s cost roughly the same as one 4090 with twice the VRAM. So which wins.

What the 4090 wins at

• 8B-13B models with long context. The 4090 has 1008 GB/s bandwidth versus 936 on the 3090, and far better compute. You will hit ~250 tok/s on Llama 3.1 8B Q4 vs ~180 on the 3090.
• Single-user simplicity. No tensor parallelism, no NCCL config, no second PSU spike.
• Power efficiency. One card at 450W beats two cards at 350W each.
• Image and video generation as a side use. The 4090 is dramatically faster at SDXL and FLUX.

What the 2x 3090 wins at

• Anything 30B and up. 48 GB total VRAM lets you fit Qwen 2.5 72B, Llama 3.1 70B, Gemma 2 27B at FP16 or higher quality quants.
• Long-context 13B models. Plenty of room for 32k+ KV cache plus weights.
• Multi-user serving. Tensor parallelism scales batched throughput well on two cards.

What the 2x 3090 makes painful

• A motherboard with two PCIe x8/x8 slots and the case to fit two 3-slot cards.
• A 1200W+ PSU. Power spikes during inference can trip lower-rated supplies.
• Tensor parallel debugging on Windows is rough. Linux strongly recommended.
• Used cards have unknown remaining life and no warranty.

My rule of thumb

If your largest model is under 24 GB at your preferred quant, buy the 4090. If you actually want 70B, buy the 2x 3090 setup and accept the operational tax. If you might want 70B in six months, plan for 2x 3090 from day one.

Wait for the 5090?

The 5090 has 32 GB VRAM and 1792 GB/s bandwidth. Single card 70B at Q3 fits, Q4 is tight. Faster at everything else. Price is rough at $1999 MSRP. If you are not in a rush, it is the better single-card option.