Qwen 27b Q3_K_M 2x rx 580 8gb (~50$ each in my country, edge cases 60$ per gpu) gives us 16gb vram We used it on an old already existing ddr3 motherboard with 2 gpu slots(you can buy it ror around 200$ with 32 gb of ddr3 ram, a workstation xeon cpu and a workstation motherboard, used) Its not the best option, but it makes running this model possible for many people, its even cheaper than system ram Limitations: very low processing speed(only 14t/s) means an mtp model would be a loss, and high input tokens would be a painful experiance Not recommanded if you care about ease of life, very recommanded if you need something cheap to work no matter the compromise submitted by /u/Whole_Alternative_18 [link] [comments]
Read original ↗ Content from Reddit r/LocalLLaMA（Community）