My current specs are 7950x cpu, 96gb 6000 mhz ram and a rtx 4090. Would a buying a p40 make bigger models run noticbly faster? If it does is there anything I should know about buying p40’s? Like do they take normal connectors or anything like that. I’m pretty sure I’ll have to buy a cooler for it but idk what else.
Also are there better alternatives to the p40 that don’t cost a crazy amount?
The P40 is about as cost effective as it gets. But it’s a bit of work to get it working. Also keep in mind that air cooling a P40 is either ineffective or extremely loud.
3090 is ~$650 and will be a lot faster and pair better with a 4090
I keep seeing these low numbers. Where are people finding 3090s that cheap? I was lucky to get a pair at about 900 each, but most places I see them listed, even refurbished ones are $800-1K.
Yea, I got mine by using an eBay auction using a sniper tool patience. Make sure to look at the seller and auctions.
If you get lucky, you can get a 3090 for about $650 on ebay. But you have to be patient.
even refurbished ones are $800-1K.
You can get open box ones with 2 year warranty for less than that. Zotac had open box 3090s for like $729(If I remember right) earlier this week. If you get really lucky they rarely have the watercooled ready ones new for $799. But you have to supply your own block.
Yea I got a watercooled zotac trinity from ebay, around 800 bucks but for EU that’s good.
The big issue is that you’re going to have to disable 16bit floats for doing all the work and do it all in 32bit floats (not storing weights, but the calculations themselves) once you try to combine with a P40, you can still get alright performance on them (I’m using 4 of them) but you’ll cripple the performance of the 4090 doing that. I don’t know if any of the libraries for running things will handle conversion and different kernels on different cards to avoid that since it’s a completely different set of code for that.
You’d do much much better with adding a used 3090 from ebay (assuming it works) really.
Could you elaborate on “disabling 16 bit floats” alittle bit more?
I run 2x P40s with 70b chat and 8k ctx I get 7-8 T/s and I’m very happy with that. Anything above 5 is awesome for me.