minus-squareHoppss@alien.topBtoLocalLLaMA•Hardware question: combining a 3090 and a p40linkfedilinkEnglisharrow-up1·1 year agoThis is not true, I have split two separate LLM models partially across a 4090 and a 3080 and have had them both run inference at the same time. This can be done in oobabooga’s repo with just a little tinkering. linkfedilink
This is not true, I have split two separate LLM models partially across a 4090 and a 3080 and have had them both run inference at the same time.
This can be done in oobabooga’s repo with just a little tinkering.