minus-squaredeviantkindle@alien.topBtoLocalLLaMA•Is there a technical reason that distributed LLMs don't exist?linkfedilinkEnglisharrow-up1·1 year agoI was thinking of distributed MoEs as well. Question I have is how do you route queries? I don’t know how to do that if all the Es are in the same cluster let alone distrivuted. linkfedilink
I was thinking of distributed MoEs as well.
Question I have is how do you route queries? I don’t know how to do that if all the Es are in the same cluster let alone distrivuted.