minus-squarequaquaversal_@alien.topBtoLocalLLaMA•How much more stupid is the 120B goliath Q3_K_M than the larger options?linkfedilinkEnglisharrow-up1·1 year agoWhat’s the tok/s for each of those models on that system? Edit: also, if you don’t mind my asking, how much context are you able to use before inference degrades? linkfedilink
What’s the tok/s for each of those models on that system?
Edit: also, if you don’t mind my asking, how much context are you able to use before inference degrades?