Somehow I keep coming back to MythoMax. I dunno if I’m prompting newer models wrong or what but in the 13B space MythoMax just keeps giving me the best results.
Anyone have someone else they like and can recommend? Maybe something with a longer context? I feel like I have to be screwing something up is why newer models aren’t performing as well for me but I also kind of want a head nod saying that’s the case and that there’s better stuff out there.
Edit: Sorry for typo in the title but I can’t fix it. T_T
GGUF? Even on gtx 1080 you get like 4t/s with q8 which is almost as fast as average person read speed, with 16gb it should be 4-5x faster
Hadn’t thought of that. I have 24gb so I’ve always used GPTQ and with that, you really need more than 16gb.