Hyddro26@alien.topB to LocalLLaMAEnglish · 1 year agoWhat is considered the best uncensored LLM right now?message-squaremessage-square38fedilinkarrow-up11arrow-down10file-text
arrow-up11arrow-down1message-squareWhat is considered the best uncensored LLM right now?Hyddro26@alien.topB to LocalLLaMAEnglish · 1 year agomessage-square38fedilinkfile-text
minus-squareHerr_Drosselmeyer@alien.topBlinkfedilinkEnglisharrow-up1·1 year agoWhat are you looking for? With a 3090, you can run any 13b model in 8 bit, group size 128, act order true, at decent speed. Go-tos for the more spicy stuff would be Mythomax and Tie fighter.
minus-squareshaman-warrior@alien.topBlinkfedilinkEnglisharrow-up1·1 year agoDo you know if 13b-8bit is better than 70b quantized?
minus-squareTuuNo_@alien.topBlinkfedilinkEnglisharrow-up1·1 year agohttps://github.com/ggerganov/llama.cpp/pull/1684 Higher parameter should be always better
What are you looking for?
With a 3090, you can run any 13b model in 8 bit, group size 128, act order true, at decent speed.
Go-tos for the more spicy stuff would be Mythomax and Tie fighter.
Do you know if 13b-8bit is better than 70b quantized?
https://github.com/ggerganov/llama.cpp/pull/1684 Higher parameter should be always better