Hyddro26@alien.topB to LocalLLaMAEnglish · 1 year agoWhat is considered the best uncensored LLM right now?message-squaremessage-square38fedilinkarrow-up11arrow-down10file-text
arrow-up11arrow-down1message-squareWhat is considered the best uncensored LLM right now?Hyddro26@alien.topB to LocalLLaMAEnglish · 1 year agomessage-square38fedilinkfile-text
minus-squaredrifter_VR@alien.topBlinkfedilinkEnglisharrow-up1·1 year agoKoboldcpp is the easiest way. Get nous-capybara-34b.Q4_K_M.gguf (it just fits into 24GB VRAM with 8K context). Here are my Koboldcpp settings (not sure if they are optimal but they work) https://preview.redd.it/dco0bokvic1c1.jpeg?width=540&format=pjpg&auto=webp&s=bf188ea61481a9464593db79d690b26eb7989883
Koboldcpp is the easiest way.
Get nous-capybara-34b.Q4_K_M.gguf (it just fits into 24GB VRAM with 8K context).
Here are my Koboldcpp settings (not sure if they are optimal but they work)
https://preview.redd.it/dco0bokvic1c1.jpeg?width=540&format=pjpg&auto=webp&s=bf188ea61481a9464593db79d690b26eb7989883