I use in both cases q4_K_M

  • nixudos@alien.topB
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    Does anyone have a setup that works with a Yi34b model on 12GB vram and 32GB ram? I have tried using GGUF but I always end up with the <s> answer. I’m on Oobabooga and any specific settings to overcome this hurdle with GGUF would be greatly appreciated! I</s>