I’m using 4.5bpw quant courtesy u/Panchovix for RP.
Every time we speak about loneliness, lost love or something like this, it start t-to… s-speak… *sob* speak like… this.
And I can’t find a way to recover the conversation from this state. The narrations are fine, only character’s speech is affected. It especially prominent when using Mirostat due to lack of repetition control.
So far I tried, with little to no success:
- instruct it to speak as usual,
- rewriting it’s reply,
- temporary switching to sampler strategy with repetition penalty,
Anyone else experiences this, and how else to deal with it?
Horniness aside, is Goliath really the best model right now for roleplaying? I’m getting a bit of fomo from not being able to run this model locally, so I would like to know if there are 70B or 34B models that hold their own against Goliath in terms of RP. I have 24GB vram so a 2.6bpw 70B (a little unstable) or a 5bpw 34B is the best I can run.
It is, the 3bpw quant is noticably better then lzlv 70b. Goliath is an unruly horse. It will allow itself to be controlled until it doesn’t a s just goes and does its own thing. But it’s prose is so much better then lzlv that I’m never going back. It’s the first model that doesn’t speak like ChatGPT.