Hi everybody, that gonna be my first post ever I think :)

I constantly looking for local models that would provide answers to questions like “Write a story about ABC. The story should include xyz.”

I currently use MxLewd-L2-Q5-GUFF as one that basically follows my prompt, answer long reply (I usually set up a new token generation count at 1024) and have no issues with situations when the main character is something else than usual (a cow for example)

Yesterday I tested 70B like Twix, Dawn, and lvlz (exl2 2. x quantization allows me to load it to vram ) and only Opus eventually reached a similar level of creativity and following prompt as MxLewd, but there were some flaws (it gave up when I should write about cow :D so I expect it’s limited to human-like scenarios only)

As my system is very decent, so I’m open to many options But with I prefer to load stuff to Vram (24vram, 128ram).

For now, I gave up with gptq and awq as those lose to 5K_M GUFF and I decided to explore the wonders of exl2 as it works really nicely.

Thanks, everyone