I’m using the TheBloke/U-Amethyst-20B-GGUF and of all the popular models between 13b and 33b i found it’s the sweet spot. Not many regenerations needed and very good for roleplaying, not overdoing the storytelling and holds up the character card really well.
- 0 Posts
- 2 Comments
Joined 2 years ago
Cake day: November 14th, 2023
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
If you download GGUF models from “thebloke” you can read on the models card page how much RAM is required for the specific model without offloading to the GPU.
I have included a screenshot as an example of a 13b model.
https://preview.redd.it/3bv297j8c92c1.jpeg?width=1426&format=pjpg&auto=webp&s=94fac2937e8a2e0f6b3886d42401b0b50b0010b3