oobabooga4@alien.topB to

LocalLLaMAEnglish · 1 year ago

QuIP#: SOTA 2-bit quantization method, now implemented in text-generation-webui (experimental)

6

1

QuIP#: SOTA 2-bit quantization method, now implemented in text-generation-webui (experimental)

oobabooga4@alien.topB to

LocalLLaMAEnglish · 1 year ago

6

Add QuIP# support by oobabooga · Pull Request #4803 · oobabooga/text-generation-webui

QuIP# is a novel quantization method. Its 2-bit performance is better than anything previously available. Repository: https://github.com/Cornell-RelaxML/quip-sharp Blog post: https://cornell-relaxm...

Chat

a_beautiful_rhind@alien.topB
link
fedilink
English
arrow-up
1·
1 year ago
From the issue about this in the exllamav2 repo, quip was using more memory and slower than exl. How much context can you fit?