Power User
  • Communities
  • Create Post
  • Create Community
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
oobabooga4@alien.topB to LocalLLaMAEnglish · 1 year ago

QuIP#: SOTA 2-bit quantization method, now implemented in text-generation-webui (experimental)

github.com

external-link
message-square
6
fedilink
1
external-link

QuIP#: SOTA 2-bit quantization method, now implemented in text-generation-webui (experimental)

github.com

oobabooga4@alien.topB to LocalLLaMAEnglish · 1 year ago
message-square
6
fedilink
Add QuIP# support by oobabooga · Pull Request #4803 · oobabooga/text-generation-webui
github.com
external-link
QuIP# is a novel quantization method. Its 2-bit performance is better than anything previously available. Repository: https://github.com/Cornell-RelaxML/quip-sharp Blog post: https://cornell-relaxm...
  • iChrist@alien.topB
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    Omg how can I run it on 3090?

LocalLLaMA

localllama

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !localllama@poweruser.forum

Community to discuss about Llama, the family of large language models created by Meta AI.

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 4 users / day
  • 4 users / week
  • 4 users / month
  • 4 users / 6 months
  • 1 local subscriber
  • 5 subscribers
  • 1.02K Posts
  • 5.82K Comments
  • Modlog
  • mods:
  • communick
  • BE: 0.19.8
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org