What is the app you’re using it in? I tried the 13b in Ooga Booga and wasn’t able to make it work consistently (goes and replies instead of me after a short while)
I just recently wrote my own pure python/chromadb program but before i had great success in oogabooga and this model. I think maybe there is a setting that is overlooked that maybe i enabled in oobabooga or maybe its one of the generation kwargs that just seems to work flawlessly. The model has issues with keeping its self separate from the user so take care in your wording in the system message too.
having seen the model’s tokenizer.default_chat_template that isnt unbelievable, its a real mess with impossible conditions.
My health is keeping me from making a better response but If you’re dead set on using it message me and we’ll work it out together.
I like this model the most.
What is the app you’re using it in? I tried the 13b in Ooga Booga and wasn’t able to make it work consistently (goes and replies instead of me after a short while)
I just recently wrote my own pure python/chromadb program but before i had great success in oogabooga and this model. I think maybe there is a setting that is overlooked that maybe i enabled in oobabooga or maybe its one of the generation kwargs that just seems to work flawlessly. The model has issues with keeping its self separate from the user so take care in your wording in the system message too.
having seen the model’s tokenizer.default_chat_template that isnt unbelievable, its a real mess with impossible conditions.
My health is keeping me from making a better response but If you’re dead set on using it message me and we’ll work it out together. I like this model the most.