I’m struggling to get the 7b models to do something useful, obviously I’m doing something wrong as it appears many people strive for 7b models.
But myself I can not get them to follow instructions, they keep repeating stuff and occasionally they start to converse with themselves.
Does anyone have any pointers what I’m doing wrong?
OpenHermes 2.5 is amazing from what I’ve seen. it can call functions, summarize text, is extremely competitive, all the works
How does it function call? Some internal api?
It returns a JSON with function name and respective arguments which you can parse later in the program and call the function with those arguments given by the model.
it outputs the call
https://twitter.com/abacaj/status/1727747892922769751
Can you provide the prompt for function call?
I’m seconding that. I’m actually amazed by how it performs, frequently getting similar or better answers than bigger models. I start to think that we do lose a lot with quantization from the bigger models…
Haven’t you noticed slower inference from OpenHermes 2.5 compared to other 7B models?