Have changed from llama-cpp-python[server] to llama.cpp server, working great with OAI API calls, except multimodal which is not working. Patched it with one line and voilà, works like a charm!
- 0 Posts
- 5 Comments
Joined 1 year ago
Cake day: November 9th, 2023
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
No, nothing I am working on or will be working on will go to any uncontrolled whereabouts. Period. Besides, it’ll get banned immediately anyway, so why bother lol
1·1 year agoMisread title as “… thanks to F FFS” 😂 Joke aside, we should be seeing a 10-20x speed up for the current gen AI including text, sound and images. That would be a pivotal moment for gen AI.
From my experience, give it a lot, I mean really a lot, of consistent examples in the prompt how you wish it to respond in different kinds of situations, including the tones and styles. It would work. Try to give it at least 10 to 20 examples for each scenario. Consistency is the key.
Already raised an issue, couldn’t create a PR as I’m with my phone only. Solution also included:
https://github.com/ggerganov/llama.cpp/issues/4245