Hi I am a newbie c# dev, I am trying to create a home project and until recently I was using Llamasharp. There is little support for it and since the last updates I’ve been unable to get it to work at all with the recent updates.
i’m trying to build a little chat wpf application which can either load AWQ or GGUF LLM files. Are there any simple and easy to use libraries out there which I can facilitate in c#?
I have a GTX 3060 and I’d preferably like to use my GPU RAM if it’s faster than using DDR4 RAM. I admit I am under a few misconceptions. Ideally I’d like to be able to load the Mistral models in c#.
This answer is somewhat OT, but may be the best answer for your situation. Take it from someone who started coding C# in 2001.
The worst mistake a Dev can make is call themselves “Im a ___ Dev”. This is an option limiting mental handicap.
Way back I sunk all my interest in the Semantic Web on porting Jena into NJena. Almost finished the conversion but never built anything useful.
For your problem, dockerize Ooba, llamacpp, etc exposing an api endpoint, call API via ms semantic kernel from your wpf app. Profit…
Better spend your time on learning containerisation then on coping with missing options in you chosen ecosystem.