K024/llm-sharp: Language models in C# (github.com)
I’ve recently drafted this. But adding more models & features & tests & documentation will just cost too much time. Seeking for comments & colaborators.
K024/llm-sharp: Language models in C# (github.com)
I’ve recently drafted this. But adding more models & features & tests & documentation will just cost too much time. Seeking for comments & colaborators.
Definitely interested, I loved C#/.NET but haven’t used it much in recent years. Have you used python for inference? I’m wondering how it compares to C# for LLM related programming
You have exllama, vllm, lmdeploy in python. And in most cases fastapi is used for serving an http endpoint.
I wrote llm-sharp just for dropping python (GIL, pip deps) and getting flexible adaptation to dynamic model structures apart from standard llama.