Hi community,
i am writing my own GUI in wich i want to use a LLM completely local. The problem is i dont know how to start with the LLM.
Can someone explain to me how the first steps work to integrate/work with the LLM or does someone know some good tutorials?
The LLM is downloaded localy. Now i need to integrate a library or something? sry i could not find a lot useful/direct information about the topic.
Thank you very much in advance!
If you have a GPU then I’d suggest setting up a TGI container with the correct model. If no GPU is available then use the server.cpp example in llama.cpp repository & simply invoke it from your GUI.