I’m curious how using local LLMs has helped your industry and why you’d opt for a local LLM over something like an API from ChatGPT. I’ve been working quite a bit with Stable Diffusion and am interested in branching out into other AI like LLMs. I do use ChatGPT for various small tasks but I’m kind of trying to wrap my mind around the scope of how LLMs can help business, particularly locally run LLMs. Also, is the draw of local LLMs mainly privacy, training, etc?
For me it’s privacy and multi tasking NLP. Traditional NLP needed a lot of vertical fine tuning for each task whereas a capable LLM really just needs a single lateral fine tune.
And obvious, privacy. Just offering semantic search and query of a companies knowledge base is a huge value add. More companies than not don’t want to rely on an API accessing all their sales data etc, but you can make an air gapped LLM and give them comparable capabilities.
Also I do creative audio and video work as a side gig and just normal LLM things helping work through ideas or make sure I’m not missing an obvious creative choice. A lot of the local models have fine tunes just for creative writing and they are so much fun to play with as a creative tool.