I am a newbie to running a llm locally, but I would like to make one and use it to convert long books to short notes. I am a med student with a slightly above average knowledge of this tech (compared to general public) will it be possible for me to achieve this with a non tech background? If it is then it would mean a lot if someone can point me in the right direction.
I am not really an expert myself but I will give it my shot.
The problem:
If you load a downloaded LLM and just try to feed it a book and some instructions it will breakdown within seconds. LLMs have limits on their context window that can barely handle a roleplay after 10 minutes of chatting. If my roleplay character can’t even remember their best friend’s name after 10 minutes of chatting then it will not be able to process a whole book.
The Solution:
What you need to do is actually include the book in the training dataset. Instead of feed a LLM the book via a prompt you would need it to already know the book from it’s original training. The strong AI nerds here can probably do this but most people here are downloading models from the HuggingFace and testing/trying them out. Training a model with your own dataset could be fun but it will require you do some research/self teaching on how to do it and then you will still need the GPU processing power to build it.