Is the 3060 enough to be used in a small office for document questions and answers ? What is the best model to use? The cpu and RAM will be 4790k 32 gb.
You must log in or register to comment.
My office PC had a RTX2060 12G and it runs 13b models at 4bit no problem.
That’s pretty much it’s limit though. 13b 4bit + 4096 context would max out the vram, but it is stable.
thanks