1·
1 year agoThe problem with 4060ti on an old hardware is that it runs using only 8x PCIe lanes, and it shows on even PCIe gen3. And your I5 2500 uses even older PCIe gen2. For LLMs, if you only going to use 7B, which might fit entirely on 16GB VRAM, it should be kinda OK. But for everything else you are going to have a very bad time.
You can change it by yourself, although it’s required to be edited with every update as for now. Just put something like 200000 in these 2 places:
https://github.com/oobabooga/text-generation-webui/blob/454fcf39a95691f5e375c48fbc6fe6aa96f0c738/modules/shared.py#L46
https://github.com/oobabooga/text-generation-webui/blob/454fcf39a95691f5e375c48fbc6fe6aa96f0c738/modules/ui_model_menu.py#L100