Longjumping-Bake-557@alien.topB to LocalLLaMAEnglish · 3 years agoDiscrepancy between TheBloke_Orca-2-13B-GPTQ and the original one with the tested logic questionalien.topimagemessage-square2linkfedilinkarrow-up11arrow-down10
arrow-up11arrow-down1imageDiscrepancy between TheBloke_Orca-2-13B-GPTQ and the original one with the tested logic questionalien.topLongjumping-Bake-557@alien.topB to LocalLLaMAEnglish · 3 years agomessage-square2linkfedilink
minus-squarepanchovix@alien.topBlinkfedilinkEnglisharrow-up1·3 years agoTry at FP16 or 8bit at most, probably a 13B models suffers too much at 4bits.
Try at FP16 or 8bit at most, probably a 13B models suffers too much at 4bits.