a_slay_nub@alien.topBtoLocalLLaMA•is there any other tools like vLLM or TensorRT that can be used to speed up LLM inference?English
1·
1 year agoLmdeploy is another one
Lmdeploy is another one
Care to elaborate on what the actual reality of this EO is?
Bit disappointed by the coding performance but it is a general use case model. It’s insane how good gpt 3.5 is for how fast it is.