rihard7854@alien.topB to LocalLLaMAEnglish · 1 year agoNVidia H200 achieves nearly 12,000 tokens/sec on Llama2-13B with TensorRT-LLMgithub.comexternal-linkmessage-square23fedilinkarrow-up11arrow-down10
arrow-up11arrow-down1external-linkNVidia H200 achieves nearly 12,000 tokens/sec on Llama2-13B with TensorRT-LLMgithub.comrihard7854@alien.topB to LocalLLaMAEnglish · 1 year agomessage-square23fedilink
minus-squareyamosin@alien.topBlinkfedilinkEnglisharrow-up1·1 year agoH100 price is 30,000 dollars so i guess this one will be 70,000
minus-squareFullOf_Bad_Ideas@alien.topBlinkfedilinkEnglisharrow-up1·1 year agoThe same bench on H100 gives about 9000 tokens. And you can rent H100 for $5/h on runpod.
H100 price is 30,000 dollars so i guess this one will be 70,000
The same bench on H100 gives about 9000 tokens. And you can rent H100 for $5/h on runpod.