panchovix@alien.topB to LocalLLaMAEnglish · 2 years agoTabbyAPI released! A pure LLM API for exllama v2.github.comexternal-linkmessage-square6linkfedilinkarrow-up11arrow-down10
arrow-up11arrow-down1external-linkTabbyAPI released! A pure LLM API for exllama v2.github.companchovix@alien.topB to LocalLLaMAEnglish · 2 years agomessage-square6linkfedilink
minus-squareRight-Structure-1619@alien.topBlinkfedilinkEnglisharrow-up1·2 years agoDoes anyone know if they expose all the good stuff that Guidance uses for their guided generation and speedup? This plus guidance (kv cache, grammar control, etc) would be fast fast!
Does anyone know if they expose all the good stuff that Guidance uses for their guided generation and speedup? This plus guidance (kv cache, grammar control, etc) would be fast fast!