Break the Sequential Dependency of LLM Inference Using Lookahead Decoding | LMSYS Org
lmsys.org

TL;DR: We introduce lookahead decoding, a new, exact, and parallel decoding algorithm to accelerate LLM inference. Look...

  • wind_dude@alien.topBEnglish
    1·
    1 year ago

    What would happen if you replace the decoder during finetuning? Would you also see a speed up but at the expense of vram?