First use probably won’t be deploying live to games.
It will be to turbocharge developers to develop / tweak storylines and content faster. Get inspiration and first drafts done 10x quicker and then polish.
It will be part of the tooling, not the actual game. But just a guess.
Mistral 7B is very good and can be run on 8gb vram. It was blazing fast on my 3070. I have a 4090 as well and for all intents and purposes its indistinguishable.
Right now Mistral7B competes with the best 13B paramater models. Unless you plan on using code LLMs there aren’t many new 30B parameter models that matter that much.
I have a 3070 on my proxmox home server with I think only 2 physical cores and 16gb ram allocated and I’m getting 40 + tokens per second.
You wouldn’t be futureproofed but would work fine now.
Hard to make a broad use case here until power consumption drops. Best approach is still push to cloud.
Edge cases like robotics / cars / high availbility likely exist though and could be big niche.