Use case is that I want to create a service based on Mistral 7b that will server an internal office of 8-10 users.
I’ve been looking at modal.com, and runpod. Are there any other recommendations?
Use case is that I want to create a service based on Mistral 7b that will server an internal office of 8-10 users.
I’ve been looking at modal.com, and runpod. Are there any other recommendations?
Huge fan of modal, have been using them for a couple serverless LLM and Diffusion models. Can be definitely on the costly side, but like that the cost directly scales based on requests and setup is trivial.
recent project with modal: https://github.com/sshh12/llm-chat-web-ui/tree/main/modal