My rule of thumb has been to LoRA (r between 4 and 16) until unsatisfied with results. It of course depends on data/task but imo most cases don’t require full fine-tune and perf/compute ROI is low.

+1, when in doubt, LLM it out.

You could also ask for explanations so when it gets it wrong, you can work on modifying your prompts/examples to get better performance.

Potentially you wouldn’t want to do this if:

• Your classification problem is very unusual/cannot be explained by a prompt
• You want to be able to run this extremely fast or on a ton of data
• You want to learn non-LLM deep learning/NLP (in which case I would’ve suggested basically some form of finetuning BERT)

Huge fan of modal, have been using them for a couple serverless LLM and Diffusion models. Can be definitely on the costly side, but like that the cost directly scales based on requests and setup is trivial.

​

recent project with modal: https://github.com/sshh12/llm-chat-web-ui/tree/main/modal