If i have multiple 7b models where each model is trained on one specific topic (e.g. roleplay, math, coding, history, politic…) and i have an interface which decides depending on the context which model to use. Could this outperform bigger models while being faster?
Several days ago, WizardLM team promote an interesting work: https://x.com/WizardLM_AI/status/1727672799391842468?s=20, we could just utilize multiple models without re-train.