With high-end Android phones now packing upwards of 24GB of RAM, I think there’s huge potential for an app like this. It would be amazing to have something as powerful as the future Mistral 13B model running natively on smartphones!
You could interact with it privately without an internet connection. The convenience and capabilities would be incredible.
It’s a lot of work. Phones use a different OS and a different processor instruction set. The latter can be a big pain, especially if you’re really dependant on low-level optimizations.
I also feel that -most- people who would choose a phone over PC for this kind of thing would rather just use a high quality easily-accessible commercial option (chatGPT, etc) instead of a homebrew option that required some work to get running. So demand for such a thing is pretty low.
I’m not so sure, chatgpt has privacy issues and a small model but completely uncensored it has value too. There is a market for this. Convenient and privacy.