Part of it is trying to find models that will work in my own projects so I don’t have to rely on OpenAI or some other API. And I’m also interested in the lightweight end of things that may run well on isolated devices without a connection like robots or wearable assistants in isolated places.
If I’m developing something with AutoGen or MemGPT that uses a lot of API calls, testing using a local LLM is free.
Not to mention the company I work for is interested in using AI but would rather not send customer data to any of these big companies.
Part of it is trying to find models that will work in my own projects so I don’t have to rely on OpenAI or some other API. And I’m also interested in the lightweight end of things that may run well on isolated devices without a connection like robots or wearable assistants in isolated places.
If I’m developing something with AutoGen or MemGPT that uses a lot of API calls, testing using a local LLM is free.
Not to mention the company I work for is interested in using AI but would rather not send customer data to any of these big companies.