Ah come on, lazy? As long as it works with the runtime you wanna use, instead of hardcoding their own solution, should work fine. If you want to use Candle and have to implement new architectures with it to be able to use it, you still can, just expose it over HTTP.
I think one of the major problems with the current incarnation of AI solutions is that they're extremely brittle and hacked-together. It's a fun exciting time, especially for us technical people, but normies just want stuff to "work."
Even copy-pasting an API key is probably too much of a hurdle for regular folks, let alone running a local ollama server in a Docker container.
Unlike in image/video gen, at least with LLMs the "best" solution available isn’t a graph/node-based interface with an ecosystem of hundreds of hacky undocumented custom nodes that break every few days and way too complex workflows made up of a spaghetti of two dozen nodes with numerous parameters each, half of which have no discernible effect on output quality and tweaking the rest is entirely trial and error.
That's not the best solution for image or video (or audio, or 3D) any more than it is for LLMs (which it also supports.)
OTOH, its the most flexible and likely to have some support for what you are doing for a lot of those, and especially if yoj are combining multiple of them in the same process.
Yes, "best" is subjective and that’s why I put it in quotes. But in the community it’s definitely seen as something users should and do "upgrade" to from less intimidating but less flexible tools if they want the most power, and most importantly, support for bleeding-edge models. I rarely use Comfy myself, FWIW.
Where in the world are you getting that this project is for "normies"? Installation steps are terminal instructions and it's a CLI, clearly meant for technical people already.
If you think copying-pasting an API key is too much, don't you think cloning a git repository, installing the Rust compiler and compiling the project might be too much and hit those normies in the face sooner than the API key?
