"Naveen Rao, the Gen AI VP of Databricks, phrased it quite well:
all closed AI model providers will stop selling APIs in the next 2-3 years. Only open models will be available via APIs (…) Closed model providers are trying to build non-commodity capabilities and they need great UIs to deliver those. It's not just a model anymore, but an app with a UI for a purpose."
Already made a harness for Claude to make R/W plans, not write once like they are usually implemented. They can modify themselves as they work through the task at hand. Also relying on a collection of patterns for writing coding task plans which evolves by reflection. Everything is designed so I could run Claude in yolo-mode in a sandbox for long stretches of time.
I think that a curl wrapper could serve as a universal integration with arbitrary services (as long as they have a public API), saving you a lot of setup complexity and context space. Authentication is the hard part; ideally, you shouldn't even need an OAuth intermediary. Which I think is doable: https://news.ycombinator.com/item?id=46876636. It's still a bit of an experiment, though.
Whatever we do now to "steer" the model to do the job, my 5 cents, it will all get sucked into the model itself; skate where the puck is going as they say, and relentlessly focus on user experience and the overall product, that's how you get something like Granola.
reply