Not every coding task is something you want to check into your repo. I have mostly used Gemini to generate random crud. For example I had a huge JSON representation of a graph, and I wanted the graph modified in a given way, and I wanted it printed out on my terminal in color. None of which I was remotely interested in writing, so I let a robot do it and it was fine.
Fair, but I am seeing so much talk about how it is completing actual SDE tickets. Maybe not this model specifically, but to be honest I don't care about generating dummy data, I care about the claims that these newer models are on par with junior engineers.
Junior engineers will complete a task to update an API, or fix a bug on the front-end, within a couple days with lets say 80 percent certainty they hit the mark (maybe an inflated metric). How are people comparing the output of these models to that of a junior engineer if they generally just say "Here is some of my code, what's wrong with it?". That certainly isn't taking a real ticket and completing it in any capacity.
I am obviously very skeptical but mostly I want to try one of these models myself but in reality I think that my higher-ups would think that they introduce both risk AND the potential for major slacking off haha.
