If it generates a booting kernel and passes the test suite at 99% it's probably good enough to use, yeah.
The point isn't to replace GCC per se, it's to demonstrate that reasonably working software of equivalent complexity is within reach for $20k to solve whatever problem it is you do have.
> that reasonably working software of equivalent complexity is within reach for $20k to solve
But if this can't come close to replacing GCC and can't be modified without introducing bugs then it hasn't proven this yet. I learned some new hacks from the paper and that's great and all but from my experiencing of trying to harness even 4 claude sessions in parallel on a complex task it just goes off the rails in terms of coherence. I'll try the new techniques but my intuition is that its not really as good as you are selling it.
What does that mean, though? I mean, it's already meeting a very high quality bar by booting at all and passing those tests. No, it doesn't beat existing solutions on all the checkboxes, but that's not what the demo is about.
The point being demonstrated is that if you need a "custom compiler" or something similar for your own new, greenfield requirement, you can have it at pretty-clearly-near-shippable quality in two weeks for $20k.
And if people can't smell the disruption there, I don't know what to say.
Is it really shippable if it is strictly worse than the thing it copied. Do you know anyone who would use a vibe coded compiler that cant be modified without introducing regressions (as the researcher admitted)?
> if you spend months writing a tight spec, tests and have a better version of the compiler around to use when everything else fails.
Doesn't matter because your competitors will have beaten you to market. That's just a simple Darwinian point, no AI magic needed.
No one doubts that things will be different in the coming Claudepocalypse, and new ideas about quality and process will need to happen to manage it. But sticking our heads in the sand and pretending that our stone tools are still better is just a path to early retirement at this point.
I feel like maybe you spend too much time watching hypefluencers. AI tools are great but if they are already super intelligent why haven't you gotten a swarm of agents to build yourself a billion dollar SaaS?
It's hard to separate the bullshit from reality when the hype is just turned to the max everywhere you turn. It feels like I'm in some elaborate psy-op where my experiences with these tools are just an order of magnitude lower than the hype and I can't even express those thoughts without having "luddite" patch attached to me. And if you read between the lines of what Karpathy wrote in his famous "anxiety" post, it kind of echoes my point. Its "an alien technology and we can't yield it right" yada yada. Which is an odd way to say "sometimes this thing works magically but a lot of the time its total shit so you aren't as productive as you would like".
The point isn't to replace GCC per se, it's to demonstrate that reasonably working software of equivalent complexity is within reach for $20k to solve whatever problem it is you do have.