Using AI is like trying to build a product with only extremely junior devs. You can get something working fairly quickly but sooner rather than later it's going to collapse in on itself and there won't be anyone around capable of fixing it. Moving past that requires someone with more experience to figure out what they did and eventually fix it, which ends up being far more expensive than correcting things along the way.
The error rate needs to come down significantly from where it's currently at, and we'd also need AIs that cover each others weaknesses - at the moment you can't just run another AI to fix the 20% of errors the first one generated because they tend to have overlapping blind spots.
Having said all that, I think AI is hugely useful as a programming tool. Here are the places I've got most value:
- Better IDE auto-complete.
- Repetetive re-factors that are beyond what's possible to do with a regex. For example, replaing one function with another whose usage is subtly different. LLMs can crack out those changes across a codebase and it's easy to review.
- Summarizing and extracting intent from dense technical content. I'm often implementing some part of a web standard, and being able to quickly ask an AI questions, and have it give references back to specific parts of the standard is super useful.
- Bootstrapping tests. The hardest part of writing tests in my experience is writing the first one - there's always a bit of boilerplate that's annoying to write. The AI is pretty good at generating a smoke test and then you can use that a basis to do proper test coverage.
I'm sure there are more. The point is that it's currently incredibly valuable as a tool, but it's not yet capable of being an agent.
The error rate needs to come down significantly from where it's currently at, and we'd also need AIs that cover each others weaknesses - at the moment you can't just run another AI to fix the 20% of errors the first one generated because they tend to have overlapping blind spots.
Having said all that, I think AI is hugely useful as a programming tool. Here are the places I've got most value:
- Better IDE auto-complete.
- Repetetive re-factors that are beyond what's possible to do with a regex. For example, replaing one function with another whose usage is subtly different. LLMs can crack out those changes across a codebase and it's easy to review.
- Summarizing and extracting intent from dense technical content. I'm often implementing some part of a web standard, and being able to quickly ask an AI questions, and have it give references back to specific parts of the standard is super useful.
- Bootstrapping tests. The hardest part of writing tests in my experience is writing the first one - there's always a bit of boilerplate that's annoying to write. The AI is pretty good at generating a smoke test and then you can use that a basis to do proper test coverage.
I'm sure there are more. The point is that it's currently incredibly valuable as a tool, but it's not yet capable of being an agent.