Josh On Design

If you live on planet earth and you are at all connected to software you've probably tried out 'vibe coding', ie: using an AI tool to write software for you. As someone who's been writing software for close to four decades I've seen code automation tools come and go. Believe it or not, people used to talk about Visual Basic in the same breathless tones of today's vibe coding tools. The outcome has always been that the new tech never lives up to the hype but can be useful in some contexts. The tricky part is always figuring out what it's good at and what it isn't. I believe these AI tools will go the same way; but hey, maybe this time it's different. I can't talk about what I haven't experienced, so I tried experiments this week. Here's what I learned.

These tools work better on new code than existing code bases. I can describe a new application in prose and get a pretty good prototype.

Significantly struggled to fix overscroll issues with this webbased music player. To be fair, plenty of human coders struggle with this as well. When I build such layouts by hand I have to plan ahead and stick to strict rules. I'm not surprised the AI failed as well.

It is good at things with a well defined answer. If something can be checked by a unit test then Codex can easily write and verify it. UI is harder because it's open ended and requires a lot of intuition, so it does far less well at.

Working with Codex essentially turned me into a product manager. I would ask for something, check the results, make suggestions, and repeat. I was able to build a web -based itunes like app very quickly, but I spent hours on the little details. At some point it would have become faster for me to make the changes myself. That said, describing exactly what I wanted in prose forced me to really think throiugh my goals and be explicit. I would not have had to describe to a human programmer (who had some UI building experience) that "When jumping to an artist by typing in the name the list should scroll so that the newly selected item is in the viewport.", but I did with Codex. I have to carefully use the UI to find issues. It was refereshing, however, to be able to act as just a product manager and not also the programmer when working on personal projects (as this experiment is).

I think that AI coders, at least for UI work, care free up the human from drugery, but still require close monitoring. I definitely worry about the quality of the generated code. It's not bad, but it is very rote and can be hard to follow. It looks like what I'd expect of a junior engineer.