Please don’t use a vibe-coded app for anything important.
I use Claude. I like Claude. But I’ve backed away from having Claude actually write my code other than in the most limited circumstances.
I caught it copying one of my TS Interfaces, for example. And modifying, then using, the copy. So my type-checks pass, yay! But wait what?
It wrote a test for a tricky bit of code. The test wouldn’t pass. So it re-wrote it in a way that couldn’t possibly fail, mocking all elements inside the test itself.
I’m not anti-AI. But I wouldn’t trust anything vibe-coded above the importance of, say, Wordle.
Copilot is also WAY too eager to pass tests. It will write tests to pass a completely broken function.
We have aggressively high coverage requirements at work, but also really terrible tooling/support for good tests, so everybody uses copilot. The result is a paper-thin test suite that only tests the implementation and none of the intended behavior. It’s so clunky and the tests are so mock-ridden that the only way to make sense of them… is by using copilot!! Thus it continues ;-;
Review and good rules are still critical. Current agent state is still hyperspeed junior engineer. Providing examples helps a lot when scaffolding something similar to something else.
You need to develop an intuition to it, the same kind of intuition you developed for systems we work with. For example, your test issue: start by writing tests before making the implementation. LLMs are quite capable but they are not AGI. If your pipeline is good they can produce solid results.
I use Claude. I like Claude. But I’ve backed away from having Claude actually write my code other than in the most limited circumstances.
I caught it copying one of my TS Interfaces, for example. And modifying, then using, the copy. So my type-checks pass, yay! But wait what?
It wrote a test for a tricky bit of code. The test wouldn’t pass. So it re-wrote it in a way that couldn’t possibly fail, mocking all elements inside the test itself.
I’m not anti-AI. But I wouldn’t trust anything vibe-coded above the importance of, say, Wordle.