Why AI Generated Code Stops Being Maintainable

I stumbled upon the article The Black Box Problem: Why AI-Generated Code Stops Being Maintainable, which outlines pretty well the trap, in which most of the software engineering organizations quickly fall, once they start using AI.

The problem: AI gives you the (misleading) feeling how quick to production a feature could go. You have a specification, you as a set of AI agents to create the code, create the tests, test, and push. And that’s the ideal situation - in most cases you just ask one agent to produce code, that looks working. And then you push the code and you jump into the next feature. Everyone is happy. Today.

Rinse and repeat.

Until it comes the time to support it. Then you’re lost without AI. And then the AI reads its code, creates a fix, tests it, commits it. Done. “Yet another satisfied customer”.

Wrong. Maybe the given bug is fixed, but the code becomes more and more unmaintainable by a developer.

If your organization decided that this is the way they’d like to go - then fine! Go for it, take all the risks, and maybe a few years later see what’s at the bottom of the rabbit hole. Could be a treasure. Or could be just a massive pile of rabbit ********, which you will need to bring all to the top, before returning to maintainable product. Usually, that might mean you’ll need to reinvest massive hours, rewriting the code (with or without AI - org’s choice) - this time, in the right way.

I agree with the author, that the structure and the approach are critical for each system. If you lack such structured approach, you’ll be in trouble no matter if you use AI or not. With AI you’ll just get in trouble quicker, as it provides quicker “time to production” metric. The true question here is what kind of code quality goes to production - but that’s your concern, not AI’s.

AI amplifies. It amplifies time to production, it amplifies time to resolution. But if there’s no process and no structure, it amplifies the mess as well. I think we should all have that in mind, when we decide to implement it in our org. With AI “agent-engineers” the discipline, the processes and the tooling that will keep the mess away will become much more critical long-term than without AI.