The Easiest Mistake AI Programming Makes: Not Writing Wrong Code, but Building Software from Unclear Requirements

Do you know what the easiest mistake AI programming makes is?
It is not writing wrong code, but rather –

you clearly have not explained the requirements, yet it earnestly builds the software for you anyway.

For example.
A manufacturing enterprise wants to develop an intelligent scheduling system and tells AI:

I want to improve equipment utilisation, reduce changeover frequency, ensure orders are delivered on time, and prioritise urgent orders when they appear.

Look, is there anything wrong with these requirements?
They all seem fine.
AI can completely generate pages, design databases, and even write a scheduling algorithm based on these requirements.

But the real trouble begins.

If, to reduce changeovers, orders with the same process are batched together, some orders may be delayed.If, to ensure all orders are delivered on time, production tasks may be switched frequently, increasing changeover costs.If an urgent order suddenly arrives, should it be prioritised, or should the already scheduled orders be protected from disruption?

At this point, several requirements may conflict with one another.

The problem is, the customer may tell you: all these requirements are important, just find a way to satisfy them all.
But in the real world, resources are limited, time is limited, and many objectives simply cannot be optimised simultaneously.

So how should AI choose?
It can propose different options and design algorithms based on the rules you give it.
But if you have not told it whether on-time delivery, equipment utilisation, or changeover cost has the higher priority; if you have not explicitly specified what an urgent order should interrupt and what it must not interrupt, then it can only make assumptions based on the available information.

And those assumptions may not align with the enterprise's true business strategy.

The most troublesome part is:

AI may write all these assumptions into the code; the system runs normally and the pages look beautiful.Only when it is truly put into production does the enterprise discover that the scheduling plan this system produces is not what it wanted at all.

01.Is the Problem with AI's Coding Ability?

At this point, is the problem with AI's coding ability?
Not necessarily.

Because there is a frequently overlooked fact in software development:

Requirements are not about recording what the customer says, but about clarifying what the customer truly wants to solve.

Of course, AI can also help analyse requirements, identify contradictions, raise clarifying questions, and even generate prototypes to make it easier for customers to express their ideas.
But it cannot decide, on behalf of the business owner, what should be sacrificed and what should be prioritised when business objectives conflict.

This requires the participation of people who understand the business, and it also requires people with real decision-making authority to make choices.

02.A Particularly Important Capability in the AI Programming Era

So in the AI programming era, a particularly important capability may not be how to write longer prompts, but rather:

how to ask the right questions,uncover the implicit rules,and clarify the conflicting requirements.

Because sometimes, it is not that AI has not done the job well.

But that it has completed, with great diligence, something that was never clearly defined from the very beginning.