eccher.
PT
AI in practice

AI doesn't fix what the team was already doing badly.

· · 5 min read
Translated from Portuguese. Read the original

Put an AI agent to work monitoring a team that wasn't clear on who decides what, and within three weeks you'll have a good-looking report pointing at the wrong place. The problem lies with the team rather than the model: the team had been winging it, and now that improvisation comes in dashboard form.

For at least two years, a promise has been going around that AI will handle the tedious part of management. It will read the tickets, summarize the meetings, flag where the project is slipping before the manager notices, and suggest what needs to be reprioritized. On well-structured teams, it does all of that. On poorly structured teams it does all of that too, but on top of the wrong structure. The polished output ends up more dangerous than the original mess, because it makes the problem look solved.

AI amplifies whatever process it observes, so it isn't neutral toward that process.

Well-formatted garbage is still garbage

The oldest rule of any information system is garbage in, garbage out. The rule still holds with AI, but the garbage now comes out with statistical confidence, a coherent narrative, and executive-ready formatting. It lands on a slide before anyone questions it.

Take an operations team that never clearly defined what counts as a resolved incident. One engineer closes the ticket when the symptom goes away, another closes it only after validating the root cause, and a third closes it because the customer stopped responding.

Without AI, this is a known inconsistency: someone complains, the team talks it through, and a shared definition eventually emerges.

With AI monitoring, it becomes a chart of average resolution time that blends three different definitions of the same word. Nobody notices, because the chart has axes, a legend, and a trend. The chart is accurate and represents exactly what's in the data. What's wrong is the premise that it means something.

The obvious objection is that this applies to any BI tool and has nothing specific to do with AI. That's only half true. In traditional BI, someone has to build the report, and the inconsistencies surface along the way. With agents that generate analyses on demand, the inconsistency gets swept under the rug: the agent answers the question it was asked using the data that exists, with neither the incentive nor the ability to question the definition of "resolved."

The management layer that used to sit between the raw data and the report, which is where the questioning happened, was removed, and the team itself removed it, thinking it was gaining speed.

Accountability doesn't scale automatically

One thing AI does well is distribute work: summarizing meetings, writing follow-ups, identifying who agreed to do what, sending reminders. All of that works. What doesn't work is the team assuming that automatic distribution replaces explicit accountability.

On a small team, everyone knows João handles the database side because that's how it's been for two years. Bring in an agent that takes every request and assigns it automatically based on declared capacity, history, and keywords in the ticket, and you get faster assignment but lose the unspoken agreement about who owns what. João keeps handling database work, but now he also handles cache work because the agent decided he had capacity. When something breaks in the cache next week, nobody knows whether João is responsible or whether that was a one-off assignment.

The failure comes from leaving the basic question of who is accountable for what unanswered before switching on the AI, rather than from the AI itself. Without that answer, the agent fills the vacuum with its own logic, which is local optimization, and the team finds out too late that its accountability structure has become a probability function.

A bad process with AI produces the illusion of a good process faster than a bad process without AI produces the realization that the process is bad. That's the real risk. It comes from AI speeding up the path to the point where nobody understands anymore why things are the way they are, and not from AI breaking what was working.

The right question is what you're automating

When a team decides to bring in AI to monitor operations, the conversation usually jumps straight to which tool, which model, which integration. The conversation that matters comes earlier: what exactly are you asking the AI to watch, and would watching it make sense without the AI?

If the answer is that the team needs AI because otherwise nobody can keep track of what's going on, that's a design problem rather than a tool problem. The operation is producing more signal than the team, as it's structured, can process, and the proposed fix is a layer that processes the signal without asking whether that much signal should exist in the first place.

The teams that benefit from AI monitoring already knew what was worth watching and were doing it by hand, at high cost. AI removes the cost. Teams without that clarity get an automated observer of things that may not matter, with the added bonus that they now have a dashboard.

The difference between the two scenarios lies in the question asked before the technology was switched on, rather than in the technology. And nobody outsources that question, not to a consultancy, not to an agent, not to a framework.

Tools amplify. The team decides what gets amplified, before the tool comes in. After that, it's too late.