Still counting logins?

How it shows up

Four things that have probably happened at your place.

Tick whatever you recognise. Nothing gets scored, nothing gets sent. It is a faster way to find out where you stand than reading three more paragraphs.

• The number that went to finance was the number of active users, because it was the number the dashboard produced.
• Everyone says it saves them hours. The time it takes to get something to a customer is the same as last year.
• The licences were renewed in one line of an email. Nobody evaluated the year before it.
• The success story in the all-hands is one team, told well. Nobody knows whether it works anywhere else.

Three or four of those and you are in the normal case, which is worth saying out loud, because the version of this conversation where somebody looks careless is the version that goes nowhere.

In plain terms

Start with the number everyone quotes at you.

If you have sat in a meeting about AI in the last year, somebody has said that 95% of AI projects fail. It gets used to stop a budget and to justify one, depending on who is talking.

That number comes from a report by MIT's Project NANDA, published in July 2025. Behind it sit 52 interviews with executives, 153 survey responses and a review of about 300 public projects. Twenty-six pages, marked by the authors as preliminary findings, never peer reviewed. Success was defined narrowly as direct revenue or profit impact, so a project that cut costs or kept customers counted as a failure.2

I am not saying the number is wrong. I have no idea, and neither does anyone quoting it. That is the point. The most repeated fact about whether AI pays off is itself barely measured, and it filled a hole that nothing better was filling.

There is one thing in that same MIT report worth keeping, and it explains a lot about where AI money goes. More than half of AI budgets land in sales and marketing, because those functions already have dashboards and their numbers go on a board slide without much work. The back office gets less. The report saw better returns there, and proving it takes work nobody has time for.2

So spending follows what is easy to count. Not what works. That is the whole challenge in one sentence, and it applies to your company as much as to the ones in the report.

Why nobody notices

Counting logins was easy, and easy became the measure.

The tool hands you a dashboard, so you use it

Every vendor ships usage numbers. Seats, logins, prompts, minutes. It arrives free, it looks like measurement, and it goes straight into the quarterly update. Nobody chose to measure activity. It was simply the thing that came in the box, and building anything else takes a week that nobody has.

Nobody wrote down what success would look like

Ask what the goal was for your biggest AI investment, in business terms, from before it started. In most companies that sentence was never written. Then afterwards you are looking for evidence of something you never defined, which is a game you lose whatever the tool did.

Everybody is a little afraid of the answer

This one gets said to me privately and never in the meeting. The person who pushed for AI does not want a number that says it did nothing. The team that uses it every day does not want their licence taken away. Finance can wait another quarter. So the honest measurement keeps not getting built, and everybody is being perfectly reasonable about their own part.

The way out of that is to measure the workflow rather than the tool. Nobody feels judged when you time how long an order takes to get to a customer, and it tells you more than any survey about hours saved.

Where it bites

The companies that measure are the ones furthest along.

We read companies on five levels, from everyone doing their own thing to AI being part of how the place runs. Here is the whole ladder, with how many companies sit on each step.

Level 4, Engine, is where AI sits inside how the company runs rather than next to it. Things that are part of how a company runs get measured, the way you measure a production line or a sales team. Things that sit next to it get a usage report instead.

At Level 2, Wild West, everyone works their own way, so there is nothing shared to measure. At Level 3, Blueprint, the method is written down and you can at least count who follows it. At Level 4 somebody owns a business number that AI is supposed to move, and has to explain that number every quarter. About one company in twenty-five is there, which sits close to the 6% McKinsey found getting real financial results.13

Worth saying, because it is a fair objection. Measuring will not by itself move you up the ladder. What it does is stop you spending another year guessing. Every client I have watched climb a step started with somebody putting an honest number on a wall, and that first number was usually disappointing.

What a measurement per person adds

An Operating Profile in use. Personality type and AI level in one profile, with the agents that fit it.

A usage report tells you a team used the tool 400 times last month. It cannot tell you whether those 400 were four people doing serious work or forty people asking it to shorten an email. Those two need completely different next steps, and one licence report looks the same for both.

It gets measured one person at a time: how somebody thinks and works, and their AI level. You get a picture of your company at person level, which is what tells you where to put the next euro. Usually that turns out to be training two teams rather than buying eighty more seats.

Fifteen minutes per person. No credit card, no strings.

The cost of waiting

Renewal season arrives and there is nothing to show.

The first cost is money going to the wrong place. More than half of AI budgets land where the dashboards already exist, which is marketing and sales, while the back office quietly pays back better.2 Nobody decided that. It happened because one side could show a chart and the other side could not.

Then there is the year you lose. Twenty-eight percent of companies now put more than a tenth of their IT budget into AI, and 60% plan to spend more next year.1 Spend that without a number and you are repeating last year at a larger scale. Meanwhile a few of your competitors are moving money towards the things that worked.

The one that hurts most is what happens to the argument inside your company. Without evidence it becomes a matter of belief, and belief arguments get won by whoever is loudest or most senior. I have watched a good AI programme get cut because the person defending it had adoption numbers and the person attacking it had a story about a bad output. A dull number about handling time would have ended that in a minute.

Where this goes next

Once you can see it, three other things come into view.

These come up in the same quarter, usually in the same meeting.