How to put a number on your AI project, and know if it worked

Why put a number on an AI project?

Because measuring is one of the clearest habits of the businesses seeing results. In McKinsey’s research, the companies getting the most value from AI are twice as likely as others to have defined processes for measuring the impact of their AI work².

Without a number, nobody has agreed what success looks like, so projects drift. S&P Global found the share of companies abandoning most of their AI initiatives before they reach production rose from 17% to 42%, and the average organisation scrapped 46% of its proofs of concept³. Some of those should have been stopped. But a project with a clear number can be stopped early and cheaply, or kept with confidence.

Which KPI should you use to measure an AI project?

One you already track, that the task directly moves, plus one quality check. Good candidates are easy to count and matter to the business:

• Time: hours spent on quotes each week, time to prepare the monthly report.
• Money: days to get paid, cost to process an invoice.
• Customers: emails answered the same day, response time to enquiries.
• Quality: errors caught before they reach a customer, complaints, rework.

Why a quality check too? Because speed alone can hide problems. A good example comes from medicine: Lund University’s large breast-screening trial measured both sides. AI support cut radiologists’ screen-reading workload by 44%⁴, and the trial also tracked outcomes, finding 29% more cancers detected⁴ and 12% fewer cancers missed between screening rounds⁴. Measuring both speed and quality is what made the result convincing. Your version might be “hours spent on quotes” and “quotes sent back for corrections”.

How do you measure the starting point?

Write down today’s numbers for two to four weeks before you change anything. A simple spreadsheet is enough. Count how long the task takes on a normal week, how often it goes wrong, and how much it costs. If the task involves writing or answering, save a few dozen real examples with the answers you’d expect. They become your own test set, which you can also use to check any new AI model later.

Resist the temptation to start the AI tool first and measure afterwards. Once the work has changed, the “before” is gone, and every estimate becomes a guess.

How do you count time saved honestly?

Count the net result: the time saved minus the time spent checking and fixing the AI’s work. In a survey of more than 1,100 US enterprise AI users by Zapier, a company that sells automation software, 92% said AI boosts their productivity, yet the average employee spent 4.5 hours a week revising, correcting or redoing AI output⁵. The same survey found trained employees were 6x more likely to see productivity gains⁵.

Across the whole US workforce, the St. Louis Fed estimates generative AI time savings at 1.6% of all work hours so far⁶. Real, but modest when spread thinly. That’s why a specific task with a specific number beats a vague promise of “saving time everywhere”.

How do you work out the ROI of an AI project?

Value the time saved, set it against what the AI costs, and only count time that goes somewhere useful. The simple sum is the net hours saved each week, multiplied by the cost of that person’s time, minus what the AI tools and their upkeep cost you. But that value is only real if the hours move to work that earns or keeps customers: more quotes sent, faster replies, better service.

Cutting jobs isn’t a reliable shortcut either. From its survey of executives, Gartner concluded that “workforce reductions may create budget room, but they do not create return”⁷. In the US, 66% of businesses using AI use it only to help people with tasks⁸, and only 2% of firms reported AI-related job cuts⁸.

When should you review it, and what if the number doesn’t move?

Set the review date before you start, and be ready to keep, fix or stop. Check after a month of real use, then once a quarter. At each review, there are three honest answers:

• Keep it: the number moved and the quality held. Look for the next task.
• Fix it: the number moved a little, or quality slipped. Often the fix is training or changing the process, not a new tool.
• Stop it: nothing moved. Stopping is cheap when you decide on a date, and it frees time for a task that will pay off.