Two measures that will justify any design change

We’ve known for some time that usability improvements can have a massive effect on the bottom line, so what stops usability professionals from actually measuring the return on investment of their work?

Is it because they think the benefit of a usability-induced change is obvious, so there’s no need to measure anything?

If so, then they’re taking a risk. The development team is unlikely to share their vision of "obviousness". The project manager may push back, claiming that the change is too difficult to implement, that it will need to wait for the next release, or that “user satisfaction” isn’t as important as selling more product.

Or is it that they can’t be bothered — perhaps because return on investment calculations aren’t the only way to sell usability to your manager.

Again, they are missing an opportunity. There is something uniquely persuasive about being able to prove that a design change can make or save the company money — especially in the current economic climate.

Or is it simply because people think the computations are too difficult?

In fact, using 2 common measures from usability tests, the calculations are easy enough to do with a paper and pencil.

When a 5% improvement in completion rate is worth £7 million

Success rate is the most important usability measure with an e-commerce site: how many users can complete the task?

Let’s say you carry out a large sample, benchmark usability test of an e-commerce web site. You find that 70% of your sample manages to complete the task successfully.

By making design changes to the web site, we should be able to improve this figure. Usability improvements invariably have a dramatic effect on improving success rates, but for argument’s sake, let’s assume we can improve the success rate by a measly 5%, making the new success rate 75%. Most usability professionals would bet their house that they could make at least that improvement.

How much is that 5% improvement worth?

We can do a simple calculation using this equation:

Sales = Success Rate x Potential Sales

“Sales” is the actual turnover of the e-commerce site. “Success Rate” is the figure we’re about to improve. “Potential Sales” is the amount of sales we would make if every user could complete the task.

At this point, we need to know the sales of the e-commerce site. Let’s assume it’s £100m per annum. (For most serious e-commerce outfits, this is a conservative figure. A high street store like John Lewis in Oxford Street takes £1million per day. And Jared Spool describes a design change to an e-commerce site that was worth $300m per annum).

So at the moment, 70% of people manage to checkout and the sales figure is £100m. In other words:

• £100,000,000 = 70% x Potential Sales

The Potential Sales is therefore £100,000,000/70% = £142,857,142.

We think we can increase the success rate to 75%, so:

• Sales = Success Rate x Potential Sales
• Sales = 75% x £142,857,142

Which comes to £107,142,857. The difference between this figure and our original £100m is over £7m. This is how much our 5% improvement in success rate is worth.

When improving time on task by 15 seconds is worth £1.3 million

An important measure for an intranet is time on task: how long do people take to complete a task?

A good example for an intranet would be looking up a colleague in the phone book. This is something that most intranet users do most days, often more than once per day. What would be the cost savings of reducing the amount of time people spent on this task?

Let’s assume that we’ve measured the time it takes an employee to find someone’s email address in the Intranet address book and begin an email message with that person’s name in the “To:” field. Assume it takes 60 seconds. We reckon we can reduce the time on task to 45 seconds by displaying a clickable email address alongside the employee’s name in the search results.

What’s the financial benefit of 15 seconds?

To do this calculation properly, we need to know how many times per day people carry out this task. Let’s make the conservative assumption that people do this task once per working day on average. If your organisation employs 100,000 people that do this task and the average loaded salary is £15/hour (again, a conservative figure), we can calculate that:

• Current time on task = 45 seconds = 45/60 minutes = 45 / (60 x 60) hours = 0.0125 hours
• Daily cost of task per employee = 0.0125 hours x £15/hour = £0.1875
• Daily cost of task for all employees = £0.1875 x 100,000 employees = £18,750
• Most people work an average of 220 days per year, so the yearly cost is = £18,750 x 220 days = £4,125,000.

Now let’s work out what the cost would be if we reduced the time from 45 seconds to 30 seconds. Working through the same calculations as above, we get:

• New time on task = 30 seconds = 30/60 minutes = 30 / (60 x 60) hours = 0.0083 hours
• Daily cost of task per employee = 0.0083 hours x £15 = £0.125
• Daily cost of task for all employees = £0.125 x 100,000 employees = £12,500
• The new yearly cost is = £12,500 x 220 days = £2,750,000.

The difference between the before and after figures is over £1.3 million. Not bad for a day’s work.

The easy way to make sure design changes happen

These kinds of calculation are very simple to make, so long as you have the kind of robust usability measures you get from a large sample, well-executed usabilty test. But when computing your own numbers, here's the golden rule: err on the conservative side. So for example, if you’re not sure if all employees use the Intranet phone book, reduce the number of users to a more persuasive value.

The fact is that virtually any improvement in usability will have a knock-on improvement in your organisation’s bottom line — but to make sure your voice is heard, you’ll need to collect the data and crunch the numbers.

Love it? Hate it? Join the discussion