At the end of a long day of working on a customer's website or application, how can we tell if we are achieving our goals? Sure, a site can look great, and have impressive functions, but until we measure whether they are accomplishing business objectives, we won't know whether it is a job well done!
In the book "The Lean Startup" by Eric Ries, the author encourages the continual measurement of the product with customers. In order to do this, you must identify hypotheses and carry out "experiments" to test them out (as early as possible). The business objective is essentially a key metric (or KPI) that the experiments are designed to measure. The hypotheses are based on research, surveys, etc. and are outcomes believed to lead to the successful achievement of the business objective.
So, how do we begin this journey of measuring user experience?
Step 1: Document the key business objectives and validate these with stakeholders. What does the business hope to achieve with the website?
Step 2: Identify the UX attributes that you think are critical to achieving the business objective. UX attributes can be found by use of research, surveys, examining competitor sites, etc.
Step 3: Identify activities and design work that can be done to improve the UX attributes. The International Standard of Usability defines usability as "the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use".
1. Effectiveness: can the user achieve specified goals? To measure effectiveness, you can look at:
- Success rate - % of users who correctly and completely achieve the goal unassisted.
- Disaster rate - % of users who think they were successful but failed.
- Number of errors per unit of time.
- % of tasks completed successfully on first try.
- Number of requests for assistance accomplishing task.
- Objective measure of quality of output.
2. Efficiency: How much work was required for the user to achieve specified goals?
- The average time taken to complete each task.
- Time taken on first attempt.
- Time spent relearning functions.
- Time to perform task compared to an expert.
- Time to achieve expert performance.
- Number of clicks taken to achieve task.
- Time correcting errors.
- % of time using manual.
3. Satisfaction: How does the user feel about the website or application, and their experience with it?
- Mean score using an established questionnaire.
- Ratio of + to - adjectives used to describe the website.
- % of customers who would recommend it to a friend.
- Customer rating of quality of output.
- % of customers that rate the website "easier to use" than a competitor's.
Step 4: Measure the benchmark state of each UX attribute and define it as unacceptable, minimum or on target.
Step 5: Track changes in each UX attribute until target values are achieved.
Step 6: Implement the change and test if the business objective is being met. A common test is "AB Testing" whereby random users are assigned and asked to use Site A or Site B. The Version B site will have one major change implemented (the one that is tested). If the business objectives were met more successfully with Version B, then you know that change should be implemented.
It's really important to continually measure 'user experience' before and after launching a website or application. Although it may look good to you, until you can prove to the business that it's accomplishing their goals, and unless you can satisfy the users who are on them, usability has not yet been achieved.