insightThere is a beautiful hypothesis that of all the touch-points a business has with its customers there are just a small handful that are Moment’s of Truth, essentially interactions that make or break the relationship (and in turn any future profits, customer loyalty, etc).

For example in case of your credit card company the moment of truth is when you call them to report your card lost. You are desperate and probably freaked out about charges someone could be making, and when you press 4 (or whatever) and the first question out of the operators mouth is: What is your credit card number? Or something equally silly. You just pressed the option to report a lost credit card, they should know better.

They probably just lost a customer, they definitely lost any good will and positive word of mouth that they had with you thus far.

For most online businesses a critical moment of truth with their customers is when the customer has a problem and they come to the company website for help. What you do from then on determines any future relationship you might have with the customer.

This post focuses on something all of us in “web analytics” struggle with: how to measure success of a Support website in delivering an optimal experience to customers in that moment of truth. 

Most of the tools we have, books that have been published and a the material we have access to focuses on ecommerce / selling. Support is a unique challenge and we struggle with it because “success” is harder to measure, success is not a page view or submit order button.

Here is my little contribution, a point of view if you will, to the discussion:

# 5 Don’t measure Unique or Total Visitors.

This is a bit extreme but on a support site we are not in the “demand generation” business, why should visitor counts be any part of success measurement? Your company makes crappy software / hardware and more people will come. Many corporate support websites are hard to find, so fewer people will come.

Not measuring Visitors is not just an advice to save time but it is more about changing mindsets. You don’t care how many visitors come to your website. What you care is how many of those your helped, for how many of them you delivered a positive moment of truth experience. We have to get this in our gut and ensure that everyone around us gets it as well.

We throw an expensive web analytics tool on a support website and we run the standard reports. Most standard analytics reports are quite sub-optimal for a support website and need to be thrown away, below I’ll talk about some that IMHO are helpful.

# 4 Identify Top Methods to Find Information.

Most support website owners toil away in vain spending time with path analysis (full disclosure, I am not a fan of traditional path analysis, for reasons click here). Most support sites sit on top of complex knowledge base systems and if you have bad navigation, or “ultra cool” navigation, and path analysis is instantly sub optimal because you influence the path so much.

cMy post on Path Analysis shows one way of how we can identify how people find relevant information on your website. The picture on the right is from that post.

We really need to understand what options our customers are using: Internal Search Engine, Top FAQ Pages List, Product Downloads list, links from forums, etc etc.  This is important because, as is usual on support websites, if most of your support content can only be found using your internal search engine and only 10% of the site traffic is using that search engine, it is guaranteed that your customers will be deeply unhappy.

Your company’s analytics tool might have other strategies that you can use to identify top methods that customers are using to find critical content on your website. If you are not satisfied with the methods your customers are using it is time to start testing and experimentation to try other methods.

# 3 Click Density Analysis of Top 10 – 20 FAQ’s (solutions / answers)

I use the term Click Density rather than Site Overlay because to me it communicates more effectively the purpose: what links have the highest density of clicks.

You probably know what the top problems are that your company is having. Either from your internal sources or simply from the top solutions to your product problems being viewed on your website (from your analytics package). Perform click density analysis on just the top 10 or 20 of those pages and identify if people are taking the action on that solutions page that you expect them to take.

[I am recommending top 10 or 20 both to avoid paralysis by analysis and also because on most support websites given the nature of the business, bugs tend to be concentrated, just a few pages will get 90% of the traffic. YMMV.]

If to fix the problem you want them to download a patch, are they doing that. If there are links on that page that take them to different solutions, are the customers able to find those links and are they clicking on them. What is the % of clicks on links below the fold (and are any of those links important).

If we find that customers are not doing what we expect them to do on the solutions page it is time to rethink the content, layout etc to make it more useful.

# 2 What percent of Site Visitors Call the Support Phone Number.

{start bitching} Most support websites do their best to hide the tech phone number deep in the site, clearly these companies don’t get the concept of Customer Satisfaction and Moment of Truth. In my book the only exception to this can be a company like Amazon who has excellent email support, they reply to support emails in less than 24 hours, very few fortune 500 companies have that record. {/start bitching}

Put a unique 800 phone number on your support website and measure the number of phone calls to that distinct phone number. My recommendation is to measure that call volume over time and if you are driving improvements to the website experience the phone calls should go down (as a % of traffic, ok so you should measure traffic for this one purpose : )).

Also measure successful resolutions in the phone channel for the callers who came from the website. The hypothesis is that even if the website was not able to eliminate the call perhaps it was good enough to provide the customers with the basic information that would make for a shorter phone call (and in turn some cost savings to your company).

# 1 Measure Problem Resolution, Timeliness & Likelihood to Recommend.

Most web analytics packages will be quite limited in their ability to provide insights on a support websites. That’s because the What is of such little importance, it is the Why that is of extreme importance (for background on these two concepts click here).

Implement a some kind of continuous measurement system for your website. You could do constant lab usability studies or follow me homes or surveys. The last one is something I recommend (ForeSee or iPerceptions). Using any of those qualitative methodologies I recommend measuring three metrics:

  • Problem Resolution Rate: What % of respondents were successfully able to resolve the problem they had?
  • Timeliness: What % were satisfied with the time it took them to resolve their problem?
  • Likelihood to Recommend: What % are so satisfied that they are likely to recommend your support website to others.

These three metrics measure the core essence of if you are delivering against the customer expectations for a moment of truth. You will be humbled by the first results but of all the recommendations above this one will drive the deepest insights and the work that will actually have a positive impact on customer experience.

(There might be a temptation that you could use clickstream data to get the first two bullets, there might be a detailed post in the future but I assure you that will be fundamentally flawed and only a actual customer qualitative response will suffice.)

There is a tendency not to trust surveys. In our experience survey response rate is on average seven percent (vs internet standard survey response rate of one percent), more than enough to get statistically significant results. That is both a reflection of the quality of the survey and of when and to whom the survey is served up (using behavior targeting). You can complement surveys with other methodologies described above, Follow Me Homes or Lab Usability etc.

Of course one final bonus recommendation for you…..

# 0 Correlate Your Top Solutions Viewed with Top Problems Identified “Off-site”

Ask for the top ten product problems being reported in your phone call center and check if solutions to those problems are the top ten pages on your site. It is possible that customers call and come to the website for different problems, but if the solutions to the top ten call center problems are not top on your website it is highly likely that your site is failing your customers and you need to take immediate action.

Do the same for top problems people are talking about in User Forums (on your site or others) and in the great blogosphere.

Doing analytics on a support website is extremely difficult because we are dealing with irrationality. People are stressed, our websites are not optimal and there is no patience in getting answers. Standard Analytics tools can only go so far. I deeply recommend that 70% of your analysis should be qualitative (as the question: “If you were not able to solve your problem on our site, what was your problem?”) and 30% clickstream (quantitative).

Agree with the recommendations above? Disagree? Are there alternative strategies that have worked for you better? Please share your feedback via comments.

[Like this post? For more posts like this please click here.]

Social Bookmarks:

  • services sprite
  • services sprite
  • services sprite
  • services sprite
  • services sprite
  • services sprite
  • services sprite
  • services sprite
  • services sprite
  • services sprite
  • services sprite