March 2009


25 Mar 2009 01:52 am

morningWhen in doubt, ask your customers to help you.

I am quite fond of saying that in the context of using methodologies like Surveys and Experimentation and Testing in service of improving web experiences.

Today I used that idea in a different context, ask you what was top of your mind so I could try and answer some of them. My question was very open ended.

Here's my simple tweet :

twitter avinashkaushik

I got a boat load of questions on my twitter and facebook accounts. Some were a delight, others were like "what?" :), and there were some that made me pull my hair out. [Is that social media in a nutshell? :)]

In this post I'll cover all the questions I got on Twitter. I'll answer the ones from Facebook in the next post (this one got too long!).

But before I go on one important point of emphasis.

It's all about Outcomes baby!

A lot of questions fell into this bucket "what should I do" / "where should I start" / "how can I convince xyz of something". I am going to answer all those questions with this: Focus on what the Outcomes are that you / your company is driving towards.

I know it is the lamest thing someone could say, it even sounds like a cop out. It is not. This game is all about Outcomes!

Let me go out on a limb and say there are only three types of Outcomes any website delivers (except of corner cases – all you anal retentive folks take two steps back!):

  1. Increase Revenue
  2. Reduce Cost
  3. Improve Customer Loyalty / Satisfaction

That's it.

three eggsWhen in doubt ask your self if what you are doing falls into one of those three buckets. If it does, keep going. If not then I suggest you revisit what you are doing.

I know there might be others Outcomes, but for most "business" sites it will be about those three and their derivatives. [Business sites could be for-profits or non-profits, ecommerce or non-ecommerce, blogs or tech support etc.]

Let's get this puppy on the road.

A selection of your questions on Twitter and my answers. . . .

@JudithLewis : How do you convince people to look beyond page impressions for usable measurable metrics?

I am not trying to be glib here, but you have to show them how pathetic those kinds of metrics are. If your HiPPO's quite large then replace pathetic with primitive. : )

Recently I have started saying that the only metrics that truly help find actionable insights are those that measure / reflect customer behavior.

Page impressions is a "aggregate metric" (like Visits or Unique Visitors or Page Views). They are all great to know the What. But they are not truly reflective of customer behavior.

Examples of customer behavior metrics are Visitor Loyalty, Visitor Recency or Distributions of Depth and Length of Visits etc. All things Visitors actually do on your site.

impressions hand prints

So the first time give them Impressions.

The next time be an Analysis Ninja and give them a little extra analysis focused on something related to outcomes ("we had these many page impressions but Visitors that came from source x had more impressions" or "I dug a bit deeper and found at the first page impression was key in getting people not to bounce and see more impressions" etc etc etc).

Change the world, one small step at a time.

@mthinker : How do we account for the one viewer who has more than one machine? for the user who is selectively blocking scripts?

Short answer: You can't.

Long answer: You can't. But….

If you have a non login, non spyware site like say this blog then "people" are transparent to you/me. I use cookies (see my privacy policy) and as you switch computers (or browsers on the same computers) you are a different Unique Visitor to me.

If you have a login site, like say www.turbotax.com (buy tax software from them!), where people are not logged in by default, then it is a bit difficult for you to "stitch" their behavior into one piece. But if you extract data our of your analytics tool or switch your unique tracking to something like the login id (that you have stored in a cookie or pass to your analytics tool in some other way) then for those that do log in you can "stitch behavior".

major operation

If you have a login site, like say www.amazon.com, where anyone who has shopped there is logged in my default then that makes it easier to "stitch behavior".

To the last part of your question, if they are blocking scripts then you are out of luck with your javascript tag based solutions (Google Analytics, Omniture, IndexTools, WebTrends etc). But if you use log based solutions (Urchin, WebTrends etc) then you can still track these folks. Expect a minor nightmare trying to "merge" the two sources together.

Important. Do ask yourself this question: What is the ROI for perfection? or Do I really need 100% "accuracy" to do my job and create a great site?

There is more data on the web than on any other channel (TV, Radio, Magazines etc). It is a lot more and with a lot more confidence (just dig one inch below the surface of how TV is measured). Even with no cookies, even with cookie deletion, even with people switching computers you can make decisions and improve your site and add to your bottomline.

Don't get conned into chasing your tail (and let me stress I am not saying that you in particular are!). Leave that to Consultants, Gurus, Bloggers (me!).

Bonus reading: Perfection Is Dead, Long Live Perfection

@seanpower : How do you define and measure ROI in social media? SM is great, but when is active participation enough or overkill?

Outcomes baby! : )

I have to admit that I think that any company should participate in Social Media simply because it is important to be where your customers are, it is important to be a genuine part of the conversation.

It does not cost much (just buy Red Bull for the knowledgeable 23 yr old in your company). It can yield long term ROI.

With that in mind the most important measure of ROI is to know that you are actually participating in the conversation and (more importantly) creating conversation.

feed subscribers occams razor

So for blogs it is measuring RSS Subscribers (people want to pull your messages, yea!), Ripple Index (using technorati "authority" until something better comes along, no not Google Blog Search!) and Conversation Rate.

More here: Blog Metrics: Six Recommendations For Measuring Your Success

For Twitter there are such measures as well. To me twitter is a great channel to magnify your message (if it is of value). So I use retweetrank.

So is IntelSoftware doing better than avinashkaushik? No. How about TXInstruments? Yes!

It's a simple example of Outcomes (magnify the message), but should give you a feel for how to move forward.

@mlvalentine : Most of my WA challenges are w/ explaining how it works and why it won't add up like a balance sheet to non-WAs.

I wish I had a good answer, I don't. It is pretty hard.

tough luckWhat is worse is that we live in a world where most Sr. Management / Decision Makers (and many Web Analysts!) have grown up in a world where two plus two equals four. Not three point five or four point seven.

It is really hard to change their minds, or get them to believe that anything that does not add up is still worth a lot.

In my career I have always tried to show that the data collection is as rigorous as possible (clean tags, first party cookies etc) and then switch to showing value from what is available.

For example even with 90% confidence it is hard to argue with actionability of most web analytics data. Or even with 70% confidence. We just have so dammed much and there is so much in there of value.

There is a list of why data won't be clean in this post: Data Quality Sucks, Let’s Just Get Over It

It also contains a six step plan that you can execute to win your lovely HiPPO's over.

@nectarios : How do you convince c-level execs of the power of web analytics and more importantly to invest in the human power necessary?

You are in luck. I have a specific post that covers just this:

Here are the five specific recommendations:

# 1: Implement a Experimentation & Testing Program.

# 2: Capture Voice of Customer. Surveys, Remote Usability, Whatever.

# 3: Deploy the Benchmarks I Say, Deploy ‘em Now!

# 4: Competitive Intelligence is Your New Best Friend.

# 5: Hijack a Friendly Website (/ Earn Your Right to be Heard).

Check out the details in the post.

@midlakewinter : For a blog & corporate site on same domain. Measure together or as separate entities?

To a great extent it depends on your company, the types of customers you are targeting and their technical sophistication. So do please consider that.

But.

Based on my experience in analyzing a bunch of these cases I have come to realize that it is often optimal to measure them as separate entities because they tend to solve two different problems.

A good example is Bonobos. The site is at http://www.bonobos.com/ and the blog is at http://www.bonobos.com/blog/

bonobos

I had analyzed their site last year for Inc Magazine and one of the things the data showed was that the audience that came to the blog would buy some pants from time to time but mostly they were there for the community and learning what the company was up to etc. So high bounces (which is ok in this case) etc.

On the other hand the "corp" site was there as the main store front, to attract and covert new visitors, to sell etc. While some of them would read the blog and perhaps be convinced to buy the pants, that was a small number.

The content consumption pattern (and Primary Purpose!) of each audience was different so it would have been optimal to measure and report separately.

Just as a small example high bounce rate in one case would be ok, but not in the other. So you could understand data better.

I have seen this frequently. Hope this helps you with a framework of how to think of deciding in your case.

[Full disclosure: I did not get a pair of pants for doing the bonobos analysis. Though it would not have hurt my feelings if I had gotten one! :)]

@brandingme : Can i pass a techie question by you?!?!?! see explanation here: http://bit.ly/h70XO

I'll summarize Adrian's issue.

He was using this to track a link to a flash app:

http://www.example.com/#/register?utm_source=source&utm_medium=medium&utm_campaign=campaign

The # does not work optimally.

So he moved to this:

http://www.example.com/?utm_source=source&utm_medium=medium&utm_campaign=campaign/#/register

gerbera daisyThe problem is that it works. Ok that is not his problem. :) His problem is that he does not like the url stem to stay, its "ugly". [Did I get that right Adrian?]

Here is what can be done in this case.

You can still use the first url stem /#/register? but you'll use a special Google Analytics function called setAllowAnchor.

Here's more information on exactly how to use setAllowAnchor.

Then birds will sing, the sun will shine and life will be a bed of roses. :)

[I want to thank my buddy Nick for helping me with this answer.]

@Omni_man : I'd be interested in best practices of connecting online browsing to offline success when there is little to connect the two.

Adam is absolutely right, with the missing Primary Key to tie the two data sets it is really hard to connect online (anonymous) visitors to, say, store visits.

If people say they can do "holistic view analysis" or "multichannel magnificence" easily then view them with suspicion. Please. Especially if they are selling you a tool. Pretty please.

There are some cases where the Primary Key exists and this is possible. Two good examples are www.walmart.com and www.safeway.com where online Visitors and, surely, online Purchasers are encouraged to use their Store Card (safeway) or pick up in a store (walmart). Both of those things with some difficulty allow you to merge the data and get a offline to online view.

Remember it would still happen outside your web analtyics tool, usually in a corporate data warehouse.

So what about the rest of us?

hope 11

You can follow seven different strategies to measure Multi Channel impact. Each will give you a better idea of a website's incremental offline impact (even if not as direct as we discussed above).

Here's the post: Tracking Offline Conversions: Hope, Seven Best Practices, Bonus Tips

[Dear Readers: If you use Omniture I highly recommend Adam's blog, it is very good.]

@jarmes : I want analytics data justifying the removal of main navigation :-)

I commend you on your bravery. :)

You have only one option: Test!

More here:

Let me know how it turns out.

@ncabane : My juicest. mundane daily challenge is to attract people to my blog and to the company i work for, using analytics.

Use tools like Insights for Search to identify related keywords and optimize your paid search and organic efforts to attract a wider repertoire of searchers.

Absolutely positively have a search long tail strategy, that's were the "impression virgins" are.

Use tools like www.compete.com to do a diff between your referrers and your competitors referrers. Then steal them!

Its the Referral Analytics report, looks like this (comparing www.intel.com and www.amd.com):

compete referral analytics

Seriously though far too often we focus on our own web analytics data, Google Analtyics and Omniture and WebTrends and what not. What you want is to look outside, that's were joy exists.

@Edw3rd : A hard question is determining salability (predictability) of a particular approach or campaign…success is historical

This is the problem with twitter, 140 characters are simply not enough.

So Edward I don't really understand your question. But let me try. . .

With regards to campaigns it is important to realize there are three problems that stymie your ability to replicate success or, importantly, scale it.

1) Diminishing Margins of Return. You will surely hit this, all the time. So just because this campaign worked so well from 100 to 10,000 users, you can bet every use after that will be really hard.

2) Inventory. Most of us don't realize there is a finite inventory out there. For people who search for something. For people who want to buy something from you. For your raw ability to keep succeeding. These are usually hard to predict.

3) You got lucky. There is no other channel quite as complex as the web, in terms of the number of variables. It is also friction less. So your past success for a campaign, or a series of campaigns could be 100% luck. It could be because of things your competitor did. Or that it was your lucky day and you got on the front page of www.digg.com . Or….

Keep those three things in mind. The only way out? You try, you live, you learn. Remember the greatest gift the web gives you is the ability to fail faster (cheaply).

Hope I am a bit close to what you wanted to know.

Bonus Read: Data Mining And Predictive Analytics On Web Data Works?

newspapers media

@tonysingh : What are the key measurements for media sites?

I am sure you know all the ones related to ad sales etc etc.

Let me recommend two more: Visitor Loyalty and Visitor Recency .

I fundamentally believe that many media sites focus first on making money and secondarily on creating loyal customers. It should be reverse.

If you want my humble prescription for media sites (publishers) then here it is:

@airsibel : Missing or inaccurate data that does not align with database figures. Perfect data in web analytics, is it an Utopia?

Sadly it is difficult to achieve, its just the nature of the beast. For now.

Things will change.

But this does not mean you can't take action. The first post has a six step action plan you can follow, the second shares my view of the value of perfection:

@itwop : What insight/action does my competitor get out of his analytics that I'm overlooking/ignoring?

The only way to find out is call her! :)

@abba_dad : What did my customers do before/after/between visits to my website?

For before you'll get some clues in your referrers, keywords, etc data.

For after use the Destinations report in Referral Analytics in Compete.

For between visits, call them (your customers)!

zen sandbox

@pere_rovira : What reports should i look at today and why?

The best person to answer that question is you.

If I had to recommend one simply based on what is in that tweet, I would say focus on reports that measure one or all three of the Outcomes mentioned at the start of this post.

Because that will ensure you'll show value of your job / consulting contract.

@wuterence : Where Does the Everyday Marketing Manager Start????

I have the most perfect blog post for you:

Hurray, all done!

I hope you had fun and found the answers to be relevant and meaningful to you in some small way, even if you did not ask the question.

Ok now your turn.

Would you have answered these questions as I did? What would you have done differently? Please add your own thoughts.

Thanks.

10 Mar 2009 12:55 am

centerWe are constantly on a quest to conquer the next big thing. Mountain. Ocean. Planet. "Conversion Buster." The next million dollar opportunity.

Not that there's anything fundamentally wrong with that quest.

The challenge is that frequently in that quest we ignore the immediately achievable. And that tradeoff is a crime.

The title of this post comes from a blog post that my friend David Hughes had written in September 2008: What can Digital Marketers learn from Olympic Cyclists? He started that post with this wonderful quote:

Back in the 1980's Jan Carlzon was trying to breathe new life into an ailing Scandinavian Air Services. He was famous for saying "You cannot improve one thing by 1000% but you can improve 1000 little things by 1%".

Aggregation of marginal gains!

Fantastic concept, loved it instantly, captured my heart.

I have mentioned this issue in the past, though never quite as eloquently. Here's my humble tweet from Aug 2008:

aggregration of marginal gains

This post is all about the low hanging fruits. Small and medium sized ideas for finding opportunities that collectively should add up to something remarkable for your website.

This being the recession and what not, please share your own stories of simple every day things you do to find actionable insights for your company. Together we can! : )

So before your go boiling the ocean here are a few things you can and should do today to ensure your company is benefiting from immediately fixable things:

1) Figure out where you are making money, where you ain't.
2) Cover all the bases in your email campaigns.
3) Funnels baby!
4) Stop The "Puking": Fix Your Top Landing / Entry Pages.
5) Identify Paid Search Keyword Opportunities.
6) When In Doubt, Ask Your Customers.
7) Stop Doing "Dumb" Things. (Examples included!)

Batten down the hatches, let's deep dive. . . .

1) Figure out where you are making money, where you ain't.

I have covered this in the past: Pick One, Just One Web Analytics Report, Go!

greatest web analytics report on earth1

Why is it great?

You come to work to: 1) Increase Revenue 2) Reduce Costs 3) Improve Customer Satisfaction (create Brand Evangelists).

This analysis focuses on the first two of those things.

It helps you identify which sources are working well (and hence need more love, resulting in increased revenue.

It will help you identify sources that might be sucking wind, and help you reduce cost.

Possible outcomes?

Focus in your advertising, sales, marketing spend. Identification of valuable sources that might have been below the radar so far (#2 or #10 for me above).

2) Cover all the bases in your email campaigns.

This one comes from David's post.

All of us run email campaigns, most of us professionally (and we live in Russia and sell spicy things!). But have you covered all the simple basic things you should doing to maximize value?

improving email campaigns

He recommends that by fixing five simple things he was able to really crank up the conversion rate for a seminar, by 486%.

[Of course having penned the Four Not Useful KPI Measurement Techniques post I hasten to add that improvement was from 7 to 41. Now before you roll your eyes consider this, would you not kill to have a 20% improvement in your email campaigns?]

Why is it great?

It is achievable.

Each thing he recommends is quite simple. For example fix the Hard Bounce Rate by using email repair software, or Soft Bounce Rate by identifying spam blocks, or Open Rate by testing subject fields (or, my fav, optimizing for the Preview Pane), or creating illusions of personalization for Click Thru Rates, or continuing the "scent" to landing pages to improving conversion.

Small things that add up to a lot.

Possible outcomes?

Email continues to be one of the most lucrative channels for most businesses (even spammers!). Doing things above means you make a better connection with your customers (that itself is worth it) and improve Revenue.

3) Funnels baby!

I rarely talk about funnels (specifically cart/checkout or credit card application or lead submission). That's because I think everyone's doing 'em.

But it turns out I am wrong.

A small fraction of the people use funnels and that's a real shame because you can find actionable things to fix pretty fast.

So if you are Chase and have a multi step credit card application:

chase credit card application

Or you are Burger King and have multiple steps for new franchises:

burger king franchise

Or you are Caterpillar trying to "unload" a whole bunch of Backhoe Loaders:

caterpillar backhoe loader pricing

You need a funnel report in your web analytics tool! Demand / command that you have a funnel created for each structured experience on your site. Ok beg if that is what it takes.

analytics funnel visualization

They look so pretty too don't they?

Why is it great?

For structured experiences funnels can be awesomely actionable. They can identify problems quickly, is 73% abandonment on step one acceptable (seems a bit harsh I think) or how about 75% on step two.

And talk about low hanging fruit. At the end of the day you are dealing with one page. One. Uno. Ek.

You know why that page exists. You know where people come to it from (look at the box on the left). You know where people exit to (look on the right). Not that hard to come up with two or three simple fixes to put in place (or A/B tests to get going with the Google Website Optimizer!).

Possible outcomes?

Most structured experience on sites are money making / saving opportunities. Shopping Carts (or Baskets to my British friends :)). Lead requests. Applications. What not.

clicktracks funnel report You will rarely get this close to having a direct impact on the bottom line.

Initially you'll be ok without segmentation but later you'll learn that segmentation is your BFF. It is hard to do this with Google Analytics (you can apply the filters and profiles magic) but it is pretty easy to do it in a tool like ClickTracks . Do it.

4) Stop The "Puking": Fix Your Top Landing / Entry Pages.

Another eternal favorite of mine when I want to get some quick wins under my belt. It is one of the first things I look at when I analyze any website.

Most web analytics tools have a standard reports that show the Top Landing Pages by Bounce Rate. Go get it.

Here's how it looks in ClickTracks (column: Short Visits):

clicktracks data dissection report

Yahoo! Web Analytics (IndexTools) does not have a standard report but click on Customize Report and in two seconds you have:

yahoo web analytics bounce rate top landing pages

Note what I am doing with the stars: looking for landing pages which really high bounce rates. That's what you'll do on your site, find the pages that are "losers" who can't even earn one click.

[Sidebar: if you look at bounce rates for blogs remember it is not the best metric, unless you segment out new visitors in which case it is still helpful. Read how to analyze bounce rates for more.]

Why is it great?

Like we did before, you are looking at high value individual pages. Lots of traffic, lots of bouncing.

You already know why the page exists (just read it!). And when you look at these pages it is easy to get to the thing you covet the most: Customer Intent!

Check two important things (both available on the same report): External sources referring traffic to this page ("why?"). External keywords referring traffic to this page ("what do they want?").

individual website page analysis

But give you a quick understanding of what the mismatch might be between your intent on the page (sell sexy pants) and what the customer wants (v1agra!).

If you want to go one step deeper go to your internal site search report and look for keywords people in your internal search engine on that page.

So on your sexy pants page people are searching for sexy pants something's wrong with the page.

Possible outcomes?

Increased possibility of conversions / leads / watching videos / whatever the purpose of your main pages are.

And remember these are your "head" pages, small improvements yield high ROI.

5) Identify Paid Search Keyword Opportunities.

Search, organic or paid or both, tends to be a corner stone of the acquisition strategy for many companies. Lots of gains can be gotten from wise investments in search.

search metrics

One beautiful actionable idea for you.

Do a diff between your paid and organic search keywords and look for anomalies.

paid search traffic keywords

Keywords that have a better conversion / time on site / bounce rate / your fav KPI under the Organic bucket when compared to their performance in the Paid search bucket.

Why is it great?

If you find keyword that perform better when they bring Organic Search traffic then investigate the landing pages and see what is better / different about them then the Paid Search landing pages for that traffic.

wrong or right direction 1Compare the organic listing in Google (ok, ok, or in Live or Yahoo! :) with the Paid Search ad. Is there any different (better) about the text and call to action in the Organic listing?

In both cases learn from Organic and fix in Paid. If the Paid performance is truly sucky then rethink PPC for those keywords.

Do the reverse for Paid Search. Look at the above report and see where are places where your PPC campaigns are doing better. Why? What can you learn and implement on the Organic results pages?

Often there are lots of learnings from the Paid campaigns because your landing pages are much simpler and more targeted than your Organic pages. Take the lessons and apply them to the other.

Possible outcomes?

Search is a complex and tough thing to do right. Covering the basics above means that your company will start to be a learning machine where you are not discarding any scraps of information you are collecting on your site.

At the minimum you'll improve conversion rates. With just a little effort you'll also reduce cost of Paid Search.

6) When In Doubt, Ask Your Customers.

How many times have you heard me talk about limitations of ClickStream data?

Far too often.

One more time: You can try to torture your data and guess. But why not ask your customers?

Use the free 4Q site level survey to get this key information:

task completion rates

"Dear Visitor, why are you here and pray tell us all the ways in which we have disappointed you?"

See? No guessing. Fix those things in Orange and you have a leg up on your competition.

The team keeps pouring more love into improving 4Q. Now you can add these two advanced segmentation questions:

survey segmentation questions

More insights into Customer Intent! The nice thing is you can get some offline signals as well ("came from a tv ad", "your radio campaigns were sweet" etc).

4Q's for site level. If you want to improve individual pages then I recommend getting a page level survey to collect tactical can be quickly fixed information. Check out Kampyle and GetSatisfaction , both are wonderful.

Why is it great?

Why rely on your opinions and then fall flat on your face? Why try to argue with a HiPPO and let her decide what's best for your customers?

By getting your customers involved means you magnificently increase the chances that you'll get it right. More often.

Possible outcomes?

You will certainly increase Outcomes (conversions / leads / movie tickets / electric shocks to sub par customers, sorry I meant employees!).

But what is cooler is that you'll focus on the 98% that will never convert and you'll improve their task completion and satisfaction. So more offline conversions. More micro conversions! [See this post: Measure Macro AND Micro Conversions .]

7) Stop Doing "Dumb" Things.

Pardon my french, and please know I use that word with love.

It is hard to believe that in 2009 we still have websites that ask us every time we visit to pick a country (www.burgerking.com), even if I visit an hour later! How lame is that (!!). Or that have pages that look like this:

content website with no content

That is a page from a major content site where I have covered the navigation elements in blue and the ads in black. 8 ads.

In white is the actual content delivered to me. At 1440 screen resolution!

Astonishing right?

I call it the "selfish lover" strategy. Just think of yourself and what you can get out of it rather than the other person (customer!).

Have you ever loved a selfish lover back?

Why do you think it would work on your website?

If you have Web Analytics: An Hour A Day then on Page 59 you'll find a Heuristic Evaluation checklist. Basic things you should do to make sure your have a good website experience.

If you don't have my book, no worries, let me recommend my friend Dr. Pete Meyers's 25-point Website Usability Checklist. Partial excerpt:

website usability checklist

I think he has done a fantastic job of capturing all the "dumb" things that we should not be doing.

You can download the complete pdf on his site.

Why is it great?

Do you really need me to tell you why? I am not going to insult you.

Except perhaps to say that before you pay thousands of dollars for to a external consultant / agency to "improve" your site, this simple checklist (or mine in the book) will yield huge benefits.

And they are all really simple easy to fix things.

Possible outcomes?

Happy customers. Engaging experience.

And a nice bonus, no one will call you out in their presentation / blog for being a poster child of how not to do website usability. :)

That's it! End of story! Actionability awaits you!

Your turn now.

Please share your own ideas for "aggregation of marginal gains". Is there a report / piece of analysis that always works for you? When you crack open Omniture or WebTrends or Google Analytics for a new site what do you look at first? Have a to die for KPI or Metric? What is one completely flawed assumption I have made in this post?

I would love to hear from you.

Thanks.

PS:
Couple other related posts you might find interesting:

Next Page »