April 2007

30 Apr 2007 01:18 am

incomparable1A frequent query I get via email is to recommend a web analytics tool. Something like: “is Omniture better than WebTrends” or “would you recommend CoreMetrics or WebSideStory” or “is Visual Sciences worth its cost” or ….. you get the idea.

I rarely recommend a specific tool in reply to emails from readers (see Disclaimers & Disclosures), I usually reply back with things to consider when selecting a tool. I recommend these posts:

But the other day I got this very interesting email: 

As I am fairly new to reading posts and I am spending a lot of time off work to find solutions I was wondering if you have ever done any comparison on the different tools out there? Webtrends vs Omniture vs HBX and so on? If so can you point me to some good comparisons?

I am able find a lot of information on all the products but no true comparisons. Not sure if this information is in your book but I will be picking that up this week so I will see then

The nuance on looking for comparisons was interesting and also that the reader had already looked and not found anything really helpful.

planets size comparisonI am afraid that in my reply I could not point to any comparisons that might be really helpful. I have of course read loads of ‘em from individuals and agencies and consulting companies and industry Analysts.  But either the quality is uncertain or the author’s experience with real world needs / implementation / uses is sorely lacking.

What is available is well meaning but rarely will it give you, as the reader put it, true comparison. Lots of check boxes where X tool meets Y criteria or not compared to its competitors, but enough to give you confidence to really choose? Not really.

Buying a web analytics tool is like buying a car, you can do all the research on the web you want but you have to get your butt into the seat of a real car and take it for a test drive and see how you feel about it. Press the buttons, get a sense for how your hands feel on the jamie test drivesteering wheel, what happens when you take quick turns or go over pot holes on the road. Then you start to get a real idea of what the difference is between a Saturn Ion and Toyota Matrix.

Each web organization is unique when it comes to its culture, existing systems, IT skills and infrastructure, the need for reporting or analysis etc (even if different organizations are in exactly the same line of business). What works in your favor is that in our world you don’t have to take the tools you want to compare for a 30 min test drive. Any vendor, especially one that wants your business, will let you implement their tool on your site for free for a few weeks and take it for a spin. 

Bottom-line: If you want to pick the right tool for your company you’ll have to actually try a couple in real life on your real site before you can decide which one is optimal for you.

If you do decide that you are going to choose a tool not based on a read-only (or worse Vendor Marketing VP PowerPoint presentation) then here is something to make your process less painful: Pick one tool from each of the three buckets below to ensure you get the broadest exposure to what is possible.

  • Bucket One : Omniture, HBX (WebSideStory), WebTrends, CoreMetrics.

  • Bucket Two : ClickTracks, IndexTools, Unica NetInsight.

  • Bucket Three : Google Analytics, Microsoft Gatineau.

Rather than choosing from tools that will give you kinda – sorta the same kinds of features or functionality the bucketing recommended above will ensure that you makeBuckets the most optimal decision from a diverse set of choices.

I find that very often people choose from tools that are very similar and ultimately their final choice is not the best. Every once in a while you need a Ralph Nader in the mix!

Important: There is no implication in terms of features or greatness or price etc in the bucketing system. 75% of what you’ll need is actually present in any tool in any bucket. The reason each tool ended up in its bucket is because that bucket offers something compelling and different.

Bucket One “What’s Unique About Me” Sound bites:

    Omniture will literally do anything you want it to do with a ever expanding set of features, HBX is your friend if your company’s best friend is Excel, WebTrends is morphing into messaging itself as a customer insights company, CoreMetrics has a few unique features for retailers. These are tools that are often top of mind when people think of web analytics. Each does something better than the other, but there is also a large overlap.

Bucket Two “What’s Unique About Me” Sound bites:

    If you want to do real post facto analysis then ClickTracks will shine, I wish a lot more people realized how awesome IndexTools’s custom reporting interface is (Ajax!!) and it comes at a great price, NetInsight integrates efficiently with your online and offline campaigns. Each of these tools provides a truly compelling alternative to bucket one and three.

Bucket Three “What’s Unique About Me” Sound bites: 

    Google Analytics brings 100% of the standard web analytics reports with integrated end to end adwords reporting, for free. Microsoft Gatineauwill join the group hug soon (Ian: we are all waiting with a bated breath!). Both of these tools will prove to you that you only have to pay if you can prove to yourself that your needs are complex enough to need something special.

[I am not including Visual Sciences in the mix above because it does stand by itself. It is a true analysis tool, that some might say is expensive (based on pricing shared by a competitor at Emetrics 2007 in London). But if you have true advanced Analyst skills to apply in your web organization, and support from your IT team for implementation, then then you’ll be able to glean far superior actionable insights.]

Summary: Compare by implementing. Learn by using. Identify the best cost – benefit choice by having atleast one tool from each of bucket above in your comparison.


I had written a post in September 2006 that lays out how you can choose the right tool for you. It recommends a linear process (start with a free tool, implement and learn, identify gaps, then go for comparing paid tools – ones in bucket one and two above).

Here is a recap of what was listed in that post:

    How to Choose a Web Analytics Tool: A Radical Alternative

      Step 0: Assign optimal ownership. (Day One)

      Step 1: Implement a web analytics solution. (Day Two)

      Step 2: Start using simple reports and process of creating a intelligent audience in your company. (Day Three)

      Step 3: Teach yourself the limitations of web analytics, tagging, numbers not matching, need to go redo your website IA / URL’s / ID’s / Cookies / data providing facilities. (Day 17)

      Step 4: IT “rules”! Cross your fingers, dive in. (Day 27)

      Step 5: Do a honest and deeply critical self review of where you are. (Day 57 or infinity)

      [Updated] Step 6: Pick one tool from bucket One and Two, above, and make a informed and intelligent the right tool for your company.

      Details of each step: Click Here.

See what I mean? Picking the right tool for you is harder than finding a wife / husband!!

Have web based or Analyst comparisons helped you choose the right tool? Do you agree with how each tool has been bucketed? No? Have a different soundbite for a tool? Please share your feedback, critique, expertise via comments.

[Like this post? For more posts like this please click here.]

23 Apr 2007 12:59 am

challenge smallLast week I had the distinct pleasure of doing a interview with Alan Rimm-Kaufman of the Rimm-Kaufman Group. It was a wide ranging interview that covered insights analytics can provide with a focus on retailers, the 10/90 rule, multivariate testing, my lessons from blogging and more.

Life has been a bit rushed for me recently and with all that I forgot to answer one very interesting question that Alan had asked. Since I have a few minutes now I thought that I'll answer the question now.

Alan asked for my thoughts on how a full-time web analytics person should spend their working week in order to maximize their opportunities for massive success both for themselves and their company.

My answer is below, preceding that is some context about the 80/20 rule for reporting (that I had first mentioned at the emetrics summit a couple years back).  The 80/20 rule for Analysts is a perfect lead into to the next question, as you’ll see…….

ALAN: You've also suggested a 80/20 rule for reporting: analysts should spend 80% of their time doing analysis and only 20% reporting. How much data should a good web analytics dashboard be? How many key metrics should an online retailer track to understand their site? And which ones?

The stress on analysis is because with reporting all you are doing is throwing data out there, usually highly aggregated or deeply detailed, both good exercises to flex data flow muscles but rarely would they make you the lean mean fighting machine that you want to be. Analysis is the art of probing data, of discovering trends, digging beneath the surface, marrying the improbables and in the end finding insights. That’s how you make money.

Take any organization with remote success with analytics, none of them got there by having robust reporting programs. They got there by the sheer courage, hard work of solid analysts who took time to dig and probe and recommend.

80 2D20I think I am on the record on the blog saying that you should only have eight to ten metrics on a dashboard, that you should be able to print it on one page, in eleven size font or greater. For more please see the post: Six rules for actionable dashboards.

There is no pat answer how many metrics each retailer should track. It depends. That’s not a cop out, rather it is a reflection at how there is little in common in any business.

Two retailers, say Best Buy and Circuit City, can have radically different strategies to leverage the web and their analytics strategy will have to fit around their unique web strategy. For example Circuit City will give you a $24 gift card if your online order is not ready for you to pick up in the store in 24 minutes. Best Buy will do no such thing. Tiny example of how your web analytics will be different in each case.

Ok there are some metrics I love and adore that everyone should measure. On the web we all do a very poor job of understanding the customer needs and wants and thus their experience on our sites.

ClickStream data is pretty sub optimal at representing customer experience, so using qualitative methodologies I am a fan of measuring Customer Satisfaction (“were you satisfied with your experience on our site today”), Primary Purpose (“why are you here today”) and Task Completion Rate (“were you able to complete the task today”).

Measure those three using Surveys on your site, you can do Site visits with your customers, you can do remote usability studies. You will find them to be a great complement to your web analytics clickstream data.

ALAN: Let's imagine a hypothetical full-time web analytics person working for a major online retailer. Best case scenario, in a typical work week, what activities would he or she be doing, and roughly how much time would he or she be spending on each?

    Inspired 2DInternsIf it is a major online retailer the first thing I would recommend is that they also hire a young person, perhaps a intern, to take over the reporting responsibilities (or pay someone at your Vendor / Consultant to do it). It is important to realize that this is the first step.

    In any large organization you can’t get away from reporting demands from lots of folks (as in “our department needs these three reports every two days” or “we have to publish these standard metrics company wide” and so on). Your chances of massive success are low with just reporting and so please don’t expect your Analyst to do reporting and for that to in turn to bottom-line impacting results.

    A fresh person out of school should cost you a lot less than your Analyst, and they, Mr. or Ms. Fresh, will work enthusiastically on reporting (and all the while get great training on becoming a true Analyst).

    So if you are a Analyst then here is what I recommend you start with (customize for your organization as you see fit):

    20% Reporting – Sorry you can’t escape this, you are still going to do reporting. But on the bright side it is a great way to keep in touch with reality.

    20% Analyze Acquisition Strategies – What is your company doing to attract traffic to your website? SEM? Affiliate Marketing? Banners? Email Marketing? What else?

    There is no faster way to win the hearts and souls of your company stake holders than helping them understand how their efforts are performing.

    Focus your analysis on Outcomes (revenue?) that your company desires and you’ll do fine.

    20% Understanding On-site Customer Experience – Using a mix of ClickStream and qualitative methodologies analyze what the customer experience is on your website. Really. Not what you think it is, not what your company wants it to be. But what it is in reality.

    There is no better way to make money then this (and you’ll get happy customers as a bonus).

What 20Would 20Analyst 20Do

    20% Staying Plugged into the Context – Most Analysts suffer because of a lack of context. They are put away in a corner with Omniture or WebTrends or HBX or Google Analytics and expected to produce earth shattering insights.

    We need to have our Analyst use 20% of their time simply to stay plugged into what else is going on, in the marketing organization, on the websites in terms of operational changes, with senior management (anyone higher then their boss) to know what their strategic pain points are (then imagine how web data can solve them), with the phone or retail (big box) channels, etc.

    Web analysis is not a silo and the analysts needs to be plugged into the context so that they can look at the right data, better and provide relevant insights (that they can’t possibly provide when locked in their ivory tower (!!).

    10% Explore New Strategic Options – I don’t know where your company is but you always want to move the ball forward, this chunk of time should be spent in experimenting with new and different ways to move your program forward.

    Think testing, competitive intelligence analysis, multi channel integration, usability etc. If you really are a one man band this is really hard to do (especially with 10% of your time) but think of tiny ways in which you can show that web analytics is not just about ClickStream, it is about creating better customer experiences and it is about creating a strategic advantage using data.

    10% Bathroom breaks , oh and lunch! I am generous aren’t I. : )

    I realize that each company is unique and each Analyst is unique, but I hope that the above picture provides a semblance of universal guidance for anyone how has the word Analyst in their title. Do a quick back of the envelope of how you spend your day/week and compare it to the picture above. What do you find?

Bonus reading material: Top Ten: Signs You Are A Great Analyst.

What do you all think? Am I being reasonable with how Web Analysts should spend their time? Is 20% too much time for staying plugged into gleaning the context? Not enough? What about the 80/20 rule for reporting success? Please share your thoughts and critique via comments.

[Like this post? For more posts like this please click here.]

Next Page »