December 2009


21 Dec 2009 05:31 am

three of a kindYou know what is the one thing stopping you from finding truly actionable insights from your web data?

Web analytics gems lie deep in the data and we spend our lives looking at the top ten rows of data.

It does not matter which report you look at. Affiliates. Products sold. Referring URL's. Pages viewed. Search keywords. Promotions. Geographies. Really pick any report with any dimension you want to look at, we spend our time (and valuable space on our dashboards) looking at the top ten.

We look at the top ten rows of data because:

    1. Too much data from our web analytics tools.

    2. Lack of clarity from our business leaders about what the site is solving for.

    3. Not enough hours in the day to overcome challenge #1 and #2.

But if you just look at the top ten rows of anything here are the two corrosive problems:

    1. The top ten of anything rarely changes (with the exception of hourly changing content – news – sites).

    2. The top ten only focuses on the head, while the magic is in the long tail of anything. Magic related to finding challenges in your business. Magic related to finding opportunities. Magic that will help identify things you can actually action.

Allow me to make the case for you to look beyond the top ten rows in your reports by sharing three short stories. In each case I request you to look beyond the specific request and tool, rather focus on the analysis and how you could possibly apply it. I hope is to inspire, not to prescribe.

For example take a look at this report. . . .

google analytics search summary report

I am sure when you look at it now it appears all mysterious, full of potential. You can't wait to take it out for a first date and then another and by the third date if you do the same old thing it gets boring. You are done looking at the bounce rates and time on site and conversions of these keywords. In the best case scenario you have even optimized landing pages. Good.

The first week's over, now want? Why keep reporting the top ten keywords on you Executive Management Global KPI Dashboard?

Look at the top of that table. For this website 86,837 visits came from 8,939 keywords!

What's going on with the other 8,929 keywords?

We never bother with them both because it is really hard to look at more than 10 rows of data. Harder still to look at 30 or 50 or 70 rows of data. Not only do we have a hard time interpreting insights from lots of data, we can't actually physically look at that much data and find insights.

Last month this blog received 40,662 Visits from 26,137 key words. The top 12 keywords accounted for 5k visits. The other 26,125 keywords accounted for 35k visits! By analyzing just the top ten see how many visitors I would be ignoring?

Here are three techniques I use to overcome the trap of the top ten rows. . . .

1. Advanced Table Filtering.

In the past we all used the standard reports that our web analytics tools churned out.

I don't do that any more. If you show me a report and it is not a custom report that you have created to better pull relevant kpi's into one place then please know that I will think less of you.

A sign of non-laziness is that you bother to atleast create custom reports. A best practice is to pull atleast some input metrics (Visits) with some attribute metrics (% New Visits), have something that denotes customer behavior (bounce rate) and it is criminal not to have atleast a couple outcome metrics (goal conversion rate, per visit goal value).

That best practice gives me this report for my search keywords. . . .

top keywords report sm

[Click on the image for a higher resolution version.]

It is a useful report that helps me understand performance much better than the standard reports from Omniture, Google Analytics, WebTrends etc etc.

But remember I have twenty six thousand keywords referring traffic to this blog.

I want to very efficiently look through them to find something useful.

I want to locate my most important brand terms that perform magnificently for me. To accomplish that I click the link called Advanced Filter under that table and do this. . . .

advanced table filtering

I move beyond the limitation of the top ten rows by creating a simple inline filter.

Find keywords:

    containing "avinash"
    where the "per visit goal value" is greater than $1.7 (a higher bar since the average is 1.3) and
    the Goal Conversion Rate is greater than 10% (again another high bar compared to the site average)

Notice that I can only run a smarter query because I had created a custom report, without it I would have to use the "lame" metrics that I might get in the standard report (or immediately proceed to extracting all the data, all metrics and spending next four hours in excel doing what I can in two seconds in Analytics).

In a second the table transforms into. . . .

filtered search keyword report for brand terms sm

[Click on the image for a higher resolution version.]

You know what's in the table?

Keywords we should make love to because regardless of if they bring a lot of traffic or little, they hugely deliver to the bottom line. Making love might be too small a emotion when you realize compared to a site average of $1.3 the per visit goal value is $5.5. And look at that conversion rate!

Also notice up top. . . I quickly went from tens of thousands of keywords to just 193 I should focus on. I need to analyze these keywords, search engines, landing pages, products sold, leads received, and so much more to figure out:

    1. what is going on here that is so right that it works like magic and
    2. how much can I replicate the lessons learned

Far too often we focus on our losers. Here I start by focusing on winners and see if I can do more of what I know already works.

I could just as well have mined my data to look for

    people on non-branded terms
    people who come on every variation of the names of my two books
    people who come from China
    people who use a cluster of terms I consider most competitive or. . .

The limit to the data I can mine (and I use that word loosely here) is the imagination I have (or better put: the intelligence I possess about important business questions).

So do this. Use inline advanced table filters to go from tens of thousands of rows to just a few that you need to focus on.

Yes you can absolutely do this in Excel. But it will take you five times the amount of time and effort required because. . . please pay attention. . . in this case you are querying the entire dataset of your website and in Excel you'll keep going back and forth to get more data or dump it or write more complicated queried or. . . you catch my drift.

Advanced Table Filters can be used in any report in Google Analytics (Yahoo! Web Analytics also has a very similar feature, actually YWA had it before GA! :)).

The one limitation of the approach is that you'll more optimally analyze your known knowns. You'll even get to understanding your known unknowns. But not your unknown unknowns.

2. Use Tag Clouds.

For pulling lots of data from lots of rows all together into one place I do so love tag clouds.

Download all your keywords into a text file (twenty six thousand or so in my case) and upload them into Wordle and bam!

search keyword tag cloud wordle sm

[Click on the image for a higher resolution version.]

O M G!

Did you think that by doing something so simple you could get such a quickly glance-able view of so much data?

I love search keyword tag clouds because they can tell you about the health of the company when it comes to search.

In my case, above, for example I LOVE the fact that above and beyond everything the word Analytics dominates it all, and what makes me happier is that the word Survey is so prominent (you all know I love qualitative data, now here is proof that my blog attracts so much of that traffic).

I worry a smidgen that people will think this blog is about Google Analytics, it is not, so I am happy that the word Google appears atleast as much as the word Web (and variations like website and online etc). I can't drill down in wordle so easily, but I use technique #1 (above) and it turns out 30% of the time the word Google appears in search queries in the spirit of "working for google", and another 30% for queries like "google insights for search", "google ad planner" etc (tools I have blog posts on). My mind is relieved.

See how I can understand about a strategic concern, at least a bit, using a technique as simple as a tag cloud?

Another thing I am rather ecstatic about is the sheer diversity of the keywords in search queries. It is not my brand that dominates (boo hoo! cry cry!) but rather "category" terms (which bring the "impression virgins"). Metrics and Conversion and Data and Questions (look at that!) and Analysis and Customer and Intelligence and Bounce and Best. . . .

That sweet spread validates some my Search Engine Optimization strategy (something I spend a lot of time on) – go for diversity and attract new people to my "franchisee".

Or in your case it may not. It may tell you different things. The main point is it would be hard to understand some of these macro factors in your data by looking just at a table in your web metrics tool of the top ten keywords!

Here is a tag cloud for a small company you might have heard of, Gatorade. The data does not come from them (obviously), it is from Compete. . . .

search keyword tag cloud gatorade wordle sm

[Click on the image for a higher resolution version.]

I am not a SEO expert, can't underscore that enough, but everything that could possibly be sub optimal about seo/ppc is wrong with Gatorade.

The one brand term dominates their referring search keywords. This in of itself is not a bad thing, Gatorade is a huge brand. But what it indicates clearly that their site attracts people who already know Gatorade and are "pre converted", when perhaps a greater use of the website would be to attract the impression virgins and blow them away with the greatness of Gatorade so they'll never consider Powerade or any other brand.

Look at the diversity of keywords.

Find any?

People use tens and thousands of different ways to find even a web analytics blog. Look at how much different types of content there is on Gatorade.com and MissionG.com and Gssiweb.com and it is quite clear that Gatorade's tag cloud is telling a sad story.

Finally for all the money that Gatorade is handing out to premier current athletes (and the really expensive content Gatorade has on its website related to those top athletes) only two show up in the tag cloud. One that used to be important (though he is still a big brand) and the other that sadly ran over a fire hydrant a couple weeks back. That shows how exposed the Gatorade brand (atleast online) is should something unfavorable happen to these two guys.

I would humbly dramatically change Gatorade's SEO and PPC strategies tomorrow morning.

It is amazing how when you have so much data in one place, using such a simple technique, that you can find some very intriguing patterns in your data, stories that might validate what you are doing right or expose everything that is wrong with your digital strategy.

Simple but effective. Try it for your site. What do you find?

3. Use Keyword Trees.

This is my latest love. I mean it.

I was so happy when I first saw it because of this constant quest I am on to take lots of data and show it on a page.

Zach and the team at Juice Analytics have created two powerful visualizations: Referrer Flow and Keyword Tree.

I adore that last one.

You simply go to http://analyticsvisualizations.appspot.com click on Keyword Tree and you are on your way!

The visualization uses the free, open and multifaceted Google Analytics API. In a few seconds you'll get something pretty and intelligent (how often have you seen those two together :)). . . .

search keyword tree juice analytics sm

[Click on the image for a higher resolution version.]

It's a tree. With branches. :)

While in the case of tag clouds it is difficult to understand the relationships between different words that exist in your search queries, that is not the case with keyword trees.

I was looking up the relationships for the word "avinash", image above, click for a higher resolution version. I am looking at hundreds upon hundreds of rows of data visualized all in one page.

I can easily see long tail queries like "top 10 key metrics web analytics avinash" or head ones like "kaushik blog" or even "kaushik web analytics 2.0 pdf" (my book is not available in pdf form but now I know lots of people are looking for it and so maybe we should get it out fast!).

I can simply walk through the various branches of the tree and it helps me understand in a very powerful way the relationships that exist in my data. It always throws up surprises (partly because of my top ten rows table driven existence I have never actually looked at so much data in such a easy to understand way).

The fun though does not stop here. I can actually look at keyword trees using different metrics.

In this view I am looking at the data for the keyword "tracking", the colors shown highlight the bounce rate metric for each relationship. . . .

keyword tree tracking bounce rate sm

[Click on the image for a higher resolution version to be truly impressed!]

Now I know the queries that stink like a skunk, the deep reds, and find some sweet ones, "event tracking" is one such word (lots of visits with very little bounce).

But I can switch and say. . . . well our bounce rates stink so we'll not use that as a success metric :), let's use the % of New Visitors as a success metric. Ok no problem, press the button on the control panel on the right and. . . .

keyword tree tracking new visits sm

[Click on the image for a higher resolution version to be truly impressed!]

Notice the relationships change, the queries you would have paid attention to will change, what you will action will change. Just with the press of a button.

You can also flip the size of the words. I am using Visits in both cases above, but you can just as easily go for quality (in this case) and use New Visits. I would love to see some kind of Outcome metric there, given my passionate and sustained obsession with measuring end success.

You can do lots of true analysis, for free, with your data and get the kind of insights tables from Google Analytics and Yahoo! Web Analytics and WebTrends and CoreMetrics simply can't provide.

Let me share two snapshots to make that point.

I was genuinely shocked at the complexity of the tree and branches associated with the word "google". . . .

keyword tree google avinash sm

[Click on the image for a higher resolution version to absorb the whole thing!]

The darn thing did not even fit my laptop monitor (1440×900), and there was so much going on that it took me a while to absorb all the lessons.

Meanwhile for the "analytics" branch I can see, at a glance, the 70 or so queries that cause the "main flare" and it gives me a peek into the the head of my visitors unlike anything else. Talk about collecting VOC (and actually understanding it!!).

On the other hand I was quite saddened to see the report for the word "metrics". . . .

keyword tree metrics avinash sm

[Click on the image for a higher resolution version.]

Remember this blog is all about data and metrics. Yet the tree is so "shallow". Or better put it is less a tree and more a bush. Or maybe just a shrub.

For all of the reasons I was less than thrilled with the gatorade data in #2, I am less than thrilled here. From competitive intelligence analysis I know that there is a ton of volume on Google for queries related to "web metrics", and variations, yet I have not done a good job of attracting that traffic.

The above picture does not simply tell me that I need to do a better job of doing SEO for "web metrics", the real lesson is that I need to put in a ton of effort to attract the long tail for "web metrics" because that is where most of the volume is.

You will probably find other lessons from this exercise on your data. Hopefully there is no doubt by now that valuable lessons do await you if you put in the effort to start switching from using tables and excel and shift to using other data analysis / visualization techniques.

Each effort above uses something very simple and while none of them are a panacea, your understanding of web metrics data will not be the same boring self.

Have fun.

[Bonus Item: #4: One strategy to escape the top x rows is listed in the second half of this post, jump to just after the picture of Tiger Woods (!!): Focus On “What’s Changed”.]

Ok… your turn now.

What do you think of these strategies? Have you used them before? Worked for you? Have you used other data visualization techniques that liberate you from the trap of top ten rows? What tools do you use? Got models / approaches / strategies you want to share?

We would all love to learn form you. Please share. Thank you.

PS:
Couple other related posts you might find interesting:

09 Dec 2009 02:30 am

symmetryIt is rare for me to work with a organization where the root cause for their faith based decision making (rather than data driven) was not the org structure.

It is almost never tools. Not any more.

Surprisingly it is often not their will to use data, that is there in many cases.

Sometimes it is that they don't follow the 10/90 rule.

It is always the organization structure.

Specifically: Who owns web analytics / who it reports to from a org structure perspective.

[Let me hasten to add that this, web analytics ownership, does not exist in a vacuum. If your overall web business is misaligned from an org perspective then honestly there is no hope for you, regardless of where analytics sits.]

This is a topic I cover in my new book, Web Analytics 2.0. Chapter 14: HiPPOs, Ninjas, and the Masses: Creating a Data-Driven Culture.

In this blog post I'll share a unique "case study", more like one person's problem, and my advice to them about how to think about the organization problem.

Here's the question / challenge:

I’m facing an issue I’m sure many large organizations struggle with: where should an organization place its web analysts? Currently, I lead a small team of analysts at a medium-sized bank. We are part of the Web Sales division, along with an e-commerce (online media) team and the content crew.

Web Sales is considered a channel in the same way our call-centre, local branches and customer account managers are. As such, we are not a part of the central Marketing (and Marketing Intelligence) teams at corporate. I see a few different options but would be happy to hear your opinion.

You will all agree that it is really hard to answer a question like the one above without spending time with the company and understanding its strengths and meeting the political players involved.

In this post let me share with you a common sense framework I use in my consulting engagements to figure out a home for web analysts.

Each facet of the framework also contains a peek into what I am thinking, best practices I have developed from all the bruises I have (as a Practitioner and a Consultant) and how I end up making the choices I do. I hope it is of value to you all (and now you don't have to pay me large sums of money to do this for you!).

The four pronged real world tested probing and loaded with politics framework to find a home for Web Analytics:

1. How long has the company been doing web analytics, what is the landscape of tools?

timeAre there standard tools deployed? Or is it all cowboy country with "Analysts", if any, running with as much freedom as free range chickens (which by the way I highly recommend!).

I use this as the first filter because I am trying to gauge how to have the highest impact, quickly.

[A] If there is some level of standardization of tools, if there are some analysts (an analyst!), some reports going out on schedule (even if data pukes) then an optimal path might be to centralize some where (see item #2 below).

[B] If it is free range chicken cowboy country then the fight might not be worth it, I lean towards identifying "accelerators" with the goal of finding the best fit division / site / HiPPO and getting them, just them, to embrace web analytics and show the macro organization how value flows from moving from faith based to being data driven. I call "them" (combination of analytical marketer, analyst, HiPPO, Google Analytics, small site – or atleast two of those things) accelerators because rather than waiting for the CEO to save the world, my optimal path is to embarrass the CEO and VP's by showing proof.

That breaks log-jammed discussions and politics like nothing else.

2. What's the state of analytical maturity of the organization (either the center or the division/silos)?

I am trying to get a feel for three things with this:

* How hard to fight?
* How long will the struggle be to move away from faith?
* Should I go with a centralized or decentralized or some other strategy (more on this below)?

If the overall organization is not very savvy analytically (and it is large) then the strategy will be very different. I don't have much patience and I am not going to try and rebuild the entire darn organization in one day. maturityWhen I consult with large companies when they are in this (messy) state my deliverable is a 90 day plan (that relies on the aforementioned accelerators) and a 180 day plan and a 365 day plan.

If you make the mistake of just creating a 365 day plan for your company that is not analytically savvy then…. well you are making a mistake.

If it turns out that the org overall is not savvy but a division / silo is, then they are my new BFF's and any analytical resource that I might have I am going to send their way, even if that analytical resource is a Marketer or a Salesperson who knows how to log into Google Analytics and interpret bounce rates and analytics intelligence.

If it turns out that the org is savvy then this becomes a discussion where I try to interview, chat, unearth the politics, identify the true power centers and make a recommendation about centralization, decentralization or (centralized decentralization).

I wish there was a standard option for every organization, even one that is analytically savvy, but there rarely is. Every business I have delivered the 90, 180, 365 day plans to has gotten something unique.

3. Who owns the power to make changes to the site (not who owns updating pages or hosting the site)?

This is a nuance to the discussion above. But a very important nuance.

Web Analysts (or call them data driven missionaries!) get crushed (and ignored) very often because they end up sitting in an org, reporting to people, who actually don't have the power to make authorize changes to pages, campaigns, acquisitions strategies, testing paths, surveys etc etc.

The Analysts / Marketers / IT dudes keep churning data and sending the insights but nothing every changes.

authority It matters who your boss is and how much power she has to make stuff happen.

So… not a surprise… if you can align Web Analysts (and based on #1 and #2 above the Web Analytics program) with the actual human being who has the power.

The closer you can get to her (direct report?) the better off you are. It does not matter if she (or he :)) is in Sales or Marketing or …. anywhere.

Getting access to data is easy. Finding insights is harder. Taking action on insights is nearly impossible.

If you need to sleep with someone to get your data folks/tools directly aligned with the person than makes decisions, take one for the team and do it! [Ok, only if it's legal where you live. ;)]

4. Which physical organizational model will work best for you? Centralized? Decentralized? Something else?

Every large or small company has to deal with this. Atleast when they a implementation roadmap from me (or you) that looks beyond 90 days, and certainly beyond 180.

Before I go on let me point out that I very deliberately talk about this here, #4. And that's regardless of how analytically savvy your organization is, from pathetic to magnificent, you'll want to come to this last (even as in #2 you are collecting data that will influence you here).

My organization redesign plans have recommended either one of the three models. I have come to realize that from my humble experience that it is the trajectory of the arc of evolution that makes one model better than the other (and, amazingly, independent of the first three questions!).

These models are discussed in Ch 14 of the book but let me give you a hyper fast summary here:

Centralized models (where there is one analytics team, usually in the center, and it serves the entire organization and every need from an ad hoc report to when to go to the bathroom) are a fit for organizations that are earlier in their evolution arc. They are exceptionally good at standardizing tools, best practices, teaching, getting everyone in the org to rise to a local maxima.

They have a nasty tendency to become, and I use this word in its dirtiest possible uses, bureaucracies. Slow moving, disconnected from reality (they are rarely on the front lines and even rarer still connected to anyone's particular business goals) glorified data pukers. Sorry. Had to be said.

If you are executing on a centralized model be aware of the pros and cons.

centralized decentralized distributed

Decentralized models (free range chicken cowboy land where everyone is doing their own thing) are fast moving, directly aligned to someone's (a division / business unit's) P&L and contain people who can get fired pretty fast if the data is not adding value. Just try to implement a paid tool for half a million dollars and dare to not deliver actual usable insights. You are out man!

They also tend to generate inefficiencies (everyone's doing their own thing after all) be it with tools or work or metrics definitions or testing platforms or….. Decentralized organizations optimize for a local maxima and it happens all the time that while individual divisions in a company win, that the company as a whole loses. Pantene and Tide win but P&G as a whole still gets screwed.

I share in the book that the best model in the universe for an analytics team is a hybrid, something I call Centralized Decentralization. There is a lean (# of people) and agile central tem that is responsible for all the pro's you see mentioned above and also satellite lean team (of one or a very small number of people) in the BU's / divisions, that are responsible for the pro's you see mentioned above for decentralized teams.

Everyone wins.

There is a way to structure the leadership of the organizations, there is a way to align incentives and bonuses, there is a specific method to picking the skills required in each part, there is a perfect time to create such a centralized-decentralized organization. But that's for another post.

Oh and one more thing…

it hope

Before you get upset (if you are in IT) please please know that the tweet above comes from someone has spent three years in IT, lived the life and paid the dues. It sadly simply does not work. A mismatch of skills, motivations and what the core existence is supposed to deliver. I'll reluctantly agree with you that there are perhaps exceptions to the rule, I'll believe it if you show them to me. :)

Which division / department offers the best possible home for Web Analytics?

After a lot of experimentation and failures I have come to realize that often (if above conditions are met) Marketing is the best organization for Web Analytics to be in. It is optimal because Marketing is in the business of raising awareness, connecting with customers, presenting the company's value proposition etc etc.

Unlike say Sales that is there to make a quota at any cost each quarter. Or PR that is there to pimp the company and it's greatness to the world (not that there's anything wrong with that). Or Corp Comm whose job it is to share information and where folks are not hired for their business savvy. Or…. other divisions. In my humble experience Marketing tends to have the right set of skills, motivations and their core existence is around current and future customers.

If they have the power in the company, Analytics will be happy there.

Caveat: Remember Marketing ownership is not a panacea. You'll have to go through the questions in the framework above and ensure that there is a strong business leader who owns driving changes on the site and that the company is on the right evolutionary path and…. all the things you read above. And even if Marketing owns web analytics the ideal you are shooting for is Centralized Decentralization.

[Update: Please see Jim Novo's thought on value of Finance as an option for owning Web Analytics.]

Now you know.

I hope you've found the four pronged real world tested probing and loaded with politics framework to be of value and that it helps you make better decisions about how to organize web analytics in your company. It is one of the hardest things to pull off right, and with all my heart I wish you all the very best in your journey.

Ok… your turn now.

What is the organization structure like in your company? Where does web analytics fit? Does it work? If not why not? What would you do differently? What do you think I am missing in my four pronged framework? From your experience how would you make it better? What is one thing I got completely wrong?

Please share your feedback via comments. Thank you.

PS:
Couple other related posts you might find interesting: