April 2009


20 Apr 2009 01:54 am

bright purple Do you have a sneaking, yet unshakable, suspicion that your Web Analtyics Vendor is sometimes just trying to mess with you?

Guess what?

It's true!

All web analytics tools have a smattering of metrics and key performance indicators that were created just because someone decided it would be cute to add / subtract / multiply / divide some numbers.

Many of these don't pass the first sniff test and when if they do you are still left wondering: "What in God's name and all that is holy in this world am I supposed to action based on this metric?"

The answer?

Nothing.

With that gloriously upbeat set up let me tell you what we are going to cover today: Three metrics that are available in pretty much all "adult" web analytics tools. Daily, Weekly, Monthly Unique Visitors.

daily weekly monthly unique visitors They are so common yet most people don't understand them well enough and fewer still realize how harmful these can be to your health even in day to day use.

So in this post we try to understand the most basic of the web analtyics basics, the Unique Visitor computation.

What's a Unique Visitor?

It is simple really. . . .

Technical Definition: Count of all the Unique cookie_id’s during a given time period.

English Definition: The first time someone visits your site a first party persistent cookie is set in their browser. This cookie lasts any where from several months to several years. Each time that person visits your site that cookie identifies them as the same browser.

unique visitor really 1Notice I said browser, not person. It is likely, but not always true, that each a unique visitor is a unique person.

You can learn a lot more about Visits and Unique Visitors in this post: Standard Metrics Revisited: #1: Visitors.

Very predictably every 18 months or so the blogosphere goes wild with how accurate, or not, the Unique Visitor metric is. Much mud is thrown around. Indignations are foisted on the world. Name calling ensues.

Regardless of that Unique Visitors remains a valuable metric that used correctly, in place of Visits, measures success of your online marketing efforts.

Oh and your best weapon against ignorance? Education. See above post on Visitors. And this one: A Primer On Web Analytics Visitor Tracking Cookies. It covers cookies and deletion rates and other such yummy stuff. Read that and you have my word you'll be the smartest cookie in the room.

See what I did there? :)

Daily, Weekly, Monthly Unique Visitors:

In many web analytics tools (say Yahoo! Web Analytics, Omniture, WebTrends etc, but not in Google Analytics ) you'll also see Daily Unique Visitors, Weekly Unique Visitors, Monthly Unique Visitors and, sometimes, Absolute Unique Visitors.

monthly trend of daily unique visitors

Each is trying to tell you something about Unique Visitors, yet if you pause and think about it, I mean really pause and think about it, you'll realize two of these are really bad for your health, and the third should be used with caution.

The core reason is that what looks attractive initially becomes progressively worse as you extend the time period. The Daily metric, so to speak, does not even last in value beyond two days!

So let's spend a second understanding this slightly yucky phenomenon.

Here's the data, from omniture.com, where WebTrends is used for tracking Visitors. . .

visits by unique visitors

Now let's go measure the complex set of metrics that'll stare at you, let's say when you crack open Omniture or WebTrends (or pretty much any other competitive web analtyics tool).

The Web Analytics Unique Visitors Story:

Before that realize that what you see will depend on the time period you are looking at. [Arrrh!]

And before I really really jump in… you'll see a metric called Absolute Unique Visitor. I am going to use that as a proxy for how unique visitors should be computed correctly, regardless of what time period you are computing it for. Keep an eye on that number.

Looking at Month 1 and Week 1 at the end of Day One:

daily unique visitors

If you ran your reports at the end of day one here is what your analytics tool will report to you, with some delight and joy I might add. . .

Daily Unique Visitors: 3
Weekly Unique Visitors: 3
Monthly Unique Visitors: 3

Makes sense right? Do a happy dance, high five someone next to you, heck give them a hug and a kiss.

Now let's make this more "complicated".

Looking at Month 1 and Week 1 at the end of Day Two:

unique visitors for two days of a week

If you ran your reports at the end of day two here is what you'll see. . .

Daily Unique Visitors: 5
Weekly Unique Visitors: 3
Monthly Unique Visitors: 3
Absolute Unique Visitors: 3

Slow down the happy dance a bit.

Note the silly effect on Daily Unique Visitors, even though it was the exact same folks, Dennis and Matt, from the earlier day who visited on day two. They get counted twice.

Life lesson: Daily Unique Visitors is a useless number if you are looking at a time period of more than one day!

Let's keep going.

Looking at Month 1 at the end of Week One:

unique visitors at the end of week one

Crack open your analytics tool, it has been a long week, look at the metrics, here's what you'll see. . .

Daily Unique Visitors: 6 (!)
Weekly Unique Visitors: 3
Monthly Unique Visitors: 3
Absolute Unique Visitors: 3

Note the continuing uselessness of the Daily Unique Visitor number (and even if you trend it over time, as in the blue graph above, analyze what it is actually showing you? what's the insight?).

In your Web Analytics Tool you might see a report that looks like this:


summing daily unique visitors no

By know you know why there is a sad frowny face in that last Total row. Right?

Repeat: Life lesson: Daily Unique Visitors is a useless number if you are looking at a time period of more than one day!

Looking at Month 1 at the end of Week Two:

weekly unique visitors

Gather everyone in your close proximity in the office, form a circle, hold hands, close your eyes, say a quite prayer, now open your analytics tool. . .

Daily Unique Visitors: 10 (!!)
Weekly Unique Visitors: 6 (!)
Monthly Unique Visitors: 5
Absolute Unique Visitors: 5

The Weekly number is wrong because it counts: Avinash, Dennis, Matt, Matt again, Ian and Jim. It counts Matt again because he visited during both weekly time periods.

Life lesson: Weekly Unique Visitors metric is useless if you are looking across multiple weeks. We've covered above why Daily Unique Visitors is, to put it mildly, sub optimal.

Ok only two more scenarios left, hang in there, it gets better.

Looking at the end of Month 1, for the whole month:

monthly unique visitors

By now I am sure you are 100% up to speed on what you are going to see. . .

Daily Unique Visitors: 13 (!!!)
Weekly Unique Visitors: 9 (!)
Monthly Unique Visitors: 6
Absolute Unique Visitors: 6

There is now triple or double counting happening in both the Daily Unique Visitors and Weekly Unique Visitors numbers.

Life lesson: Both Daily Unique Visitors and Weekly Unique Visitors numbers are useless when you look at a time period of a month.

One last scenario, not to make your brain hurt but rather to ensure you reach the state of maximum Analysis Ninja enlightenment!

Looking at the end of Month 2, for the two months:

visits by unique visitors 1

Tingling with excitement. . . here's what you'll see. . .

Daily Unique Visitors: 19 (kill me now!)
Weekly Unique Visitors: 15 (can't breathe!)
Monthly Unique Visitors: 12 (!)
Absolute Unique Visitors: 9

There is now triple or double counting happening everywhere, the Daily Unique Visitors, Weekly Unique Visitors and Monthly Unique Visitors numbers.

The correct measure of unique is the Absolute Unique Visitors metric because it de-dupes the unique visitors across the entire time period you are reporting on.

Life lesson: Both Daily Unique Visitors and Weekly Unique Visitors numbers are totally really useless when you look across months. Use Monthly Unique Visitors with caution, knowing it is only de-duping for each month and then summing the number for each month.

absolute unique visitors 1

If your tool provides Absolute Unique Visitors you are in luck because then you are getting true unique visitors across whatever arbitrary time period you choose.

Google Analytics provides you with the Absolute Unique Visitors metric.

google analtyics true unique visitors across time periods

It will do that across set time periods, like the month of March (or any number of months). . .

march unique visitors

or across arbitrary time periods, as Monday March 9th through Thursday March 19th. . .

random date range unique visitors

It will dedupe the numbers when it reports to you, rather than adding the totals of each day, week or month.

Complex but bonus for Ninjas: Depending on which graph you look at, daily, weekly or monthly, it will intelligently compute the number for each time period and also show you the aggregate deduped number for that time period.

Fly in the otherwise rather healing ointment?

Google Analtyics does not compute Absolute Unique Visitors when you segment the data, when you use the Advanced Segmentation feature. Those of you who read the blog know my utter infatuation with segmentation, so you can easily understand how sad this makes me.

You can get Absolute Unique Visitors for segments by using the "create a filtered profile that just data for the segment" method and that works if you have forethought. But it is sub optimal, just like some "enterprise" web analytics vendors telling you that you can only segment if you tell them before the fact what you might want to segment later.

Why do Web Analytics Vendors torture you with Daily, Weekly, Monthly Unique Visitors?

why so painfulI knew you were asking yourself this question!

Good on you Mate.

If these metrics are that sub optimal, why do web analytics vendors put us through this torture?

Simple: Compute power (translation: cost, for them).

It is very computationally intensive to calculate for you the true real (Absolute) Unique Visitor number across any arbitrary time period or across multiple weeks or months.

Increased computational intensity for the vendor means more processing time and higher costs.

So doing Daily, Weekly and Monthly counts (and then summing them up) is cheaper for them.

After the first vendor decided to do this, and there were no major outcry from Web Analytics Users (or even Ninjas!), others quickly followed.

For the more prevalent vendors in the space Google Analytics is one the rarest that provides the truly de-duped Absolute Unique Visitor metric (in aggregate, not segmented, boo!). Only time will tell when Google will buckle under the computation/cost weight and stop providing it true Absolute Unique Visitors.

[Update: Both NedStat and Xiti, two wonderful European companies do allow for computation of Absolute Unique Visitors out of their standard packages, no additional payment or gyrations required. Add Unica's NetInsight to that list as well! Hurray!!]

There are some vendors that will tell you that you can buy their more expensive data warehouse solutions (at an additional cost on top of what you pay today) and then compute Absolute Unique Visitors yourself. True. Ask for the cost. Ask if its really Absolute. If prudent, pay more. Regardless, be informed.

Long lesson.

But now you are truly at a Analysis Ninja black belt level of proficiency!

Now your turn.

Please share your comments / feedback / critique / hugs / non-hugs about this post. What does your tool do? How do you think we should improve things? What would you eliminate? What would you add? What did I miss?

PS:
Couple other related posts you might find interesting:

06 Apr 2009 01:36 am

curveEvery once in a long while you come across a tool that just gives you goose bumps, you are instantly infatuated.

The Search Based Keyword Tool (SbKT) was that for me. The data it brings together and the transparency it brings is just so. . . sexy.

Let's see if you feel that by the time you are done with this post. I can tell you know that you'll never think of Paid Search the same way!

[UPDATE: Recently Google has folded the Search Based Keyword Tool into the core AdWords Interface. You can now access the keyword tool from within AdWords by clicking the "Keyword Tool" link from the Opprotunities tab. You can also access the Keyword Tool outside of AdWords by clicking here.]

One of the most common phenomenon is the long tail of search. Yet precious few people understand this and even fewer actually are able to use it to their advantage.

A quick recap: The Search Long Tail.

If you have not had a chance please give my long tail of search post a quick review.

The essence of the phenomenon is that your website most likely has a “short head” (say ten or fifteen keywords that drive massive number of Visits) and usually a “long tail” (hundreds of thousands of key words and key phrases that drive five, ten, fifteen – few – Visits).

the long tail of search1

The really cool thing about the tail is that the thousands of queries in the tail are generic ("category") queries which tend to bring "impression virgins " to your site. People who are new to your franchise, people who have not made up their mind.

Find them first and you get first dibs on convincing them.

My Long Tail: The Story in Numbers.

Here's a real example, data for this blog:

87,861: # of Visits in the month of March.

40,662: # of Visits from Search (!!).

26,137: # of distinct keywords and key phrases that drove those 41k Visits!

Astonishing is that not? For a single topic blog! So what of the head and the tail?

13: # of search key words that brought 100 or more visits (the head).

26,124: # of search key words that brought 2 or 5 or 10 or a small number of visits each (the tail).

If you are not yet confused by all the numbers, please stick with me, here is another way to think about it.

13 head keywords brought 5,128 Visits to the site.

26,124 tail keywords brought 35,534 Visits to the site!
[This is the prime reason I tell Analysts, Marketers, CEO's, my children not to just obsess about the head!]

Clearly even a one topic (Analytics) blog can have a verrrrry long tail.

I can only imagine how long your tail is. You have a real business. You have thousands of pages. You have tons of products you sell, a diverse set of services and so on and so forth.

keyboard on fire

The Problem?

Despite the massively yummy opportunity, most people have a hard time monetizing the long tail because:

* All they know is their brand terms, so they take the easy way out.
I hate to say this but there is laziness involved, we rarely bother to look beyond the top 10 keywords.

* They think broad matching on their key terms will take care of the tail. The reality is the best broad matches will pick up 500 or a 1000 queries, the tail has hundreds of thousands.

* You can't possibly take every word in the text on your site and dump it into AdWords (or AdCenter or YSM).

Finding relevant long tail search queries is tough, and doing that intelligently is even harder.

The Solution?

The Search-based Keyword Tool (SbKT) from Google.

It uses two sources of data that Google has (both anonymous, non-PII) to help you identify long tail keywords that are being searched on www.google.com that are relevant to your business.

The first source of the data are the logs that store the queries that are done by users on google.com (logs that have this data in aggregate, not by individual histories – you'll see the latter is not necessary).

search queries google indexed page results

The second source of data are the logs from googlebot that indexes the world wide web and all the pages (and your site). These logs understand what your site is, what content is on it, what you sell, service, do on those pages.

SbKT simply brings these two sources together. When combined it provides you with:

A list of keywords and phrases in the search long tail that are relevant to your website (where relevance is defined by the content on your site).

It is a very simple idea, yet it is powerful because now you don't have to guess what the opportunity is for the long tail on google.com.

Google SbKT Reports & Analysis.

SbKT is a fairly easy to use tool, there really is not much to it.

This screen is what you'll see when you login. . . .

google software based keyword tool

You have several options but first thing you can try is to simply type the name of your website and hit enter.

In two seconds. . . . Bam!

skbt long tail search results

Ok so the color boxes won't be there. : )

Let me walk you through what you'll see.

The area inside the Red Box shows all the key results you need: key words and other data.

The Violet Box shows how many relevant keyword suggestions were returned, 65,863 for this site.

The data in the Blue Box allows you to filter the results by various product categories. Example, here, Apparel, Beauty & Personal Care, Sports & Fitness, Finance etc etc.

The Orange Box is the other way to filter the data, by specific product categories. Sony, Microsoft, HP, Samsung, Nokia etc in this case (so you can easily move away from the 66k results and focus just on results for Nokia, which in this case were 890).

Finally the Purple Box, at the very top, contains other filters (cost, competition, keywords etc etc).

Let's walk through the report itself.

Keywords, Monthly Searches, Competition.

The first column shows the specific keyword / key phrase that the tool recommends for your site. These are the long tail keywords identified after the analysis of Google's search query logs and the Googlebot log data.

The second column shows you how many searches were done for that keyword in the last month (+ or – 10%). It is important to know these are not match typed in any way, these are actual search queries of google.com users.

keywords monthly searches competition skbt

Finally the third piece of data is how much competition there is for an ad for that term (in your country typically, so you can change and choose to get data just for a specific country).

Suggested Bids, Ad/Search Share, Your Webpage.

Sugg. bid is the price you can expect to pay for showing your ad in the top three position on the Google search results page.

suggested bids ad share extracted from page 1

Ad share is % of time an ad from your website shows up for the query (due to broad-match). Search Share (the second percentage) is the % of time your site shows up in the first page of the organic results!

Think of both numbers, for Paid and Organic, as akin to impression share (or to those of you in the Retail Industry: "Share of Shelf")

Finally column three shows which page on your website is relevant for that keyword.

This last part is very important because you don't want to buy keywords willy nilly, you don't want irrelevant ads or, worse, irrelevant landing pages. SbKT solves this problem by telling you exactly which page on your site is the optimal landing page for a given long tail keyword.

You can click on the Export button and save the resulting Excel file. There will be two bonus items in there: 1) SbKT will attempt to classify your keywords into logical Campaigns and 2) into logical Ad Groups, thus saving you some time.

Net, Net.

You know:

exactly which keywords to buy to grab all the impression virgins

how much share you have today of that traffic

how much you can expect to pay for a top 3 paid search listing

and exactly where to send this traffic so they have a relevant experience (which translates to a higher conversion rate)

Delightful. Data driven paid search campaigns!

SbKT Bonus Features / Analysis.

There are several deeper and cooler things you can do with SbKT.

Question: I want to have a post spring break sale on my GPS related inventory, how can I find the most relevant keywords and key phrases to purchase?

From the left navigation, in this example, click Consumer Electronics, then click GPS and do a merry dance. . . .

google sbkt gps long tail keywords

What you are looking at are the keywords related to the GPS category only and only the ones that are relevant to your business.

At the far left, not in the image above, you'll see the unique landing page on your site where you should send this traffic (in case you buy these keywords).

Question: I just saw a tweet that Olympus is going to out of business. How can I identify search keywords to unload all my Olympus inventory?

From the Categories choose Consumer Electronics, then Photo & Video and then Digital Cameras. From the Brands (bottom left) choose the brands Google has identified relevant to content on your site (Olympus).

category plus brand equals targetted keywords

There you go, a relevant list of keywords for you to consider.

Question: Budgets are really tight right now, can you help me find long tail keywords relevant for my business but that cost less than 30 cents?

In the top bar click on the plus sign next to Advanced Search. . .

sbkt advanced search

You'll see a bunch of lovely options, in this case you'll choose Suggested Bid and type in the amount that you are interested in. . . .

identifying low cost long tail search keywords

When you hit enter SbKT will take the 222,720 long tail key phrases it had identified for you and show you the 75,464 that are relevant for your business – all of whom have a Suggested Bid of under 30 cents for a top three search listing.

As you see above, you can control the amount of competition you want. If I choose Medium and High ("bring it on!" :)) I get a list of 35,486.

You can also use other filters productively.

Win – Win for Organic and Paid Search Marketers.

If you are a Paid Search Marketer (PPC) then use SbKT to identify relevant long tail keywords and their corresponding landing page. Use Suggested Bid to focus on maximizing your budget and give your CFO estimates of cost (rather than guesses that you probably do today).

electronics long tail keywords

If you are a Organic Search Marketer (SEO) then Google has just provided you with a list of landing pages on your site and the keywords that are relevant to those pages. If you are not getting enough Organic share (ps2 above) then you have your work cut out for you.

For either individual there is data now to work more efficiently to improve search share.

Who Can See What?

Before you get too excited let me rush to add that you can only see data for your company using SbKT, i.e. the adwords accounts you have access to.

For other sites your view will be limited to only the top 100 long tail terms. You can see 100 in All Categories or in any Sub Category.

Tips for Maximizing The Long Tail Value.

long tail orgasm
[Image Credit: Hugh MacLeod]

#1: Don't get hung up one or two or ten keywords. Remember you are dealing with thousands of keywords.

Focus on the criteria that is important to you (low cost, high competition, particular categories or brands, a cluster of products), then put 'em in ad groups and experiment.

#2: Don't spend time torturing every single keyword. You can't. Or you'll never go home.

You are aiming to capture two, ten, fifty visits. Maximize for the portfolio.

#3: Remember all rules of measuring success for Search campaigns apply to the long tail (it must produce bottom line impacting results).

But you have a chance to think in a more sophisticated way about measuring the long tail. Please see this post: Measuring Value of “Upper Funnel” (Long Tail) Keywords

#4: SbKT works most optimally for mid and large sized websites (lots of pages, lots of products, lots of lots).

Notice the irony in that? :)

Closing Thought.

The data you get through the Search Based Keyword Tool are actual search queries that are actually being typed by people using www.google.com.

The information from the tool should be used to create effective Organic and Paid search strategies to ensure you get your share of the relevant traffic.

Just because you don't show up does not mean that your competition won't.

Good luck!

Oh and in case you did not realize it, this is Web Analtyics baby!!

Ok your turn now.

Have you used Google's SbKT? What's your experience like? If you have not used it, do you think you'll find it to be of value?

Got any tips on monetizing the long tail? Got horror stories?

Please share your critique, feedback, cheers and boos. Thanks.

PS:
Couple other related posts you might find interesting: