Excellent Analytics Tip #18: Make Love To Your Direct Traffic
Make love? Direct Traffic? Really?
I am not kidding. Direct traffic contains visitors that proactively seek you out, everyone else you have to "beg" to show up on your site!
Yet this question seems to bedevil a lot of people:
What the heck is Direct Traffic?
As if that was not sad enough, even people who do know what the definition of Direct traffic is rarely focus on it or work hard to tease out the opportunity that exists in Direct traffic.
I love analyzing Direct traffic because it contains a valuable set of visitors who deserve more love than we currently give them.
I want you to be just as excited.
So let's look at the definition, to make sure we understand, at least on paper, what this traffic is supposed to be. We'll also look at the challenges that exist in ensuring we are looking at the real unpolluted Direct traffic.
Definition.
Here is the simplest and cleanest definition:
Direct traffic contains all Visits to your website where in people arrived at your site directly (by typing the url) or via a bookmark.
Direct traffic is hence not traffic that results from people clicking on links on other sites to your site (that's referring urls traffic), it is not traffic that comes to your site by clicking on ads (that's Other in Google Analytics or Campaigns in other tools), it is not people who come from search engines (that is Search or Organic or PPC traffic).
The reason Direct traffic is a beloved of mine is that it represents (checkout the sweet contextual – red and green – numbers above):
1. People who are your existing customers / past purchasers, they'll type url and come to the site or via bookmarks.
2. People familiar with your brand. They need a solution and your name pops up into their head and they type.
3. People driven by word of mouth. Someone recommends your business / solution to someone else and boom they show up at the site. Uninvited, but we love them!
4. People driven by your offline campaigns. Saw an ad on TV, heard one on radio, saw a billboard and were motivated enough to typed the url and show up.
[If you were really smart you would use campaign tagged vanity url so you can segment them!]
5. [Remember the part below, but.] Free, non-campaign, traffic.
In a nutshell these are people show up without invitation (email, display, social campaigns) or they are people who already know you. There is an extra motivation connected to their visit which causes them to type your url of find the bookmark they made.
That little bit of extra intent, when compared to other visitor segments, is the reason that conversion numbers (on ecommerce or non-ecommerce sites) for clean direct traffic usually look like these. . . .
The only goal that is red is supposed to be red (fewer registrations from people who already know you is not unusual right?).
Now you'll agree when I say your job is to be extra sweet to them?
Segment them in your data, the delightful numbers you see in your KPI's will show you why.
So if Direct traffic is so important and often the metrics show very positive results then why don't we all obsess about it a lot more?
The Problem: Unfortunately. . . You!
Most website tag and campaign tracking implementations are poor (to put it charitably). This is always disappointing but it is particularly harmful to Direct traffic.
You see if you don't implement your links properly the person shows up to your site without any tracking parameters and thus fail to help your web analytics tool to put that visitor in the right source bucket.
Typically Direct traffic also contains all the Visits that originated from improperly tagged campaigns, untagged campaigns and problems with your JavaScript tag. I am sitting in a puddle of tears as I write this, that is how often Direct traffic is polluted and that is how painful (and profoundly sad) this is.
Here is a simple example:
You are the Acquisition manager for a company called Omniture.
You have purchased banner ads in various Android applications using AdMob to target high value analytics decision makers. You goal is to get people to buy your Discover data warehouse product.
You are using Google Analytics to track all you display campaigns.
The proper way to link your banner to your Discover2 website is:
omniture.com/discover2awesoemness/?utm_source=nytimes_mobile_homepage&utm_medium=masthead_banner
&utm_content=188_92&utm_campaign=affluent_readers
You actually use this url:
omniture.com/discover2awesoemness/
Guess where this category will be categorized?
Direct.
:(
You see mobile applications don't send a referrer and it will look like all of a sudden you got very high converting Direct traffic.
With a simple stone you've killed two beautiful birds:
> The direct traffic is polluted and you'll never be able to focus on finding real insights for actual valuable lovely people who are seeking you out directly.
> Google Analytics will show that your mobile campaigns with AdMob stink. Of course that's not true, but you'll have no way of knowing that.
Not a great situation right?
Oh and what do you think is happening to the trackability of all your shortened urls in Social Media that you are not tagging with campaign parameters? 78% of people consume Facebook and Twitter content via applications and unless you use campaign parameters all that traffic is sitting in Direct. So sad.
Result?
Direct traffic is a fantastic segment to analyze because it contains desirable Visitors and yet because it is often polluted (due to our own inability to implement web analytics tools correctly).
Let's aim to fix this because it is too important not to.
. Why Does Direct Traffic Get Polluted / Mistakes You Should Avoid:
The good and the bad are all mixed in, and it is your job to ensure that that is not happening inside your web analytics data.
Here are the main reasons traffic that should not be Direct ends up there, try, please please pretty please, to ensure this is not happening to you:
1. Missing web analytics tag from landing pages
Perhaps the most common source of traffic being miscategorized.
Your urls are all tagged correctly with campaign parameters, or maybe people are just coming to from sites that link to you.
They land on a page that is missing the web analytics tag.
They click on a link on the landing page to go deeper into the site.
Guess what's the traffic source for this traffic?
Direct.
So sad.
You worked so hard to get that referring link / execute the campaign. Now not only do you not get rewarded for that work. you actually messed up your direct traffic.
Don't be that person.
Go purchase WASP from iPerceptions or an account with ObservePoint and address the cheapest problem to fix in Web Analytics. If you are a little bit tech savvy then go get REL Software's Web Link Validator, it's pretty good.
2. Untagged campaigns (search, email, display, social media etc)
This is perhaps the second biggest reason data in web analytics ends up in wrong places.
In case of untagged emails (to people who are using Outlook, Thunderbird etc) and mobile ads and mobile application links (think of all those Twitter / Facebook apps) and Adobe AIR applications (like my beloved NY Times Reader) and in rare cases where people are clicking on links in PDF documents etc, the data ends up in Direct (no referrer).
In case of untagged display campaigns usually there is a referrer so it will end up there rather than in Campaigns were you want it.
In case of untagged paid search campaigns it usually ends up in organic search data.
On behalf of your company you are spending precious budget on acquisition, not ensuring your campaigns are tagged properly is near criminal behavior. Don't be that person. Tag.
Oh one more thing.. if you are practicing bigamy and have two tools, say Google Analytics and Adobe's Site Catalyst you better remember to have campaign parameters for both GA and SC because they use different parameters for campaigns. Whichever one you forget to tag for will show your campaign traffic as Direct!
If you want to track the campaign in the first part of this post with both Google Analytics AND Omniture the url would look like this, as an example:
omniture.com/discover2awesoemness/?utm_source=nytimes_mobile_homepage&utm_medium=masthead_banner
&utm_content=188_92&utm_campaign=affluent_readers
&s_scid=TC-10013-3159426121-e-361634984
See both set's of campaign parameters? You don't do that one of them is wrong. Not so shiny to practice bigamy is it?
3. Improperly tagged campaign parameters / site tags
- Excellent Analytics Tip: Unsuspected Correlations Are Sweet!
- Excellent Analytics Tip: The Adorable Site Abandonment Rate Metric
- Excellent Analytics Tip: Measure the Real Conversion Rate & "Opportunity Pie"?
- Seven Steps to Creating a Data Driven Decision Making Culture
- Analyze This: Five Rules For Awesome Impromptu Web Analysis
- Analysis Ninjas: Leverage Custom Reports For Better Insights!
This one is probably not that hard to understand.
Instead of utm_source and s_scid you use utm-source or s-scid and you are. how to say this politely. screwed.
In both cases your two (or one) web analytics tool will most likely ignore the improper parameters and throw the traffic where it does not belong and mess up your ROI analysis.
Auditing your campaign tracking before they go live is a great idea. Do this at the very minimum for the 20% of the campaign that are responsible for 80% of your traffic / revenue.
If you use Google Analytics grab the Google Analytics Tracking Code Debugger. See this blog post for troubleshooting guide & detailed instructions: Debug Your Tracking Code.
Omniture, WebTrends, CoreMetrics, Unica all come with such debuggers. I can't link to them as location are not public (or you need to pay first!). Please reach out to your Account Managers to get access, just in case you don't already have them. Debug!
[Update:]
Ben Gaines from Adobe/Omniture was kind enough to share that a free debugger is available to Omniture clients. Log into the Knowledge Base and look for KB ID 534 and you are set! But here's something cooler. The debugger is actually a bookmarklet and here it is:
Create a bookmark in your browser. Copy the code in the above text file. Click edit on your bookmark. Paste the code where the Link is. Go to any page on your site with Site Catalyst. Click on the bookmarklet and bathe in bugs! :)
[/Update]
4. Improperly coded redirects / vanity urls etc
Another silly issue that causes problems with direct traffic.
When you get a email or a mobile campaign, and keep a close eye on the url window, you'll notice the click goes to your campaign solution provider and is then redirected to your site.
That's one example of a redirect. We use redirects / vanity urls in our multi-channel campaigns, in our display or search campaigns or even just for the heck of it.
That is not an issue.
Make sure they are permanent, 301, redirects. The delicious type of redirects that dutifully pass the referrer string to the landing page telling your web analytics provider where the person originally came from.
You use temporary, 302, redirects and the referrer never gets passed on. Depending on how the redirect server is configured either the click looks like it came from the redirect server or with a blank referrer (direct!).
5. Really heavy tag at the bottom of the page (switch to Async!)
A smaller problem for normal sites with just text and some images, but a huge problem for fat ugly flash heavy websites (especially the, still annoying, ones with flash intros).
It takes such a long time to load the flash file itself that person might have clicked skip intro or some other link on the page well before the fat flash file loads or before the web analytics JavaScript tag loads.
The data tracking behavior is exactly as if issue #1 above existed, no tracking code on the landing page.
I would recommend putting the tag in the header, except that is the selfish lover strategy and no one likes a selfish lover.
Make your pages as lean as you can, especially campaign landing pages. Keep the tag in the footer, you don't want the page to hang because of issues at your analytics provider.
If you use Google Analytics you are in a little bit of luck. Switch to the magical GA Async Code. It goes in the header, captures data without ever hampering your page loading and as if that were not enough is leaner and meaner.
One of these days all web analytics vendors will migrate to the Asynchronous making the Internet a faster place to live in.
6. Corner cases causing traffic to end up in Direct.
Here are some reasons that don't happen a lot but you should be aware of:
~ Links encoded in JavaScript clicked in some browsers will send a null referrer (i.e put traffic into Direct). Often times you can't help his because you don't have control over people linking to you can do whatever they want. But do check that your campaigns in Facebook or Yahoo or other places are not using this method.
~ [Update, via Alec Cochrane:] https to http and vice a versa also won't have referrers passed due to (good) security reasons. So if possible make sure you put campaign tracking codes in links from https pages to ensure those visits don't end up in direct. For this you would have to know this is happening and then be able to find the person who will oblige you by changing the link. Tough to do but when you can do it!
~ Some smart folks will make changes to their browser configurations that cause referrers not to be passed. Happens in a tiny minority of cases.
~ This might impact only some tools but check with your vendor how this scenario is credited. . .
First visit: From a campaign (search, referring url, social, display, whatever).
Second visit: Direct to the site.
If you are using Google Analytics then that second visit will still be "credited" to the campaign (non-direct) because the _utmz cookie will be present in the browser.
In your web analytics tool that might not be the cause. Please check with your vendor.
~ Multi-domain / sub-domain "unique" web analytics implementations across many websites. With any tool these are really hard to do right, and really easy to do wrong. If you have one of these polka dotted puppies then get your expensive Consultant to triple check the code and cookie customizations with a special eye on Direct traffic.
~ [Update, via Pritesh Patel:] You could also have polluted Direct traffic if your entire company (hopefully of a good size!) has their home pages in browsers set as your company's website. This will clearly skew your direct traffic (and your bounce rates, after all they don't actually care about your site :)). You can easily use your tools admin settings to filter out all your internal IP's which would solve this issue.
~ [Update, via Deric Loh:] #1. iFrame: Whenever someone links to your site via an iFrame it is possible for them to code it in such a way that it does not pass referral data and the visit will look like Direct. We can do much about this but in case there are sources where you can avoid this issue or get it done properly then it is worth the effort.
#2. Company Gateways: Some companies might have a security gateway which has been set up to strip the referrers from request calls. This of course is not great for your clean Direct traffic. It won't happen a lot of times and then limited to just one source. But it is something you certainly should be aware of as a cause.
That's it. Six simple problems for you to take care of. : )
All kidding aside know that you'll accomplish a major clean-up if you address the first three issues and then YMMV.
Also know that it is totally worth it to get this data clean, the orange line below is Direct traffic conversion rate and the blue is overall conversion rate. . . .
Not bad eh?
You want to know who these people are.
You want to know what you can learn from analyzing their geographic locations.
You want to know their Visitor Loyalty and Visitor Recency profiles.
You want to know what content they are consuming.
You want to know what products they are purchasing.
You want to know what the differences between their behavior on your site is from your other campaign traffic.
You want to know if any of the spikes are correlated to you offline campaigns or catalogs you have sent out (and then establish causality between offline campaign calls to action and behavior by these people).
You want to establish the value of these visitors and then pay special attention to them if they are of value to you.
For the New York Times website I'll always be Direct traffic. I use a bookmark, I go to the site at least once a day, I click on Ads (I have nytimes.com on my adblock white-list!), I subscribe to the Times Reader, I am a big evangelist of their brand.
But only if they care to ensure their Direct traffic is clean, and then analyze that traffic will they ever know that.
If they are like every other company that obsesses with PPC and Yahoo! Banners and Facebook Display ads and Email campaigns etc etc then they'll never know that some of their best customers they should make happy are right under their nose.
I know that the NY Times web analysis team is super sharp. Are you?
In the small chance that you were not before I hope I have convinced you to truly bring the "make love" type of passion to this valuable, and usually large, segment of traffic to your site.
Good luck!
UPDATE: A clarification specific to Google Analytics:
Every tool uses delightful sets of attribution rules when it comes to assigning visits or conversion to campaigns. To share with you how Google Analytics will attribute these things here are a couple of scenarios….
Scenario 1:
Visit 1: Came from SEO click on keyword "ASOS Fashion"
A few days later…
Visit 2: Came direct to the website
In Google Analytics you will see this in your reports:
Keyword "ASOS Fashion": Visits: 2
Direct: Visits: 0
In effect Google Analytics will "understate" direct visits. It is difficult to have a perfect scenario here, some people will vehemently make the case that GA is doing it right and that the Visit did come via the organic click first so second visit should be attributed to it. I am personally in the camp that that is sub-optimal and that because we can't read too much into anything (we just don't know what is influencing what) we should report keyword visits = 1 and direct visits =1. But at least you know what GA is reporting.
Scenario 2:
Visit 1: Direct to the site.
Visit 2: Came from Affiliate Campaign click.
Visit 3: Came direct to the site.
In GA it will show:
Direct: Visits: 1
Affiliate Campaign: Visits: 2
See how that works? Regardless of how you think it should be you now know how it is. : ) Make sure you keep this in mind as you analyze the GA reports.
[My heartfelt thanks to David Williams for his help with a test for above cases.]
This stuff is complicated right? Remember none of this takes anything away from the importance of direct traffic or how hard you have to work to make sure your reporting of it is clean (tips above) or that it is worth focusing on. Whatever tool you have, do all of the above!
Ok your turn now.
Do you obsess about Direct traffic just as much as I do? What insights have you found from you analysis? What methods have you deployed to ensure that your Direct traffic segment is as clean as possible? Do you also look at any "Direct" traffic to really long complicated url's on your site and instantly doubt that could be direct?
Please share your experience / feedback / tips / critique via comments.
Thanks.
PS:
Couple other related posts you might find interesting:
September 20th, 2010 at 03:04
Great post Avinash. Agree that Direct Traffic is sometimes not given the TLC is needs. Honestly, I am guilty of this sometimes.
I recently did an anlysis of a new clients website and logged into their Google Analytics to start my analysis. I was shocked when I saw that the site was receiving 94% direct traffic. I assumed it was due to email campaigns with poor tag management and adwords campaigns not set up accordingly.
I asked the client "Do you conduct email campaigns?" To which they answered…"No". So I then asked "What about PPC?"…they replied "Whats PPC?". "Redirects??"…"nope, we have not changed the site for 12 years"
So what could be causing the 94% direct visits if they are not conducting any other form of online marketing?.
Answer: The business employs 150+ people and each computer within the business (inc. laptops) had its homepage or start up screen set as the company website homepage!!!!!
So, when each employee fires up the browser and the company homepage appears….classed as DIRECT VISIT!
P.S – The company now has changed its start up page to Google.
September 20th, 2010 at 04:24
Hi Avinash,
Great way of pointing out why you should ensure that you should put campaign codes on your links wherever possible otherwise you'll not just muck up any analysis of campaigns, but also other traffic sources!
As an interesting tangent on the topic, I like talking to our brand marketers and brand PR about 'Brand visits'. That is visits who have either typed in the url (to become direct) or have searched in a search engine for your company name. This is, of course, complicated if your company name is a common phrase as well.
Cheers,
Alec
PS – https to http and vice versa don't have referrers for security reasons, so make sure you put campaign codes on if you link from someone elses website to your own from an https page!
September 20th, 2010 at 05:42
Hey Avinash,
Great post – Direct traffic has always been important to me, it provides the ticket to the conversations about offline campaign effectiveness and is a major HIPPO slayer. All the more reason to keep it clean and credible!
All the best,
Rob.
September 20th, 2010 at 05:45
Love it… Omniture using GA…
Also, love the points on incorrect referrer/ad tagging. Great fodder for an analyst looking for love in IT to help tag up misbehaving pages. ;-)
Regards,
Garry
September 20th, 2010 at 05:54
Avinash,
Since you brought up polluted data, there's a whole area of polluted data you missed around direct traffic, and traffic sources as a whole.
What I'm talking about is actually the reverse problem of what you've discussed, but it is still a polluted data issue.
What we've noticed time after time is that if we send out an email campaign, our organic search traffic goes up, our cpc traffic goes up and our direct traffic goes up.
And I mean all on the same day.
Of course what's happening is that when people get our emails, instead of clicking the link in the email, they just go straight to the browser and
i) type in the url directly or call up a bookmark
ii) simply type the company name into address bar and wait till the search engine bring up the result then click on either the organic link or
iii) click the paid branded link (thankfully both are at the top of the page.)
It's nothing to do with improper tagging, and there's no way you can unsegment and re-segment them properly.
It's just human nature, and you have to live with it and make allowances for it.
September 20th, 2010 at 06:25
Avinash,
Fantastic post.
Would love to hear more about best practices with site experiences for direct traffic. For example, disabling registration intercept promotions, or enabling special features that show the love…
Thanks again,
Ted
September 20th, 2010 at 06:32
Hi Avinash!
First of all, it was really nice to see you again last week and chat with you for a bit. I promise to not "disappear" again from your comments on this blog! :)
I think this is a great post to highlight the impact of one's offline or direct marketing campaign. I believe that we are all guilty of not paying too much attention to direct traffic (TRUE direct traffic) – myself included. We all take it for granted, but we need to be able to analyze it and optimize for it, just like any other marketing campaign.
Just wanted to mention a quick technical note regarding direct traffic and Google Analytics: If, for example, one bookmarks a website after clicking on an AdWords ad, that user's next visit to the website will also be lumped in with the "cpc" medium category. There is a fraction of the population that is actually coming back to your website via a bookmark that Google Analytics will attribute the traffic to their original source. So – like anything in the Web Analytics world, it's not exactly perfect, but the data that you do in fact collect is more than good enough to make smart, actionable decisions from.
September 20th, 2010 at 06:38
Excellent Post!
Highly relevant to marketers who are trying to leverage inbound marketing and social media.
Derek
CEO
DMZ Interactive
September 20th, 2010 at 07:01
….oops! Looks like I overlooked the part where you talk about campaign bookmarks in GA….sorry about that!
September 20th, 2010 at 07:18
Hello Avinash,
I agree that marketers sometimes dont pay attention to Direct traffic as much as we should. This is an excellent post that makes me evaluate direct traffic all over again.
On email marketing: There has been some talk in the email marketing circles of tracking emails better in GA. It would be fantastic if Google Analytics would give a beacon to track email opens. Just a thought.
Can anyone throw some light on why most of the ESPs use 302 redirect on clicks and not 301?
September 20th, 2010 at 07:50
"They land on a page that is missing the web analytics tag.
They click on a link on the landing page to go deeper into the site.
Guess what's the traffic source for this traffic?
Direct."
Are you sure about this? I always thought that in this case, my own domain would show up as the traffic source.
September 20th, 2010 at 08:17
Lot of website open link to other websites using Javascript (window.open) When this happen, referring information at not pass along in the HTTP header. Disguising this traffic as direct.
If this happen from a partner site, you could include campaign parameter to correct this. Or you could ask them to open the window to a redirecting page from their domain. You may not have the actual referring URL, but at least, the traffic would show as referring traffic from that domain.
September 20th, 2010 at 09:15
Amen! :-)
September 20th, 2010 at 09:39
Hi Avinash,
Great post, as always. I have been answering this question a bit lately myself, as the Marketing Channels reports in SiteCatalyst have a "Direct" bucket which a few people have tried to remove/ignore.
In any case, just FYI, our Debugger requires neither login nor money to access! I won't post the bookmarklet URL here simply because it's long and full of JavaScript, but anyone can get it from our KB, and thereafter it can be used at any time, shared internally, etc.
We also offer the DigitalPulse product for a more automated way to make sure that incorrect tagging isn't messing up your direct traffic analysis.
Thanks again for dropping more wisdom upon us!
Ben Gaines
Product Manager
Adobe Systems, Inc.
September 20th, 2010 at 10:11
What about the people who type your URL into google. I'm sure these would show up as organic, but they shouldn't they really be considered direct (they're just too lazy to click on the address bar to type in the URL). Is there a way to correct for this in GA?
September 20th, 2010 at 10:36
Pritesh Patel (1st comment) mentioned a very common problem. You can change the default start page in the browser to Google, but that doesn't solve the problem. Typically, the employees will still type in the URL and bam! Direct visit.
Avanish, would you agree that it's better to block the company IP address in the analytics software to stop this?
September 20th, 2010 at 11:01
@Alec ( #2 ) makes a good point in his PS: https referrers to your http site show up as direct as well.
(example: search from https://encrypted.google.com/ – small corner case)
Also, I've noticed that searches in the location bar (on Firefox at least) use a hidden Google search + 302 redirect.
Details, see http://forwardslash.nl/herkomst-stiekem-toch-via-google/
September 20th, 2010 at 11:47
Hi Avinash,
I love the comment about utilizing vanity URL with a query string for offline media campaigns. It's like pulling teeth around here to help people see the big picture sometime.
Thanks for the thoughts!
September 20th, 2010 at 14:02
Great post Avinash.
In terms of prioritizing your efforts, do you generally focus on the direct "branded" traffic first? Or, the people who are discovering your site via a long-tail phrase that matches a product offering you sell. I've wrestled with the above decision before and almost always default to trying to improve the user experience of "non-branded" traffic first because I take the "branded" traffic for granted and expect them to have more patience for the business than the "non-branded" traffic. (terrible me!)
@Tony, if the employees use the website as a resource – it may be a good idea to create two distinct profiles : 1) that includes only the traffic from the employees and 2) one that excludes the traffic from the employees. <3 Filters.
September 21st, 2010 at 00:59
I like how you are insinuating that Omniture would use Google Analytics to track their discover campaigns.
Bravo!
September 21st, 2010 at 05:28
Hi! Thanks for a wonderful series of blog posts (and books!). I have a question that I'm unable to get a simple clear answer to, and my apologies if it's already answered on your site. (If it is, maybe some kind soul could point me there.)
We're using Google Analytics and are a bit confused about Google's definitions: As I understand it, "Visits" are the number of times your site has been visited by a particular browser (with a cookie). My problem is that Google further breaks this down as "Returning Visitors" and "New Visitors" — yes, that's right, VisitORS.
Surely, they mean "Returning Visits" and "New Visits"? Both of these are added up to calculate the total number of Visits. Meaning a user can return several times throughout a day and increase the number of Visits, and therefore also the number of Returning Visits. I.e. the number of Returning Visitors has not increased, only the number of Visits.
Thanks for anyone who can clarify Google's confusing terminology!
- JohnnyW
September 21st, 2010 at 07:10
#11 – I believe you are correct. Kind of surprised that Avinash – who is an analytics evangelist – would make this statement – unless he wasn't referring to Google Analytics, but some other unnamed provider.
September 21st, 2010 at 08:52
Yeah, Direct is often one of those "just when you thought you had tracking entirely set up . . ." challenges.
Many times what I see are correctly tagged campaign links for banners, Facebook sites, and SEM. However, that old standby email gets the short end of the stick.
Usually the excuses are along the lines of:
"But Ty, we can just use the referral domains of the email servers to tell where they come from."
or
"But Ty, our email provider will provide that data."
In the former case, if you don't properly tag your email links you'll find you leave out recipients who use a mail app rather than web mail.
In the latter case, you're introducing another data source that invariably will cause confusion when comparing it to your web analytics tool. Also, the email vendors do a great job of reporting on email-specific measures (open rate, CTR, etc.) but not so much on the site side measures. Both are important but don't assume one is the same as the other!
I love analyzing Direct traffic! More so when I am confident those visitors are meeting the definition you mentioned at the beginning of the post.
Thanks!
-Ty
September 21st, 2010 at 09:02
Very interesting stuff. I didn't realize all of those factors could muddy the "direct" category– must do some research to see if we're making any mistakes.
At my ecommerce company, we've actually seen a noticeable decline in direct visits over the past year or so, because even our loyal and dedicated customers have shifted to sharing and clicking on links through social networking much more than they ever used to. And what we're seeing is that it's not necessarily a bad thing– people are just as likely to buy something specific recommended by a friend or by us on Facebook than they are after typing our URL in and just browsing the site for something new. But we might be an edge case, because brand loyalty is extremely high and we cater to a demographic that is shifting its Internet behavior from typing URLs to interacting with the web as a whole primarily from their social network homes.
September 21st, 2010 at 10:54
Hey Avinash,
Another source of direct visits is from running AB experiments on a landing page. If the visitor is placed in B page group, the redirect from A to B happens before the web analytics tracking code executes and page B will show a direct visit.
I also think that if you use the omni bar in Chrome to do a Google search and the result is your domain name, and you then click on it to go directly to your site, this will show as a direct visit. Not unexpected of course, but just an example of Chrome "helping" you do direct type-ins.
September 21st, 2010 at 11:10
Hi Avinash,
I really dig the post you've written here because I've constantly seen Twitter traffic underreported as either direct or referring from a web client (e.g., Hootsuite).
I noticed though that you didn't put campaign parameters around your tweet yesterday that linked to this post:
http://twitter.com/avinash/status/25037288376
Given your post, I would assume that your direct/Twitter traffic has overlapped a bit as a result. Is there a reason you didn't put campaign parameters around it?
September 21st, 2010 at 14:18
I was just reading about IE9 jump lists, which seem really neat (http://msdn.microsoft.com/library/gg131029.aspx). Firefox could do well to add this feature.
I assume pinned sites would register as direct traffic as it is a form of bookmark, but I wonder if it would ever be worth it to attach a campaign code to the list URL's to compare "regular" direct traffic to site pinners. Is it reasonable to see if site pinners have a higher return rate, order rate, email sign up, aov, etc.? Or does this simply muddy the water?
September 22nd, 2010 at 13:45
Something interesting that I've found as a side-effect of understanding the impact of Google Instant that I wanted to share…
I was measuring how people perform Google searches, whether the Google site, browser search box, or directly into the address bar.
I noticed that address bar searches from Firefox (specifically) that contain certain branded terms have been triggering a 'feeling lucky' search, and the scope of the branded terms affected has been gradually widening for the last two weeks.
The issue is that these 'luckies' are showing up as DIRECT.
Depending on the brand, of course, the keyword entered in the address bar may as well count as direct, but one can imagine a case where that behavior would not be so negligible.
I would rather know exactly how a visitor arrived.
September 23rd, 2010 at 08:22
Hi Avinash,
besides redirects I often encounter landing page URLs with a wrong syntax:
http://www.landingpage.com/?param1=value1?param2=value2
Now imagine either param1 or 2 being the campaign parameter:-)
Using a debugger is great, but too technical for some people.
Clicking and checking in the WA tool afterwards can indicate problems to everyone.
so long
nic
September 23rd, 2010 at 22:44
Pritesh: Thanks for sharing this excellent tip, it does happen in more companies than would be optimal. I'll add this to my recommendation #6 as another thing to watch out for.
Alec: It is pretty hard to discern which brand visits are "url typing" and which ones are genuine search (and then attempt to bucket the former into direct). Life is hard for us Analysts. :)
Thanks for highlighting the https to http issue, totally forgot that one. I'll add it as a point under item 6 in my list.
Garry / Bjoern: I am just trying to have some fun there.
But it is also the case that it I know the intricacies of GA a bit better than the intricacies of Site Catalyst. This means using GA as an example of tracking is easier for me.
So there is a practical reason as well.
Mark: You are right that that is certainly an issue for some people. That is precisely the reason the #1 link in the PS part of the post is the one about "unsuspected correlations". Please check out that post for some techniques that might help identify this important segment of traffic.
Ted: Once you can isolate the real direct traffic then of course you have to deliver value, react to them in a differentiated manner. How you do that will depend on what each business is, but I encourage people to read the examples in your comment – all of which are wonderful!
Geeta: At the moment, perhaps due to history, what happens from the time of email leaving you, or your provider, to getting to your target (customer/prospect) is tracked by email campaign providers. What happens once people click in the email to your website is done by web analytics tools. It is just the way things are, perhaps it will change some day.
But it is important to know that these two parties, email campaign provider and WA tools, do work together pretty well. Tools like mailchimp and contactmail and exacttarget and so many more integrate with Google Analytics and Omniture and WebTrends and everyone else.
So when you choose your email provider choose one that has this integration and you'll have the end to end view. And remember to ensure all the links in the email are tagged with tracking parameters for the WA tool you use.
Lukas: This is a often misunderstood issue. In some tools landing on a non-tagged page and clicking to a tagged page will show that Visit as a "self-referrer". But in many other tools, including popular ones, that "self-referrer" will be ignored and the visit will be put in Direct (for example in Google Analytics).
So you should check with your vendor to see what is happening and then adapt your analysis.
Either way ensuring all landing pages are tagged is a lovely gift you can give yourself. :)
Tony: Yes, you can update your web analytics tool to recognize your internal ip addresses and discard them. This is a 30 second process.
But I have always maintained that I would much rather that you and I and everyone else worry a lot more about the core problem behind your employees skewing your numbers: You don't have enough real Visitors!
My recommendation: Focus on attracting more traffic, focus on campaigns, focus on a better/different product, focus on SEO, focus on…. so many other things. If you have lots of visitors and a healthy business then internal visits become a minor ignorable nuisance. :)
Ben: Thanks for sharing the tip on the Omniture Debugger. I have updated the post with the Omniture code so people can quickly implement the bookmarklet and quickly if their site catalyst code is behaving the way they expect it to.
-Avinash.
September 23rd, 2010 at 22:46
Mike: It is really simple, create an Advanced Segment that combines Direct traffic source AND the keywords from Google that have your url in it. Boom! Real direct traffic. :)
Each business will decide something different as to if the url typers are "lazy" or "direct" or "search referrals". But whatever you decide you have Advanced Segmentation to help you look at the data the way it makes sense for you.
And remember in Google Analytics (and a small number of other tools) data in Advanced Segments also applies to historical data without you having to update the javascript tag (like in some tools) or paying for reprocessing data (like in some tools). So segment away!
Brian: The case you are describing is Search and not Direct right? If so…
It is hard to decide between Branded and Non-Branded search traffic mostly because it is hard to know if the Branded queries are by existing customers or just "down in the last steps of the sales funnel" prospects.
I have made that decision based on historical value of each segment (looking at macro / micro conversions) – this just takes applying a couple segments to your goals reports. The other factor I use is to figure out what the business is currently prioritizing, sometimes there is a concerted effort to get more people into the top of the funnel (non-branded typically) and at other times we are just trying to improve the bottom of the funnel outcomes (typically via brand traffic) in which case that's an area to focus on.
Obvious ideas, but I hope they are of some value.
Jhonny: These are fairly standard terms and common in most tools. There is a simple way to think about this.
Visit: This is a session. It represents a series of clicks during one visit by a person.
Visitor: This is a close approximation of a person (or said better a person who is using a specific browser). They are tracking using an anonymous cookie.
Returning Visitor: Is a session on your site by a Visitor who has visited your site before.
New Visitor: Is a session on your site by a Visitor who has never visited your site.
Thinks of New and Returning as segments you are applying on your Visits data. How many Visits were by Visitors who had already been on my site? How many Visits were by people who had never visited before?
If it helps you think of them as Returning Visits and New Visits that is ok, use it. But note that you are 1. segmenting and 2. classifying the people and hence perhaps the WA tools went with the term Visitor.
Kalani: Thanks for sharing your use case. It is certainly a good one.
Ideally you want the loyal or valuable customers that you have to have the kind of brand connection that they'll come directly to you, rather than waiting to see a link on a social site and be prompted to visit the site. So some amount of those type of customers are optimal. Is that number 15% or 30% or whatever depends on a number of factors (including as you case a unique type of loyalty).
But I am old school, even if I have 26k followers on twitter. I want a healthy chunk of direct traffic!
Michael: Good point on A/B tests, depending on the tool it is possible that for non-control versions the visit might get classified as Direct. Hopefully it does not happen often enough to cause major issues (so says the person who keeps championing constant testing & experimenting!).
Great point on the omni bar in Chrome. It is less that you are doing a search and more that Google is doing the search and trying to find the best answer from your history or Google search (organic). Perhaps we can add this in the "win" column for Direct as in this case Chrome is causing the data in the WA tool to be cleaner (though of course that is not why it is doing what it does, it's just purely for better user experience).
Josh: Why not? Would it not be great to look at the report for a post on Direct Traffic and see lots of Direct traffic! : )
The serious reason that particular link was not tagged is that I am experimenting with some advanced pre-alpha features on three url shortners (awe.sm, goog.gl and my bitly custom domain, zqi.me). I just happened to use goo.gl because I was testing something else.
Andrew: Great idea. But I am not sure that we'll have the capacity to figure out how to add the parameters when people pin sites. At least based on my reading of how it would work. Maybe we can dynamically take everyone who comes directly to our site and then add tracking parameters to all their urls (utmsoruce=direct) and then in case they bookmark the site or pin it then the tracking parameters will go with them. Might be cumbersome.
But both you and Michael point out how browsers are advancing and becoming smarter (and more complex behind the scene). As analytics professionals we have to constantly keep an eye out on these things to understand what the impact might be on our data, direct or in other buckets.
And what is up with that Google Instant? : )
Nic: That is another mistake people make in the list of problems people have tagging their campaigns right.
You are correct that this is a technical issue that many people don't pay attention to or detect. But then that is why we have consultants out there to help address this issue. It is worth 10x to pay a consultant the required fees to ensure the tagging and tracking is implemented correctly.
-Avinash.
September 24th, 2010 at 03:08
Thanks so much for your reply, Avinash, it makes a little bit more sense now. It still baffles me why they would choose "Visitor" as a name to describe a segment of "Visits".
As you say…
A visit = Unique session.
A visitor = Unique user/browser.
So it seems confusing to me that "Returning Visitor" should mean "unique _session_ by a visitor who has been on your site before".
Thanks for clarifying my understanding of the term, though. It's much appreciated.
September 25th, 2010 at 10:51
Thanks for putting some needed focus on the value of direct traffic, Avinash.
One of my favorite new advanced segments is something I call "brand aware" traffic. It simply contains Direct traffic and all branded search. The reason for this is people like to type the website address in a search engine (derr) OR they type the brand into a search engine versus go through the hassle of typing ".com" (and let's be honest, many of us are guilty of typing 'Facebook' into Google instead of typing the web address or even clicking a bookmark). I don't want to take this as far as using a replace filter to turn branded search engine traffic into direct traffic, but I like slicing and dicing it using advanced segments.
The other thing I like about having a brand aware traffic is it helps puts some context around my SEO traffic. For example, I could have a month where my targeted phrases' rankings improve nicely, but traffic remained flat or even fell! Then I'll go back and look at brand aware traffic and see that it fell significantly more over the same period. It's not a comprehensive explanation, but if traffic from people aware of our brand fell, it might give some insight into demand over that period and could explain why SEO traffic didn't increase despite better rankings.
Two stories on inflated direct traffic:
We have a client that didn't take into account the subdomain visitors go through to convert into leads. As a result, all converting traffic was defaulted to direct traffic! This made it very difficult to analyze which keywords and campaign were effective when we first came on board. For the amount they spend on PPC, this "infertile" data was devastating.
Another client made another simple mistake so all of their ecommerce traffic was reported as coming from direct. So much money spent on a new site, SEO and PPC and they couldn't measure what channels were most effective.
For this reason I put so much emphasis on taking your time and being super, super thorough when setting up a GA account. Simple oversights can set you way back!
September 27th, 2010 at 08:50
Hey Avinash,
Great post, especially since I have the problem that the direct traffic on our homepage suddenly spiked and I need to find out why.
I checked all the possibilities, the only one I couldn't check for every page is, if the GATC is properly implemented (we have a lot pages).
What I detected is this:
- Direct traffic went from 30% to 45 %
- Search traffic went from 55% to 30%
- New visits went from 40% to 60%
- Returning Visits went from 60% to 40%
Does enybody have a clue in which I should investigate? I see all those crazy metric changes and can't explain how that happened (there was no big campaign or such).
I am very thankful for every idea.
Cheers,
Max
September 27th, 2010 at 09:07
Max: It is really hard to make general guesses based on the data below. You should consider doing some checks outlined in the post.
If I had to hazard a guess: New campaigns landing on pages with no tags. Or your search campaign tags are broken (if you do lots of paid search) – though this should not impact the new visits going up and returning going down because no JS tags / no campaign tags don't impact existing cookies.
Perhaps others on this thread could add other ideas for you. But remember none of us know your business and problems like you do! : )
-Avinash.
September 27th, 2010 at 09:18
@Max: Can you give us the raw numbers over the two time periods?
The % shifts could mean anything, either one variable going up or the other down!
Thanks,
Rob.
September 27th, 2010 at 10:36
Max: here's my hunch. It looks to me as if you HAD a reasonably high proportion of returning visits, many of them from search. If these returning visitors had been using type-ins or bookmarks for their return visits, those would have been credited to the last campaign source = search.
If, for some reason, your site managed to reset the tracking cookies so that the connection with previous visits was lost, this same chunk of users would be reported as 'new' not returning and their visit would be attributed to its actual source 'direct' not search.
Can you think of any changes to the site tracking which might have taken place lately? Have you, for example, made changes to cross-domain tracking or done anything else which may have changed the setting of the domain in GA. I've had painful experience of that causing this kind of trouble.
This is only a first guess, of course. I expect there will be plenty of other suggestions from people who know a lot more than I do.
September 27th, 2010 at 10:53
Thanks for your guesses.
Avinash:
my first shot were also the Landing pages.
I checked my major LP and found all GATC's implemented correctly.
Our AdWords campaigns don't really have a huge impact.
Rob:
Here are the numbers (per day value in one day grab sample, same weekday in both periods)
"Normal metrics period":
- Direct: 10,135
- Referring 4400
- Search: 18,000
- New: 13,500
- Returning: 19,400
"Weird metrics period":
- Direct: 17,018
- Referring 4,400
- Search: 14,700
- New: 22,700
- Returning: 13,500
Thanks,
Max
September 27th, 2010 at 10:53
@Max – the first place I would go is the place where you couldn't – ensure proper GATC. There is a great tool called WASP (not trying to be promotional, but I actually learned of it from reading this blog). I highly recommend that you bark up that tree.
That said, the numbers you're describing would make a lot more sense alongside absolute uniques for the time periods you're evaluating. If you're seeing a corellary jump in visitors, there's a chance that you've gotten a bit of press that you're unaware of.
September 27th, 2010 at 11:01
Tim:
That approach sounds really interesting. I'll investigate in this one with our web guys. I realized that some URLs (with huge impact) are handled differently in GA now:
Instead of showing up as quicklinks http://www.url.de/path?orig_ql=/quicklink (1)
they show up as:
http://www.url.com/quicklink (2)
–> GA thinks these are new URLs: Pageviews for (1) dropped to zero and for (2) took over.
BUT this happened two days after the metrics for visitor sources and types shiftet.
I'm confused.
Cheers,
Max
September 27th, 2010 at 11:01
@Max – I suppose we hit 'send' at nearly the same instant…
On the one hand, it's incredibly refreshing to know that there are people that are analyzing traffic daily.
On the other, it's important to keep in mind that larger sample sizes are more likely to reveal trends than day-to-day measurements.
And perhaps I'm too skeptical, but I find it odd that, aside from the direct, all of your numbers show an even hundred. ;-)
September 27th, 2010 at 13:03
Thank you Avinash, your article is elaborated and well written although I still need to learn the terminology to fully understand and take advantage of what you teach here.
September 27th, 2010 at 14:30
Great post.
I think the issue with Direct Traffic is how you make it actionable as so much of direct traffic is in the hands of the visitor and cannot be controlled unlike search etc. This I think creates an aura for marketers where they feel this channel is not one that they can impact unlike search traffic for example. I do agree that ensuring the data is fully clean is the first step but what interests me more is how to create actionable optimisation programmes from direct visitors.
For me the big opportunities lie in two areas:
1) Using annotations to identify the halo effect from your marketing initiatives. Direct traffic is all about context. By being focused on annotating all marketing activity its possible to identify patters e.g. offline (vanity URL's are great but not always viable OR used by customers!), online advertising campaigns as post impression visitors will show as direct etc. By taking this approach you can start to build associations and add another KPI behind each campaign – increase in direct traffic!
2) Using tools such as BTBuckets to actually create custom content based upon direct traffic to reward these visitors who regularly come to your site. How about a survey to your direct traffic to ask them what makes them return or run a promo voucher to direct visitors that have returned to your site x times in a given period.
3) Identify brand advocates by profiling your direct traffic and building a direct relationship with your core customer base. Access to new product beta's, exclusive content.
Would be great to see what other ideas people have about actionable programmes to your direct traffic….
Thanks
Paul
September 28th, 2010 at 00:29
Thanks for sharing such a complex topic in simple way, specially i love the discussion…d
September 28th, 2010 at 12:18
I really enjoyed this post, thanks! I was wondering if you have any good resources regarding link tagging and it's impact on SEO and link juice flow? I have heard of this hack for GA that allows you to remove the parameters in GA tracking codes and replace them with search engine friendliness. But it would be great to hear from the master (that's you) on this.
September 28th, 2010 at 12:57
Direct Traffic can be so polluted that I always hesitate to refer to it.
Also, add the trend of people now using Google in a navigational way, i.e. typing the brand they know (and often the actual URL) in the search field instead of the browser URL field, and we got this "branded" trafic all skewed up. More and more people do this, knowing that they won't get an error message due to a typo (or simply for plain rapidity), since Google is hyper efficient and will being them to the site they already know. They don't mind the extra click, contrary to marketers' cliché. Is this really search then? I don't think so. I usually suggest to put visits from heavily branded searches into the direct category.
September 28th, 2010 at 13:48
Jacques: In their comments several people have pointed out the issue of using a search engine as a method of navigation. The way they are dealing with it is to simply segment: clean direct + search engine queries with company url + (possibly) all the "brand" queries.
The nice thing with Advanced Segmentation in Google Analytics, Yahoo! Web Analytics is that one can accomplish this in just a few seconds (including applying it to historical data) on the fly with no changes to tags or torture.
I agree that that is the way to go.
I do urge a little bit of caution on implying that the third bucket (all brand queries on a search engine) are necessarily Direct. I have seen that in a few cases, but more often than not they are not "direct" (as defined in my post).
Hence I would advice incremental analysis of new and returning visitor segments for brand search traffic, analysis of the landing page (always a great clue of intent) and content consumption.
An Analysis Ninja would take maybe a couple of hours to do that and insights from that analysis can be used to decide if brand serarch queries belong to Direct or not.
Thanks so much for the feedback.
Miguel: You can checkout the blog post on the Google Webmaster Central blog to learn about canonical, it provides more context around getting control of your url.
+ Webmaster Central: Specify your canonical
Also remember to log into your Webmaster Tools account to specify for Google what's the right url structure for you.
-Avinash.
September 28th, 2010 at 20:05
Great post!
Would you recommend WASP? I have had some troubles with it, but i'm not sure if it's perhaps a bad install
Thanks
-Emma
September 28th, 2010 at 21:12
Thanks for sharing this descriptive post. And the article has proved that the number of direct traffic can clearly indicate your offline influence.
October 1st, 2010 at 18:54
I'd add that your marketing attribution strategy (30 day-last click, etc.) is also going to influence how you see direct traffic in your analytics tool. Things can get particularly muddy if you're using a linear or shared attribution model.
So, you can be doing everything right in terms of tagging, redirects, etc. But your very analytics implementation is going to heavily affect the way you see "direct" traffic.
Wonderful post, per usual
October 2nd, 2010 at 01:33
"Direct traffic" seems so direct but not so – especially with how customers might get came through to a nonline brand today. :)
Something to look into whenever taking over a new web analytics account(the $$$ making or losing traffic source / channel for a business!)
Ok sharing and adding onto the awesome list of "Direct Traffic":
iFrame
Something which we can't really control at times, whenever someone links to your site content via iFrame and had code it to not pass the referral data source.
Company Gateway (striping referrals)
Companies might have a security gateway which had been set up to strip referrals from requests
calls(incidentally at times !)
eDMs (Individual links)
Seen quite a number of cases whereby the individual links found on eDMs not properly tagged which leads to "direct" or even "referral" traffic source.
Will share more when I encounter any potential misleading "Direct" traffic !
Cheers,
Deric
October 4th, 2010 at 05:35
Thanks for the post. Some very interesting ideas that I hadn't though about before.
Now its back to the lab to see how our team can properly maximize the value of direct traffic…
October 4th, 2010 at 20:24
The title was very catchy (which can be half the battle for getting traffic!) and great info. Thanks
October 8th, 2010 at 12:12
I found your post right after this one very good. Got me understanding the general area more in a way to approach it (being what I never really get to).
A question though regarding this issue – or related – would be given the direct traffic grows faster than say search traffic would it be a good thing to share Analytics data with Google apps as one can click in some thing to do.
A suggestion for your blog would – might it exist – a easy to use step by step thing their you link down to the longer guides.
Best regards
Hans
October 11th, 2010 at 07:53
Very useful information…thanks as always!
From my experience, one other problem area that can affect direct traffic is specific browsers and how they choose to handle http referrer headers.
For example, Chromium has had issues handling such headers when right-clicking and opening windows in new tabs. Firefox doesn't seem to have this issue.
October 21st, 2010 at 06:47
About Improperly tagged campaign parameters / site tags.
Wasp has now this really quick way to figure this out, there is a report now call the "Explore" report that will allow you to see the tags on your site in a totally different way.
Instead of going through massing listing of pages to figure out what pages are using what account number for example you can see what pages are tag with a specific variable/value.
So if you want to see what UTMAC is used for example, you would select UTMAC in the list, this would give you all the account number used across your site and all the pages tag with each account #.
Real fast real simple :-)
November 25th, 2010 at 04:36
Hello Avinash,
Your blog is truly an inspiration. I am trying to get smarter but there are so many dimensions to it!
I hope you'll be able to help with a quick query on the point of campaign redirects. We use a .htaccess-like file in our CMS to redirect from campaign URLs to landing pages tagged in our case for Yahoo! Analytics.
For example, ordnancesurvey.co.uk/financialservices redirects to ordnancesurvey.co.uk/oswebsite/business/sectors/bfi/index.html?cmp=fs1009
Do I infer from your post that this is likely to be categorised wrongly in direct traffic?
Many thanks in advance,
Dominic
November 25th, 2010 at 18:05
Dominic: It depends on how you have set up the redirect. You want it set up as a permanent redirect. For your .htaccess file here is a post that has some guidance:
tamingthebeast.net/articles3/spiders-301-redirect.htm
Permanent redirects will pass the original referrer and your traffic won't show up as direct. And there will be world peace! :)
-Avinash.
November 26th, 2010 at 08:46
Hi Avinash,
I am trying to pull some general stats together which relate to consumer behaviour on retail websites for instance:
% of visitors that make a purchase from a retail sites
% of visitors that abandon their basket
% of visitors that make multiple transactions on retail sites
% of visitors that only buy once from a site
I've looked at a few sites Forresters etc but they want to charge me $500 to download their studies…..I dont think so! Do you know of any free sites that could garnish me with the generic info I'm looking for?
Thanks
November 26th, 2010 at 21:51
Tony: I am afraid your only option is to pony up the $500 to Forrester et al to get this data.
Before then though I encourage you to give some deep thought to what you might do with this information. Do as in action.
We often tend to overvalue information such as this. :)
Avinash.