Path Analysis: A Good Use of Time?
Path Analysis: A process of determining a sequence of pages visited in a visitor session prior to some desired outcome (a purchase, a sign up, visiting a certain part of site etc). The desired end goal is to get a sequence of pages, each of whom form a path, that lead to a desired outcome. Usually these paths are ranked by frequency.
Is doing Path Analysis a good use of time? In my humble opinion the answer is a rather emphatic no, except for one exception (which I’ll discuss below). Almost always Path Analysis tends to be a sub optimal use of our time, resources and any money that is expended on buying tools that do “great” Path Analysis.
We usually strive to do Path Analysis in a quest to find this magic pill that will tell us exactly what “paths” our visitors are following on our website. If they “follow” the path we intended we celebrate.
If as usually it turns out that our visitors don’t “follow” the path that WE want them to follow then it back to the drawing board to redesign the site structure / architecture to get them to “follow” the path or at times, worse, hours of “analysis” on: what the heck were they thinking when they click on this button or go to that page (bad customers, bad customers!).
Challenges with Path Analysis are:
- Imagine a website with five pages. Page one Start, Page five Finish. With a simple visualization in your mind you can imagine the number of paths that a visitor could take. Now imagine a website with 100 pages, now one with 5,000 pages. The number of possible paths quickly becomes infinity (well not really but you get the point).
- Most of our tools do a terrible job of representing this path: click forward, back to home, click forward, reverse to three pages ago, hit buy. In a world of linear path representation at a page level this is really hard to compute, even harder to depict. Yet this is exactly how our customers browse our websites.
- On most websites the most common path is usually followed by less than five percent of visitors, usually 1%. As responsible analysts could we make any decision on something such a small fraction of site traffic is doing?
- Even if the most common path is followed by 90% of the visitors current Path Analysis has two fatal flaws:
- It can’t show / say which page in a series was most influential in convincing a customer to move on.
- Current tools aggregate traffic into one bucket, when in reality each segment of traffic behaves differently (say DM traffic vs SEM vs “bookmarks” vs Print Ads). Segmentation is always key.
All of the above combine to make it quite sub optimal to glean any actionable insights that will lead to making our websites more endearing to our customers.
There is one exception to this rule. For structured experiences such as a Checkout or a Closed-off DM Landing Page experience (no navigation, just Next – Next – Next – Submit) Path Analysis can identify where the “fall off” can occur. Once that is identified we will still not know the Why (see Qualitative Metrics Post) but Path Analysis is helpful.
Here is example of new way of thinking about “Path Analysis” that I think is heading in the right direction. (Please see Disclaimers – Disclosures first.) There are atleast three more things I would like to see fixed in this version but ClickTracks address some of the usual fatal flaws here.
- It is possible to break down a linear process into one in which we can group a bunch of related pages (say all product pages) into “groups”. This helps fix the problem of linearity because customers can go from A to B to C or C to A to B and it does not matter for related content.
- It is possible for Visitors to show up in any stage at any point (this is actual behavior now with SEO influencing where people land). Google Analytics also has this feature(please correct me if others do as well).
- Perhaps the cutest thing is that it shows which page in the “Path” is most influential in moving people to the next stage. This is awesome because one can simply look at the “darker shaded” pages and know, for example, that no one cares about system requirements but rather the page on our 10 year no questions asked return policy is the most important one in convincing people to add to cart.
- It is also quite easy to view how different segments are influenced by different content, in my unreadable screen shot you can see All Visitors vs Visitors from Google. Imagine this intelligence then turned around and applied to personalization (!).
This is not perfect but getting there and I think all the vendors will soon coalesce around this innovation and we will all be greatly empowered.
Path Analysis as it is practiced currently ultimately is like communism (with sincerest apologies to anyone in my audience who might be offended). There are overt/covert intentions to control things, to try to regulate, to say that we know better than you what you want, to push out a certain way of thinking. I know this sounds extreme, and it is but simply for shock value and not to offend anyone.
The web on the other hand is the ultimate personal medium and one in which we all like different things, we all have specific preferences and opinions and a certain way we want to accomplish something. The beauty of the web is that all that is possible and cheaply with easily accessible technology. So why do typical Path Analysis and why try to “push” a certain way of navigation / browsing / buying? Why not get a deep and rich understanding of our customers and then provide them various different options to browse our website they want to and get to the end goal the way they want to.
Why not let democracy flourish? On our websites and in our customer experiences?
Agree? Disagree? Does Path Analysis work for you? Please share your feedback via comments.
May 26th, 2006 at 05:17
…Google Analytics also has this feature (please correct me if others do as well)…
Paths by "content group" have been available for quite some time in WebTrends, I believe ever since they implemented a "tag" version. Agree that path analysis is worthless without segmentation, but then again, (as you pointed out previously), just about any analysis is worthless without segmentation!
May 26th, 2006 at 05:18
Avinash, I can't disagree with most of what you say, yet I find that path analysis, at a low level (small sample) is incredibly useful to understand user behavior on the web. Granted, I still can't see when they clicked the back button and often don't understand how they can start on certain pages. But if I take a 100 random successful clickpaths and analyze them — not even analyze them, just read them — I start to have a richer understanding of movement online. I also start to see what customers *don't* do.
May 26th, 2006 at 07:28
Path Analysis: A Good Use of Time?…
Avinash Kaushik talks about the problems with Path Analysis which repeats what was said in Tradtional Web Analytics is Dead Podcast and Avinash and the 10,000 foot Analytics view. Listen in. Web Analytics is dead – Part II ; there's als…
May 26th, 2006 at 08:08
Jim: Thanks for checking out the blog and leaving a comment (just the kind of thing that will keep me going, hearing from greats in the industry such as yourself).
I think you are right but what is different about the latest clicktracks attempt is that in the "content group" you can get the usage of each page in the content group plus, and this is killer, the "influence" of each page in the content group in convincing our site visitors to move to the next stage.
This is simple in computation but awesome in insights.
May 26th, 2006 at 14:41
Hey Avinash – great blog! Just passing by and (almost) missing this industry… Take care -Xavier
May 26th, 2006 at 14:48
Offhand thoughts from the sidelines:
1) Adding to Robbin's comment… How about Path Analysis providing insight on user-brand/site interaction, awareness, involvement and therefore quality of audience? Thoughts?
2) To draw an association, say Path Analysis is a game of 20 questions – the user tends to or needs to assimilate certain points of information to think/feel about what is being read or seen, so that it can be actioned. If we were to work backward from desired site action (say sign up, purchase etc.) can path analysis tell us what those 20 questions or information points are, that ready up the user for an action? Given this, it only makes sense that content grouping is then a more logical way to arrive at level of importance of each factor to the user.
3) Given the movement to user-generated content, the question of "Why not let democracy flourish?" is very appropriate.
May 27th, 2006 at 00:13
Sulakshana, Thanks for the comment…..
The wonderful thing about analysis, vs reporting, is that there are always multiple answers and those answers are influenced by our experiences and perspectives. There is no right or wrong, just different.
The source of my perspective is that if our website is getting a million visitors a month then I would be on very thin ice inferring anything from the path (or think site interaction) of a thousand visitors.
This is especially true if you pause for a moment and think that any given website exists for multiple purpose, when you pick a small sample you increase the chance of seeing causality in things that you should not.
On brand / awareness / involvement I would switch from relying on Path Analysis to doing lab testing, get a few people in a room and ask them what they think. This is a bit better than Path because atleast you can ask them stuff and get into their head (even accounting for the Hawthorne Effect).
I stress that this is just a personal point of view, the great thing about our space is there is enough room for many different points of view.
It is midnight in the US as I type this and the illusion of walking backward in a user's path just seems so dreamy …… nice …. interesting. :)
Traditional Path Analysis, I think not. But as you rightly point out "influence rating", as outlined in the post above, in each content group I think certainly yes.
Thanks again for some thought provoking comments.
May 27th, 2006 at 05:47
Yes, that is "killer", I'll have to ask John Marshall what kind of algo he is using there.
I think we basically agree that "unfocused path reports" are useless. Personally, I think path analysis is critical but I never look at "paths" in the traditional sense of the reports, I only look at one path at a time defined by a specific entry page; that is, the entire report includes only one entry page. That way, you "force" the consolidation of visitor paths and you often do see "most popular path" hitting 20% – 50%.
The success of this approach depends quite a bit on the navigation, of course. If you have 100 link choices on the entry page it probably does not work as well, but I would argue that giving people more than, say, 10 link choices is insane anyway, and 5 is probably much better…
May 27th, 2006 at 09:40
Jim, Thanks for the follow up….
I think this idea quite insightful and you are both increasing the population but also using it in a sense like the exception example in the blog post. This is great.
Links on a page is one of the reasons path analysis might not give insights, not in the sense you mentioned but traditional sense. It is often not considered in analysis.
May 30th, 2006 at 19:37
:) Like they do in fine art, you turn a painting around to check if it is actually symmetrical; looking at it the right way up is something one gets used to.
May 31st, 2006 at 21:39
I'm so happy to read our design getting these compliments. We spent 10 months trying different prototypes and so many of them suffered from fatal flaws like too much complexity, or inability to compare segments side by side.
If anyone is interested in our approach and how the data expresses persuasiveness, and how segmentation works, please consider attending the next free virtual class in our series: http://www.clicktracks.com/seminars/
June 2nd, 2006 at 11:41
Very nice analysis, Avinash. And I certainly feel as though I am in the company of geniuses as I review the comments.
The path anlaysis has been a very misunderstood element of measurement, and always one of the most disappointing, IMO. I always see where clients that have analytics software with this feature always believe that they are missing something. I think because it looks to be a feature that promises valuable data, but rarely delivers.
I agree wholeheartedly that the analytics programs (Shout-out to ClickTracks) that allow the user to uncover "the long tail" paths by segmentation are the most valuable.
August 8th, 2006 at 09:49
(I just saw this post, but I'll give it my 2 cents anyway.)
When asked by clients how people navigate their sites, my customray answer is "Only in two ways: a) how you planned it, and b) not, the latter being the most common "pattern".
Apart from doing scenario analysis (in WebTrends; Fallout Analysis in Omniture, etc.) of preferably closed process (in which you can't go to step 3 if you didn't complete step 2, etc.), which is rather analyzing a pre-configured model than pathing, I too believe that the heuristic qualities of that type of analysis is very low, if not null in many instances.
I find the idea of determining (linear regression?) the influencial/causal weight of critical pages on certain actions (ex. completion of so and so) VERY interesting. So, instead of trying to base optimization decisions on vague pathing, we could identify which pages have a stronger propensity to "deliver" the expected behavior.
August 13th, 2006 at 21:03
[...] Avinash Kaushik of Occam's Razor says that path analysis of website visitors is not a good use of time. I know that in the past, I felt it was an extremely important indicator of a website visitor's experience. That is, understanding which sequence of webpages each visitor visited. [...]
August 22nd, 2006 at 13:01
I think your're spot on… you need to focus at the page groups, and then the effective pages within a group. This is a good paper on categorizing pages for analyzing: http://www.semphonic.com/resources/wpaper_005.pdf.
At each level, pages have different functions. If you look at the pages in groups of what they were designed to do, you can then analyze to see if they are doing what they were intended for.
November 13th, 2007 at 01:46
Currently, we are doing an analysis to determine the time gap of first visit to purchase, the purpose is to see if we can figure out something to make our 'pages' more attractive. But I think it is hard to get insights since it is hard to separate the time_band, there are so many unstable reasons that can cause the delay of our customer. How can I determine which time gap is more 'comfortable' for us.I hope i can get some suggestions from you upon how to dig the gold from this analysis, thanks for your advice.
January 27th, 2008 at 13:37
I work with several performers. In looking at their visitor logs, I like to track how many look at the calendar, from there, how many look at an individual show/date details and from THERE, how many buy tickets. It's not the same as a "next, next, next, checkout" type deal..but…if they get to the calendar after several prompts to do so, and ..1 in 100 buys tickets to a show, we have a conversion rate..KINDA. it's not science, really..but I was thinking that someone who is better with the log analysis tools than I am could make science otu of it :). Or is it just like Twain said: "There's lies, damn lies, and statistics."?
January 28th, 2008 at 09:22
Dana :
I think that makes sense, you are essentially trying to understand content consumption, or people moving from one "block" to the next rather than page1, page2, page3… This should be quite possible.
In fact you can use the Google Analytics funnel report to create a view that looks like what you want. Start there, possible next step is this and then that and then goal. You get the idea.
ClickTracks also had a great funnel report. I love the ultimate flexibility of it and the ability to have multiple pieces of content (individual pages or directories) in each step, and of course multiple steps to a goal. Check out this page:
http://clicktracks.com/funnel_report.php
or this flash video:
http://clicktracks.com/demos_small/funnel_report_demo.php
I am sure other tools can do this as well.
Hope this helps a bit.
-Avinash.
July 11th, 2008 at 03:06
The reason I found this article was as part of an investigation around performance testing. Such testing practically relies on having an idea of HOW 'real' users navigate your system (as opposed to just following the 'happy path'). OK, such analysis might not tell you WHY the various paths are followed, but they will help weight the tests/scenarios performed as part of load/performance testing.
September 17th, 2008 at 13:02
I like your point about using the data for personalisation (and qualitative studies). This is on my wishlist to present content to the customer (e.g. main benefits of a product) in a way that is tailored to their segment, rather than presenting/selling in the same way to all…
March 13th, 2009 at 14:08
[...]
Dan nog een algemene mening over het nut van pad analyse: het lijkt sexy, maar lees eens goed dit artikel van Avinash door. De korte samenvatting is dat pad analyse weinig zin heeft behalve wanneer de navigatie keuzes beperkt zijn (bijvoorbeeld in een bestel proces). Dit verwijst ook weer naar het 3de alternatief dat ik noem: de funnel analyse, en dan specifiek voor een bestel proces.
[...]
April 7th, 2009 at 15:09
Kudos on the article… Not sure if I am impressed with the knowledge here or how many super nerds actually exist out there. Call me if you want to talk nerd… 408 x
April 7th, 2009 at 23:43
[...]
Ga na waar bezoekers afhaken wanneer ze bijvoorbeeld iets willen bestellen? Is het proces te complex, het formulier te ingewikkeld of vraag je misschien wel te veel informatie? Heb je alle informatie wel nodig? En kun je de bezoeker misschien nog beter begeleiden en ondersteunen bij het invullen van formulieren? Het analyseren van funnels is echter alleen zinvol wanneer er een duidelijke doelstelling is en verdere navigatie grotendeels ontbreekt (denk hier dus aan bestelprocessen bij webwinkels). Immers er zijn zoveel mogelijkheden op een website om het gewenste doel te bereiken. Je zou namelijk in veel gevallen niet moeten willen dat bezoekers bepaalde pagina’s bekijken volgens een vast stramien om tot een conversie te komen. Dat het op meerdere manieren kan, is immers de (aantrekkings)kracht van internet.
[...]
May 14th, 2009 at 06:07
Is it possible to find complete path of visitor through google analytics.
I don't want to use Navigation or Entrance path which are only 1 or 2 level deep.
July 9th, 2009 at 23:27
[...] 在一个多页面复杂路径网络体验中证明了这样一个简单的事实:有的人访问了页面或者点击了链接,但这不足以为使用页面或链接计算转化率的方法提供任何帮助。如果你想计算页面的价值的话,请看这篇文章,关于page influence。 [...]
July 13th, 2009 at 06:09
[...]
在一个多页面复杂路径网络体验中证明了这样一个简单的事实:有的人访问了页面或者点击了链接,但这不足以为使用页面或链接计算转化率的方法提供任何帮助。如果你想计算页面的价值的话,请看这篇文章,关于page influence。
[...]
September 16th, 2009 at 00:07
@gargi: I use the Navigation Summary, take note of the most popular pages, then click on them and then take note of the popular pages again and so on, this way I can get the information for more than 1-2 levels. I then add this information into a mind manager program to see this information visually.
November 29th, 2009 at 17:32
[...]
表題の件ですが、
多機能なアクセス解析ソリューションには必ずあるものの、
あまり活用されていない機能にパス分析があります。
実際に僕も社内の事例を見ていても、広告や集客の分析に比べると人気が少ないようです。
そこで、パス分析に関して、
アビナッシュ=コーシックのブログの中で少し古い記事ですが、
Path Analysis: A Good Use of Time?
こちらの記事を紹介します。
ちなみにこちらの記事の内容は一部和訳されて[...]
January 21st, 2010 at 12:32
[...]
Bestaat het ideale klikpad? Zo als Avinash Kaushik al eens heeft aangegeven: nee. Binnen een kleine site zijn er misschien nog wel een aantal veel gebruikte paden te vinden, maar bij grotere sites zijn de navigatie mogelijkheden vaak zo uitgebreid dat er geen ideaal klikpad best
[...]
December 23rd, 2010 at 03:24
Hey Avinash
Great Post.lot of patience needs to read your post.
Indepth analysis reflects your dedication that you put in each of your post.