Keys To Web Analytics Maturity: Structure, Process, Hyperfocus
Take two months to tag the website with JavaScript code (nine months for "high end" tools). Realize you did some things wrong and in nine months the company now has new needs. Go back, roll up your sleeves and spend more time on re-implementation. Log into the tool. Start spewing forth glorious data. Be sad no one in the team cares. Start creating pretty Excel files. Send. Still no one cares. Start publishing prettier PowerPoint files with pretty Excel screenshots. Still No One Cares. Feel frustrated. Tweet that. Feel better. Still no one cares.
Sounds like your life story as a Web Analyst?
Perhaps not all of it. After all you are unique. :) But surely parts of it, and most surely the outcome.
Make no mistake. . . we all work really hard with the best of intentions. Yet we end up having negligible impact in terms of creating data driven organizations.
Why?
As a practitioner my root cause analysis (and ishikawa diagram!) indicates that the problem is not data. The problem (in some cases at least) is not employees or consultants. The problem is not that there isn't a need for data.
The problem is that we go about doing all of this the wrong way.
The structure we use is. . . I was going to say wrong. . . but the reality is there is no structured approach.
The process we use (above) is flawed.
It can't start with data spewing. Or with implementations. Or arguing about which tools to use. Or figuring out how to just make data pretty. Or going to analytics conferences to bitch. Or copy pasting "best practices."
We need a different process to help create data driven organizations.
And we need a structured approach to help guide that process.
The answer to those two golden requirements is very simple to grok, yet so few do it.
In this post I want to build on a secret mission I have been on in the last few months on this blog. I've been sharing real world concrete examples of how you can take a very structured approach and follow a very clean process to maximize the chances that you'll be set up for success when it comes to getting your company to use data to make informed smart decisions.
The "secret mission" I am referring to is a recent series of posts that have touched on non-data challenges Web Analytics faces. The mission highlights unappreciated hidden reasons why online marketing is still a faith based initiative. It encourages you to answer only business questions by identifying attributes and giving you specific examples (questions before data!!). It provides you with a framework to identify who should own web analytics in your company.
The mission extended to pushing you to consider using the structure and process outlined in the Web Analytics Measurement Model. It dictates ignoring all data, reports, tools, what can be measured, what can't be measured, the political infighting and backstabbing to kick things up to the highest notch.
Here's the model, so simple, as outlined in an email! (Details in blog post above.)
Starting from what's supremely important to the business to setting goals to identifying metrics to identifying success parameters to ensure focus. Actually not just focus, hyper-focus when it comes to our data efforts.
My hope to get us to think differently perhaps peaked with outlining the Clear Line of Sight Model for hyper-focus and prioritization.
No longer will our data be ignored because we are data puking or doing non-contextual or only what we think is important "analysis." Hyper-focus! By having a direct visible line between our Key Performance Indicators and the company's Net Income no one can ignore our work!
[Bonus Reading: In the above post about the web analytics measurement framework please read, digest and action the seven steps outlined in the "Win With Web Metrics" section. It will change your life!]
Notice a common stream in all the "secret mission" posts? Not one of them was about tools or capturing data or stroking your inner JavaScript God man-child.
They were all about radically focusing on the business, strategy, meeting with CMO's and Finance, making love to the Marketers and Sales folks, and understanding everything about what surrounds your work before you say "Hello Omniture. Baby I have missed you. Let's get hook up and get dirty!"
: )
If you agreed with any of that (and come on how could you not!) then you are no doubt chomping at the bit to get going.
You are going to absolutely love what comes next. Trust me.
As you know I am the co-founder of Market Motive Inc. We offer certification courses in six internet marketing disciplines, including. . . wait for it. . . Web Analytics! It's a three month course consisting of structured lessons, quizzes, exams, and meetings. You learn a lot and have to work very hard.
At the end the course you to have submit a dissertation with qualitative and quantitative analysis of two real world websites. One non-profit/non-ecommerce and one ecommerce. Our goal is simple: John and I want to know if you've absorbed the course material from not only a "here are reports, frameworks and metrics" perspective but also from a "I can now think like a marketing-data hybrid ninja" perspective.
Grading these submissions, and then doing the live, often torturous, Q&A, is a lot of work but also a time of great pride to see the student display awesomeness. Not everyone makes it unfortunately.
One such student who did make it this past quarter was Chessie Little.
She was exemplary on many fronts but I particularly loved her dissertation, specifically the approach she took to doing her analysis. With her permission I want to share it with you here hoping that it will inspire you to apply more structure and follow the optimal process when it comes to your company / non-profit / government entity and data.
[Context:]
Chessie's non-ecommerce website was a real estate website. What made it tougher was that the site was not owned by a builder but rather an entity whose job was to drum up interest from a population wanting to purchase a home in a certain area of a major American city. This site provided information and collected leads which were then funneled to the various builders who were developing the said area. Think of tough sites to do web data analysis for, this is right up there with the hardest nuts.
Note: I could not figure out how to shrink Chessie's submission to fit the width of this blog and ensure clear graphics (due to resizing). Hence I've recreated the pictures you see below in PowerPoint. But that's the only change. All the words and boxes and flow are Chessie's. All credit belongs to her.
[/Context:]
The dissertation opened with the one thing we in the web analytics world very, very rarely do: Identifying the business objectives.
Here's the very first picture we all need to paint. . .
Identifying the business objectives mandates a discussion, multiple discussions, with the senior-most leaders in your company and working with them / sweet-talking their egos with gentle encouragement, to identify why the site exists.
Based on those discussions Chessie identified three objectives: Create awareness, generate leads for the builders and highlight community events.
My recommendation in the past has been that objectives should be DUMB:
Doable. Understandable. Manageable. Beneficial.
Mission accomplished for Chessie. Her objectives are DUMB! Hurray!!
The second picture she painted was to drill down a bit more into the specifics. Objectives are usually handed down by the Management, you just have to do the aforementioned sweet talking.
Drilling down to identify Website Goals requires just a little bit more work from you, with the majority of critical thinking still coming from Management and Marketers (usually from multiple divisions – you get to know all the stakeholders if you do this!).
My definition: Goals are specific strategies you'll leverage to accomplish the business objectives.
Here's Chessie's effort. . .
Clean. Breaks down components of the business objectives. Provides clarity.
To deliver on "Create Awareness" the site needs to support the massive number of offline efforts and have an online relevant traffic acquisition strategy.
"Generating Leads" comprises the twin goals of providing all kinds of information that will help potential home buyers to make their decision and to collect e-newsletter registrations as well as e-requests for an onsite tour of the model home by the builder.
Finally, "Highlight Events" is not for prospective home buyers but for existing residents of the communities. By making them happy with delightful events they can be converted into Net Promoters whose recommendations will attract new prospects organically.
Crystal clear goals that cover ALL the jobs the site is doing, and both key customer constituents – current and prospective.
Now that we know what the Goals are it is time to pick the Key Performance Indicators (data, data, data, finally, data!). The work balance will shift to you as you are the expert here, with Management in a guidance role.
If you follow this structure then you'll end up on this picture from Chessie. . .
My rule for web metrics that are identified as KPIs should be:
Uncomplex, Relevant, Timely and Instantly Useful.
[Bonus Reading: Web Metrics Demystified]
You can see that the KPIs chosen above will help Chessie's client use web metrics to identify how website goals are performing, and that they meet the four rules.
Sweetness.
Many would stop here. That would be wrong. And Chessie did not.
The right thing to do is resist the siren song of Google Analytics or Webtrends. You have not established what success or failure looks like, even if you have all the data.
Time to do what Chessie did: SET TARGETS!
Most often you as an Analyst will play a supporting role here, providing historical performance data etc. The decision-making will be done by your senior leadership and, if the company is big enough, the Finance and Sales teams.
People who are accountable will help you identify targets for each KPI. . . .
Hurray!
Now everyone knows what the company is shooting for. When you crack open CoreMetrics you'll see data and jump with joy or weep when you see the number. You'll instantly know what is good and what is bad.
I wish I could tell you how important it is (it truly prevents data puking).
Done?
Not yet. At least, not Chessie. This last part is one that is particularly meaningful to me.
When you log into Omniture or Google Analytics or IBM Surfaid you are about to be deluged by data and you could go in a million different directions.
Remember the top of the post: We not only wanted focus, we wanted hyper-focus.
Take 10 more minutes from the people around you — the business people, the marketers, your boss — do one more thing.
Have a discussion with them about what are the most important segments to focus on when it comes to each goal. Not just the super lame New vs. Returning Visitors or Search vs. All or Browser Versions segments!
Chessie's dissertation contained this last picture next. . . (note for visual clarity I have eliminated one KPI you see above). . .
What group of visitors were important? What behavior was key to winning? What traffic sources was Marketing focused on? Who are we trying to attract? What on our site is important – at least according to us? And more such questions. [See: How To Identify Web Analytics Segments]
You will thank God and your Mother and your Cat and the Chickens and the Moon that you did the above step. Because my dear what you now have is. a structure that will guide your measurement efforts in a way that your insights will be of value and used to drive action.
Hallelujah, praise the lord you are done!
If I had not eliminated the Conversions KPI the Web Analytics Measurement Model would look like this (slightly cramped but still readable). . .
Shake on this. Meet your management team, or your client, and shake your hands on this "contract." If you can get them to sign it in blood, jump on that chance!
Here's the sexiness: You now know what's important and where to start and what to focus on. Your boss/client knows what success or failure looks like and how to connect Her/His business objectives to your data. Prioritized business focus for relevant data analysis.
How can there not be mutual love and admiration?
Do you see why I was pleased to see Chessie's dissertation?
It displayed a savvy business understanding and an ability to set her measurement effort up for success. Exactly what I want to see in anyone carrying a Market Motive Certification! It is what we so passionately hope to teach.
Punch-line: Always, always, always work with the above "analytics contract" in hand.
Print a copy of it and stick it on your cubicle wall. Make sure it is the first thing you show in any presentations you do. Revise it as needed (test for relevancy at least once a quarter, at least the targets will change if nothing else). Use it to bring a shared vision between all the stakeholders. When there is office politics and backstabbing, and there will be plenty of that, hold this document as the arbitrator of what direction to follow. Use it to break ties.
Of course the creating your own Web Analytics Measurement Model is not the end of your Web Analytics Maturity, just the optimal start of the process.
By ignoring your data itch and ensuring this type of deep engagement with your management team and stakeholders, upfront, you have set yourself and your company up for long term success.
Once the Web Analytics Measurement Model is done. . . it is time to finally begin your Omniture loving and Google Analytics adoration and WebTrends hugging!
The hard, tough, sweaty work you'll do will include, as illustrated by these pictures from Chessie's dissertation, a correlation between the huge $$$ of offline advertising (newspapers, magazines, radio) and the website traffic to see if there is any there there. You'll throw in your online spend (display and PPC) for good measure. . .
Correlations? Any causality? Makes you wish you had done true test and control right? : )
You've already established in the Web Analytics Measurement Model that a primary objective of your real estate website is to produce and publish relevant content.
Time to see if there is any connection between what you are producing (pimping) and what visitors to your site actually want. Boom! Here it is. . .
Having already identified your segments for conversion rate and what is dear to you from an acquisition perspective you'll surely spend some time with segmented conversion pictures like these. . .
The great thing is it will become easier and easier to jump to "what do I have to do next?" A key part of the strategy, as identified by the measurement model, are the tools you have built on the site to convince visitors to submit leads to you after being charmed by your beautiful tools and gorgeous data. So you'll do this. . .
And many more such things. Just like Chessie.
In the end, no blow-out preventer in the world will be able to stop your awesomeness from oozing out in the form of a relevant dashboard with key bits of data and insights clearly laid out. . .
[Note: In order to preserve anonymity the above dashboard was not Chessie's. The dashboard above is for illustrative purposes. Still. . . it is good, you should copy it!]
A dashboard that came from you following a specific structure, which drove a very specific process, to bring about the gift of Analytics Gods: Relevancy and Hyperfocus.
For you to be successful in web analytics you can't start with random data or random "dashboards" or data puking or this. . .
If you want to be promoted, if you want your company to be successful, if you want your contract with the client to be renewed, if you want to get any kind of satisfaction from your job. . . you are going to have to start with this. . .
A Web Analytics Measurement Model.
It is not easy (like data puking). It takes relationships and efforts and love. But it is your only path to success.
Good luck!
Ok it's your turn now.
How does your company / client function? Do they take a structured approach and a specific process to focus? Does your approach work? Do you think the above approach will work for you? Why? Why not?
It would be immensely gratifying to get your feedback / critique / perspective.
Thanks.
PS:
Couple other related posts you might find interesting:
November 1st, 2010 at 05:52
What I love about Chessie's model is that it rigorously stays aligned to business needs.
Why is her work so valuable? Because it directly answers the important questions. And segmentation breaks down the performance, exposing what works and what does not.
Nice work, thanks to Chessie and Avinash for sharing :)
November 1st, 2010 at 05:58
We try to use the correct analytics to track our website performance and hope we are doing things that will provide us some result..but we do make mistake.
I'm just hoping to minimize those by reading your post here.
TrafficColeman "Signing Off"
November 1st, 2010 at 06:30
Which software was used to create the "cheesie" dashboard towards the end of the post?
We'd love to get our hands on data presentation like that…
Daniel Shaw
November 1st, 2010 at 06:54
Nice post. Reinforcing the need for an actual framework that communicates real value to senior management is always a best practice.
I like your student's effort and there is no doubt she understands the importance of connecting web analytics measurement to real business outcomes.
Congrats on graduating another "analysis ninja"! Keep up the good work.
November 1st, 2010 at 11:49
This is a great framework. Often times stakeholders just see the end result and aren't aware of the process and analysis it took to arrive at the handful of KPIs that they are being presented.
This framework tells the narrative of how those metrics were derived. Thanks for sharing.
November 1st, 2010 at 22:59
Daniel: The dashboard shared at the end of the post was created using Excel. It is not very hard to do, just some medium Excel sophistication should suffice.
Marc: My perspective was that this approach will help analysts become better at picking optimal metrics and focus better.
But you are right to point out that this process will also do a great job of illuminating for the management some of the "black magic" (and effort!) behind the work we Analysis Ninjas do.
-Avinash.
November 2nd, 2010 at 09:27
A year or two ago, we emailed back and forth about what I called "website measurement plans." Your website "Measurement Models" as described above is exactly what I was talking about.
Thanks for doing this post. Like you said, doing it any other way is backwards.
November 2nd, 2010 at 11:36
Avinash – I have to thank you (as always) for a great post, but thanks to Chessie this week as well.
I'm working right now to create a dashboard for upper management. Taking some lessons from an earlier post, I managed to keep everything on one page. I even told my Director that she couldn't have certain metrics because they didn't matter (I did this of course with a super huge smile on my face just like you suggested).
What excites me is that you're right about everything. Management doesn't want a data puke report. They want insights and a path forward.
And through this framework, we can help management identify what to measure AND what to improve in a way that keeps us from chasing our tails and puking out data. You've given us a path to be heroes, really :)
November 2nd, 2010 at 11:38
Avinash your blog came in very timely. I am working on a similar model for our web analytics strategy and this is very helpful to approach it in this way.
Many thanks for sharing something so simple yet very impactful.
November 2nd, 2010 at 15:39
Avinash,
I agree with Raghu, came very timely.
Many thanks to Chessie (congrats on the 'analytics ninja' graduate!) for sharing and you Avinash for posting.
Bravo!
November 2nd, 2010 at 19:56
WOW!!! Avinash, this is truly inspirational stuff.
I am new to the web analytics game, but the first paragraph of this post sums up my experience with analytics to a tee…. do you have a camera at my desk?!.
I have spent many hours on this site (in addition to reading your books and viewing your youtube channel with Nick who is super awesome as well) and am realising how much I have to learn.
Keep up the great work, and if you ever come out to NZ, rest assured you have one dedicated fan here :)
Prem
November 2nd, 2010 at 23:40
@Daniel pages 288-293 of Web Analytics 2.0 By Avinash for more information on those 'action dashboards.'
November 3rd, 2010 at 02:22
This is one of the best blog posts I have read in a long time :-)
The model above is so simple to follow and as you said, it makes your job as an analyst a lot more focused and effective once you have all these KPIs, targets and segments in place.
There are many great ideas I will be taking from this so thank you for sharing. That dashboard at the end rocks too :)
November 3rd, 2010 at 15:28
Wow…this post is full on awesome.
Disseminating and getting buy-in to your analytics results (or the results of any data-gathering activity) is defintiely a challenge.
I like your model because analytics becomes the foundation of your efforts (part of the basic questions you ask up front) and a continual guide in optimizing the tactical side, rather than something tacked on at the end as mere "reporting" which few people really read.
I didn't have time to digest the tutorial section yet but I look forward to reading in more depth.
Thanks!
November 4th, 2010 at 05:38
Hi Avinash,
Like many have said this is very timely for me, starting to be hassled for unnecessary KPIs. What's worse is my organisation is run by analysts who should really know better.
Cheers,
November 4th, 2010 at 08:23
Great post!
Chessie's set up seems pretty awesome.
I especially like the 1 page digestible HiPPO report you posted at the end as well. It comes close to the Action Dashboard example you gave in you Web Analytics 2.0 book on page 289.
The problem I'm having is guiding individuals on setting targets that aren't completely arbitrary. In the past, I've personally had to go on a witch hunt to different individuals to figure out "where leads go" once they leave the confines of the internet. From that effort, I get an idea of how that type of lead affects the bottom line for the business ($$). Then finally, I can try to prioritize the leads we're going for as well as set the targets appropriately. Is this a case of going for accuracy and not precision?
November 4th, 2010 at 10:21
Brian: I would humbly postulate that, at least initially, it is a case for precision (Web Analytics: Accuracy or Precision?). If you can go after them once in a while, say once a quarter, and get the numbers connected from online to offline conversions (leads to customers). Then you can use that "conversion factor" to better understand how good a job the site is doing. Over time you can try to see how to move to accuracy.
With regards to getting management support… one trick you could try is quantify the impact of not tracking the "where leads go". For example: "We spend 500k on display and we can't measure how best to spend it as we don't know final conversion/profit." Or "I could reduce our cost/cpa by 50% if only I knew how many leads convert." Or something like that. Show the pain with the hope that someone appreciate the gain and give you want you want! :)
-Avinash.
November 4th, 2010 at 10:24
Avinash,
Excellent Post!! One worth hanging on my wall.
I would love more detail & explanation of the Advertising Impressions and Branded Website traffic chart.
It amazes me how frequently website owners are stumped when you ask them what the objectives or goals of their website are. I tend to see a lot of confused looks on their faces and often have to ask lots of follow up questions in order to extract those answers from them. What tasks do you want visitors to your website to complete? etc…?
I do have a queston on the Targets presented for the KPIs and their relationship to the segments. Are the targets overall traffic site targets or are they targets for the segments identified? Or am I interpretting the segments incorrectly? Meaning…are these segments that we want to keep a close eye on because they are of interest to our clients or are these segments that have been identified (through data and insights, of course!) as being ones have stood out in terms of performance?
I also thank Chessie!
Happy Belated Halloween!
Alice Cooper's Stalker
November 4th, 2010 at 10:50
Alice Cooper: I can't share more specific information about the Ad Impressions and Branded Visits chart, to preserve anonymity of the client, but the exercise undertaken is similar to what was advocated in this blog post:
~ Excellent Analytics Tip #12: Unsuspected Correlations Are Sweet!
Essentially identify current media spend by channel (offline and online) and pull together the media data (impressions, ad run times, cost etc) with the online data (visits, conversions etc). Find insights.
[Correlation is not causation! But it's the start of the process of figuring out causation.]
Regarding targets… In this case since it was a brand spanking new analysis the targets are set at a overall KPI level. So, to take the first one, 7,000 Visits per month from Branded Traffic.
What Chessie and I would do is report the Branded Traffic, see if it is "good" or "bad" when compared to the target. Then drill down into the segments (Google, Bing, etc).
Once this pump is primed and we have been doing this for a couple of months we will want to create targets for individual prioritized segments. So of the 7,000 what is our target for Google? What is it for Bing? Etc etc. That then further focuses our effort, further helps understand what is good or bad and when to freak out.
I generally encourage people not to create targets at the start at a individual prioritized segment level because 1. you have no idea what you are doing 2. you will take too long to do it and I would much rather you start analysis at least macro data. : )
Makes sense?
Avinash.
November 16th, 2010 at 05:01
Hi Avinish,
Thank you for this excellent and inspiring post! I am just in a process of creating this model for a company I work for.
Can it be created also for the key pages? Like homepage, product detail page etc.? Or is it only for website as a whole?
November 16th, 2010 at 16:51
Michal: The model is very much a business model and in as much would apply to your macro strategic objectives and then drill down to strategies, where home page or other pages could be a part of tactical steps.
Avinash.
November 18th, 2010 at 04:04
Hi Avinash,
I just wanted to thank you.
I'm learning to approach web analytics in a more "compelte" way day by day, putting valuable insights on my reports and not just sterile data. The road's long for me (i'm still a n00b), but i hope one day to approach the level of a WA ninja :)
Thx again for your efforts and your great job
Giorgio
January 24th, 2011 at 14:40
We are almost an out of college startup and in our case I myself would be the management and the analyst both.
I was looking for some posts randomly so as to start with google analytics. After this post I know where to start -> My office whiteboard :)
March 2nd, 2011 at 05:36
Dear Avinash,
I've read your paper named "Web Analytics 2.0: Empowering Customer Centricity". I noticed that in your figure "Fig. 2: The Web Analytics Process" you make the repeat arrow back to the step: "Analyze Data".
Shouldn't it be turned back to step"Collect Data" ?? I think if it turned back to it, this will allow collecting the new data after applying the changes??
What's your point of view?
Thanks..
March 3rd, 2011 at 10:05
Hi Anthony,
I had the honor of writing with Avinash the paper you mention. Our reasoning behind that figure is that we believe that website owners should always focus their attention on analyzing and (especially) implementing insights, since those are essential to improving conversion rates.
However, you are right that we should review data collection once in a while in order to check the accuracy and completeness of the numbers we use to extract insights. I published a Web Analytics Process reviewed in which I added such a step. I would love to get your feedback on it too.
But I want to reinforce that I still firmly believe that Analysis, Testing and Insight Implementation should be the focus of website owners. There lies the gold mine, the place where you can really affect the bottom line of you company.
May 24th, 2011 at 02:25
Thanks a Ton for this wonderful post. I'm delighted for understand the steps to create a proper measurement model for my organization and in the process of adapting this model for my webmarketing process
Will love & request you to include this type of examples or case studies in coming days on different topics including Web Marketing Management.
Admire the dashboard (Aug 2010) at the last, which shows a graphical representation with data’s at the top & graphs in the down with guidenlines below to it. Hope my team members will come up with a reporting template simillar to this matching our objectives.
Last but not least, though the article is old by few months, but it came down to me too at the right time.
You are the Best, Avinash.