November 2009


24 Nov 2009 03:28 am

toughTwitter is amongst new media channels that are challenging how we communicate, with whom we communicate and perhaps most fundamentally how we (Marketers) influence people.

Sadly execution and analysis of these new social media channels has been hobbled by old world thinking. When it comes to marketing because of the old world thinking from the worlds of sTelevision and Magazines, and when it comes to measurement because of the world of traditional web analytics.

These new channels, Twitter and Facebook and YouTube and Tumblr and, yes, even blogs, are very distinct customer / participant experiences. Stale marketing or measurement thinking applied to them results in terribly sub optimal results for all involved.

So in this post my hope is to share with you what is unique about measuring one such channel, Twitter. The blog post is also sprinkled with my own words of folksy wisdom as to how you should use the channel for maximum impact.

My new book Web Analytics 2.0 covers social media measurement, but I am going to cover something very different in this post.

First: An Ode to New Thinking:

One common thing between the all tools in this post is that they were built by "outsiders".

One of the things I love and adore about Twitter (besides all that connection and conversation) is how its open API has lit a fierce fire of innovation when it comes to analytics. Anyone and their brother and ma-in-law can develop a tool, and they have! Much to the benefit of the rest of us.

Perhaps the most beneficial thing to me is how much out of the box innovation this has brought.

For example just look at traditional web analytics tools, there is absolutely no fresh thinking when it comes to Social Media Measurement. Their constant focus is on "let's figure out how to collect and report ever more data and not bother with a truly immersive understanding of these channels and what makes them unique". That mental model is, sadly, extremely clear in the metrics and analysis they provide with "twitter integrations".

While there is some stale thinking in the new twitter tools, most of them have a lot of fresh thinking from people untainted by Omniture or CoreMetrics or WebTrends or, ok ok ok, Google Analytics.

I consider this massive proliferation of new thinking to be a gift from God.

To all of you developers who are toiling out there, you have my love and gratitude.

In this post four twitter analysis tools that while not yet fully developed show sweet signs of:

1. Truly understanding the medium and uniqueness of the channel and

2. Are not just reporting "hits", rather coming up with clever metrics.

Quantitative Metrics / Analyses.

Most twitter analytics tools just do data puking. They find numbers that can be computed and then proceed to puke at you as many as they can find, with wonton disregard of value being provided or outcomes being measured.

Here is one of the mild ones:

twitter data puking

You must pause and think: So what is this saying? What action can I take?

Always, always, always ask that question when faced with tools that simply puke data out at you (twitter or Google Analytics or whatever).

But as I mentioned at the start of the post one of the beauty of twitter's open API is that there are a few pockets of truly innovative thinking.

Here are some that I humbly believe look promising. . . .

Klout. Twitter Analytics.

Klout is a wonderful little tool that measures Klout Score, a proxy for "influence":

klout score formula

It is easy to understand the market demand to boil things down to one number, but this is perhaps the least useful thing in Klout.

While on the surface they might seem useful, I am always suspicious of compound metrics. They can be subjective, inapplicable to many and efficiently hide the insights you need to understand what actions to take. [See more here for Compound Metrics: Four Not Useful KPI Measurement Techniques]

Mercifully there is so much more to Klout than that.

Klout measures a bunch of lovely metrics, specifically applicable to Twitter, that are grouped into four buckets: Reach, Demand, Engagement (!!) :), Velocity.

klout reach demand engagement velocity

There are two lovely things about these computations.

1. Joe and team have wonderfully avoided the temptation make these compound metrics (as in Reach = Followers / Total Retweets * Friends + Pixie Dust). The factors used are laid out as individual metrics making it easy for you understand the data and pick metrics that work for you.

2. (My favorite) The metric definitions are not "crap". This seems like such a low bar to meet, sadly far too often metrics out there (not just for twitter) are just plain shoddy.

For example here are some clean definitions from Klout:

# Engagement

* How diverse is the group that @ messages you?
* Are you broadcasting or participating in conversation?

# Velocity

* How likely are you to be retweeted?
* Do a lot of people retweet you or is it always the same few followers?

# Reach

* Are your tweets interesting and informative enough to build an audience?
* How far has your content been spread across Twitter?
* Are people adding you to lists and are those lists being followed?

When I use Klout I simply pick the metrics that are most important to my own twitter strategy.

I would suggest that this is very very very very important, pick what is right for you rather then following a lemmings like strategy of "I am going to use metrics Y & Z that someone recommends".

Here's an example: I don't care about Follower/Follow Ratio. I think it is disingenuous to follow everyone who follows you just for appearances sake when you have no intention of reading what they all say. Why be fake?

As you might have read in the new book I like "Message Amplification" in Social Media, and hence I do care a lot about Total Retweets.

[Sidebar: my favorite twitter metric is: # Of Retweets Per Thousand Followers, it's a measure of efficiency and value provided and people voting with their clicks, all rolled into one!]

I care a lot about Follower Retweet % ("Do a lot of people retweet you or is it always the same few followers?") because I want to appeal to more people than my mom, dad, and best friend!

One of the biggest mistake companies and brands make about Twitter is that they think it is one more "shout channel" like TV and Radio and Magazine ads or Press Releases. Twitter is not that. Twitter is a "conversation channel", a place where you can find the audience relevant to you (and your company and products and services and jihad) and engage in a conversation with them. It is not pitching, it is enriching the value of the ecosystem by participating.

Hence I like the metric Messages Per Outbound Message , as a primitive measure of the fact that you are participating in a conversation and not just yelling.

With Klout I can choose the metrics that best reflect my personal twitter strategy, I can easily find them and I can monitor my progress (using a handy dandy graph) and ensure my strategy is a success.

Your strategy might be different. Walk up to the buffet and pick the metrics that will help you best measure your own success.

[ Contest: Notice the metrics I have deliberately ignored: # of followers, # of retweets, @ mention count etc. Can you guess why? :) The person with the best guess gets a copy of Web Analytics 2.0! Contest closed, thanks for the entries!]

At the bottom of the Stats tab Klout also includes a handy dandy Analysis table with trend indicators. . . .

klout analysis

As an Analyst it might be of some value to look at the trend pointers at the bottom (clearly I am doomed!), it might be cute to put this into a PowerPoint slide for the HiPPO's who might like the Chinese fortune cookie messages for each metric group.

Ok, ok, I am just teasing the Klout team, I know it is very hard to "wordify" and programmatically make valuable recommendations. :)

GraphEdge. Twitter Analytics.

The reason I believe GraphEdge is interesting is that it has a set of really cute metrics that help bring a different perspective to measuring Twitter.

If want to contrast the difference in thinking applied compare some of the metrics below with, for example, what Omniture is touting with its Twitter "integration". The difference between the old web analytics thinking and a new person's could not be more clear.

[Allow me to rush and add that while Omniture has a hack to bring some twitter data into Site Catalyst to do something, Google Analytics has nothing. Not even something that is not useful. So perhaps GA stinks even more.]

Here are some, IMHO, differentiated metrics. . . .

Legitimate Followers:

If you have spent any time on Twitter you know that spam accounts are a problem so it is very nice that the first thing you see in GraphEdge is not a follower graph but rather their attempt at telling you how many legitimate followers you have (and the trend over time, cropped out in the image below). . .

graphedge legitimate followers

To identify "legitimate" they use the following filters, direct quote:

Any of your followers who are following more than 2,000 people are considered not-Legitimate… they’re probably not really monitoring your feed, so we don't count them as "Legitimate".

Users who have been suspended by Twitter can’t read your tweets (and probably weren’t interested in the first place!). We don’t consider these Legitimate Followers.

It is ok to argue with their filters, but it's a fine start and I think good enough.

Klout also measures something called "Reach", which is also their way of identify if you've got people or bots following you.

Churn Rate:

In my days at DirecTV one of the metrics that the company was obsessed with atleast then and rightly so, was Churn Rate. It reflected the value of not just going after new customers but doing all that was possible to take care and love the customers we already had. Makes sense?

So I have always had that obsession with tracking Churn, simply to try and understand why people quit. The hope is if I can understand why then I can do something to fix the problem.

[By now I am sure you get the feeling that I am treating twitter analysis like I would business analysis. Twitter is my brand channel and I take this very seriously. It is perfectly ok to use Twitter to tell people where you are and what you are doing and not care about analysis.]

I have not really found any decent tool to track unfollows in twitter (yes I have tried the normal ones and they are either flaky or just outright stink). Hence I was happy to have this in GraphEdge. . . .

graphedge follows unfollows

Yea!

Actually ouch! 291 un-follows!! So sad.

Atleast now I know.

It is nice to have the over all trends on the right in the above image, as well as for the period you choose smarter metrics like growth rate.

GraphEdge will also show a list of your new followers and un-follows (so you can send them bad vibes! Kidding, Kidding. :).

graphedge un followers

[Qwitter was one of the first tools I used to track unfollows, sadly it does not work any more, and it had a great feature: It would try to guess and report on which tweet resulted in the un-follows. Nice.]

And here's what we were on the quest of. . .

graphedge churn rate

I will admit to not being charmed by having three different lines above, they clutter the left and the right, and get in the way of understanding the data. But you I suppose you can learn to ignore two of them.

Here's the definition from GraphEdge for Churn:

The number of removals (un-follows) over the average size of the existing base (followers) during the period measured:

Drops / (Current Followers – ((Adds – Drops) / 2))

As always look at the trends, the longer term the better. And remember that history is littered with companies that were growing just fine but they still died a painful death because of Churn Rate.

Loyalty:

Slightly along the same lines GraphEdge has a metric called Loyalty. At the moment I think it is too limited in what it actually measures, and it only starts measuring once you join GraphEdge. But there is kernel of promise in the metric, keep an eye on it.

Second Level Network Size:

Lastly… while most people overestimate their "twitter power" (I can bring you down with a single bad tweet Avinash!) I think a few people also underestimate their reach, if they participate in twitter in the right way.

Looking at the second level report can give you a feel for your network size.

graphedge network size

Followers' Friends is an "incestuous" number, it shows all your followers and the people they are following. If you have ten followers and they all follow each other that's 100 Followers' Friends. Feel free to be proud of this number, but then promptly ignore it.

Unique Names is are the unique twitter account id's in the network, less the "illegitimate" ones. Think of this as something close to, but not the same as, the Unique Visitor concept in web analytics.

This is a useful number.

Think of it this way: If you say something of incredibly profound :) . . .

avinashkaushik social media

. . .and a whole lot of other people who follow you think that and retweet it then you have a theoretical capability to reach 1.2 mil people (Unique Names in your Second Level network).

Now the reality is that that will rarely happen, if ever, but in our profoundly hyper connected world Unique Names is a good number to keep on your horizon.

Remember success in twitter comes from participating in the conversation and giving something of value, not by running "social media campaigns". If you don't internalize that be ready for a reality where both your Followers, and Second Level Network size, to be small potatoes.

Qualitative Metrics / Analyses.

Now let's tackle the much much harder analysis to do in any filed, analyze the data from a qualitative perspective.

Linguistic Analysis:

I have given up on "Sentiment Analysis". Well atleast for now. Everyone over-promises and massively under-delivers.

On paper it seems like such a great thing to want to have, this is a social / conversation medium after all. But most tools I have had the good fortune to try are simply either glorified versions of Google Alerts even if they promise you buzz metrics and the moon.

Now sentiment analysis is a very hard problem to solve. For example I just analyzed my account using an expensive "social media sentiment buzz analysis tool" and it marked this tweet from today as Negative:

avinashkaushik: There in nothing quite like AC power in your seat for a 10 hour flight. Oh and 20 hours of pending work to do.

Perhaps the tool does not find my dry wit as funny as I do, but it's hardly "negative"!

With all that context I think TweetPsych holds a lot of promise.

Tweet Psych uses the Linguistic Inquiry and Word Count (LIWC) method and the Regressive Imagery Dictionary (RID) method to build a psychological profile of a person based on the content of their last 1,000 tweets.

Dan Zarrella, founder, says: "I think the possibilities of a system like this are enormous, from matching like-minded users to identifying users that exhibit certain useful or desirable traits."

I am not sure I understand perfectly how it works (I need to send this to Joseph Carrabis!) but the analytical techniques looks very promising. . .

tweetpsych cognitive content avinashkaushik

Hmm… interesting. I do like talking about "learning, thinking, knowing etc"! :)

As always rather then looking at my data in isolation I compare / contrast it with my friend who is a web analytics twitterer. . .

tweetpsych cognitive content b

Now I understand a lot better how I am doing and how he is doing.

Remember there is nothing wrong or right here, we are both just very different people with different twitter strategy and what Tweet Psych's linguistic analysis algorithms helps us understand if our psychological profiles are aligned with our twitter goals.

Tweet Psych also provides you with Primordial, Conceptual and Emotional Content analysis, here's mine. . .

tweetpsych primordial conceptual emotional content

Nice.

Use this type of analysis to understand at a deep level what attributes are being associated with your brand, and if they are reflective of the goals that you set for yourself.

Content Visualization with Stream Graphs:

Stream graphs can be very good at visualizing data, content specifically. Twitter StreamGraphs is delightful for:

1. its visualization of highly associative words with the word you are querying and

2. viewing streams (tweets) for any associative word (hence sweet filtering, something so darn hard to do with twitter content)

Here's today's view of the data for my account (searching for @avinashkaushik). . .

twitter streamgraphs @avinashkaushik sm

Please click on the image for a higher resolution image.

You can choose the associative word stream that you are interested in most, click on it and at the bottom you can see the tweets. The size of the stream and shows you strength.

Once you choose the stream you can also click on the dates on the x-axis to filter down to the tweets for that particular stream for that particular date.

Sweet.

The stream graph for avinashkaushik would be different, as it looks for mentions. . . .

twitter streamgraphs avinashkaushik sm

Please click on the image for a higher resolution image.

It is analyzing the last 1000 tweets and such a great way for me to understand the content, and filter down and review the relevant tweets easily.

Twitter StreamGraphs helps you visualize content in a very unique way and solves a very important problem to boot.

Parting Words of Wisdom.

I hope if there is one thing I have convinced you of then it is that you need to be a lot more critical when you think of analyzing these new media channels.

It is important to put aside stale (certainly current web analytics) thinking.

It is important to participate in these mediums so that you'll truly appreciate what their real strengths are.

It is important to question metrics that have cute names, dig one step deep, just one single solitary step, to check if the metric definition passes the BS filter.

It is important to choose the metrics that help you measure your unique goals.

Finally it is important to realize there are no short cuts. Be willing to work hard. Be willing to put in the sweat equity. Be willing to try 45 things (tools / metrics / strategies) to find the 3 that work for you.

Good luck!

[Missed the contest? Go back and look for the red parenthesis, you'll win a copy of Web Analytics 2.0! Contest closed, thanks for the entries!]

Please share your feedback on this post via comments. Got any other tools that you love and adore? Please share them – with a quick comment on Why you love them. Got a piece of analysis that you think is profound? Please share that with all of us as well.

Thanks.

13 Nov 2009 02:38 am

webanalytics2 1 I am absolutely thrilled that my book Web Analytics 2.0 has been released and is in retail stores now, online and offline! Hurray!!

Even with a broken right hand I can't help but write this post!

The waterfall of positive feeling stems from the fact that this book was very hard to write.

I only had one job, at Intuit, when I wrote my first web analytics book. I now have several full time jobs, plus this blog, plus speaking around the world, plus a family, plus… so much more.

It took weekends of writing and nights of editing and days of research combined with practicing the preaching by doing oodles of analysis and, more importantly, the support of the most understanding wife in the world.

At the end of it all it is rather gratifying to see one's book at a bookstore, helps grasp the magnitude of the process. And there's absolutely nothing quite like hearing your five year old yell in a busy Borders bookstore: "I FOUND DADDY'S BOOK!"

This blog post is in three parts: The pitch. Request for help. A lovely contest [Contest closed now, thanks for the entries!].

You don't have to read the whole thing & skip ahead, but that would hurt my feelings. :)

Here we go. . .

The Pitch:

I invite you to consider buying my second web analytics book. It is not only the most current book on everything important and bleeding edge in Web Analytics, it is a labor of love that will help you transform your personal thinking and assist in revolutionizing your organization (big or small).

It is not a technical book, though it will make you technically dangerous. It is not just a business book, though every dna strand in this book is more about online marketing than online analytics. It is not a hard book to read, though it is brain food.

Here's why I think you'll love it:

Chapter 1 The Bold New World of Web Analytics 2.0

No dragging of the feet, the book starts with a bang by laying out the framework that will be the center of every company that will leverage data (qualitative, quantitative, competitive) on the web. It ends with a challenge to embrace Multiplicity – without this it's goodbye greatness.

Chapter 2 The Optimal Strategy for Choosing Your Web Analytics Soul Mate

It will be hard for you to find a more compelling four step process to choose the right web analytics tool for your company. Soul searching, questions to torture vendors with, comparing vendors, running a pilot and negotiating a contract, it's all in there. You be off to the races right.

Chapter 3 The Awesome World of Clickstream Analysis: Metrics

The thing I enjoyed about this chapter (I know I wrote it, but still. . .) was that the first half works really hard to evolve your critical thinking skills. I love that because we take too much for granted, now you'll be skeptical. A good thing. The second half shows exactly how to pick the best metrics for your org and, my absolute favorite (Page 64), how to diagnose the root cause of a metrics performance.

web analytics 2.0 cover1

Chapter 4 The Awesome World of Clickstream Analysis: Practical Solutions

When people think of web analytics everything they think about is chapter 4, and yet you'll find so many yummy treats here. The best WA report, segmentation, site search, SEO & PPC analysis, email, rich media, cookies, data sampling. . . . I am out of breath!

Chapter 5 The Key to Glory: Measuring Success

If I have one jihad it is to massively convert every person who touches the web to focus on measuring Outcomes! It is the one reason we can't achieve the greatness we so richly deserve. No more! Glory will be yours!! B2B. B2C. Small Biz. Large Biz. Non-Ecommerce. We make love to 'em all! One thing you'll read here that you'll read no where else? Computing Economic Value, a concept that will liberate you.

Chapter 6 Solving the “Why” Puzzle: Leveraging Qualitative Data

Oh, oh, oh qualitative analysis!! I am a Mechanical Engineer with a MBA, a late covert to the power of understanding the super sexy "why" by leveraging lab usability studies, surveys, card sorts, online remote testing and more. You get a jump start. The thing you'll adore: Pages 190 – 192.

Chapter 7 Failing Faster: Unleashing the Power of Testing and Experimentation

Sure you've heard of A/B and multivariate testing. But do you know how to truly win the game? There is no technical mumbo-jumbo here, just the real deal and how to get testing right. The thing you might not know / realize the power of: Controlled Experiments. I am convinced this is God's gift to online humanity, you'll agree with me by the time you reach Page 208.

web analytics 2.0 cover4

Chapter 8 Competitive Intelligence Analysis

The most magnificent advantage the web possesses: everyone's data is available for everyone else to use. If Hilton Hotels has the data for Choice Hotels why not use it to "crush" them (sorry Sarah!). This chapter shows you how. I think the thing you'll be surprised by is at the start of the chapter (Data Sources, Types and Secrets).

Chapter 9 Emerging Analytics: Social, Mobile, and Video

The chapter I had the second most fun writing. Mobile, twitter, blogs, videos etc are just so darned hard to measure and so much changes every few hours that I had to really really work hard to find the essence of each and then make specific practical measurement recommendations that will stand the test of time. It was hard.

Chapter 10 Optimal Solutions for Hidden Web Analytics Traps

This is a collection of major reasons I think people fail at web analytics, and of course I boldly try to share how to avoid that fate. Behavior targeting, dashboards, accuracy, data mining, predictive analytics, and, the thing you'll appreciate the most IMHO, five steps for intelligent analytics evolution!

Chapter 11 Guiding Principles for Becoming an Analysis Ninja

All my life learnings laid bare. . . this is where you, yes you, start to evolve from a Reporting Squirrel to an Analysis Ninja! No metrics, data pukes, guidance on creating every more reports. No, none of that. Rather… analytical techniques, tips and tricks to apply to your job, how to evolve your thinking to a higher level.

web analytics 2.0 cover3

Chapter 12 Advanced Principles for Becoming an Analysis Ninja

The chapter I had most fun writing (and rewrote the most number of times). It deals with two of the hardest practical challenges we face in the field of measurement: multi-touch campaign attribution analysis and multi channel analytics. Both are very hard to get right, both have a ton of fud out there, it was fun to share my recommendations.

Chapter 13 The Web Analytics Career

The chapter I should have had in the first book. How to plan a career in web analytics (paths, salary, longevity), and how to then cultivate the right set of skills. If you are a leader then how to spot great talent, how to interview them and make the right choice.

Chapter 14 HiPPOs, Ninjas, and the Masses: Creating a Data-Driven Culture

Some might argue, rightly so, that the most elusive thing to accomplish is to truly bring data democracy to your organization. This chapter bravely hopes to help you do exactly that: excite people about data, remove organizational barriers, use data to change behavior, dealing with data quality, and creating data driven HiPPO's.

Convinced?

Nothing, absolutely nothing, in life is easy. But if you have the will and access to knowledge then that just might help you choose an optimal path, a path where your hard work will yield above normal results. That's my hope, and promise, with Web Analytics 2.0.

Jennie and I have decided to donate 100% of our proceeds from this book, just like for the first one, to two charities. This book benefits The Smile Train and Ekal Vidyalaya. We are very excited about that.

yes check mark

Request For Help:

As you all know my philosophy for this blog is eat like a bird, poop like an elephant. But if you are up for it I would love to ask you for a bit of help.

Recommend the book.
If you know someone who needs to turbocharge their online existence, please recommend Web Analytics 2.0 to them. Even in our hyper connected world, nothing works like a personal recommendation.

If you use a link please consider using: http://bit.ly/akwa20 That link has an affiliate code, all proceeds of which go to the above mentioned charities.

Review the book.
If you have a blog, website, twitter account, any kind of platform, it would be great if you could write a review of the book and help spread the word.

If you purchased the book online then please, pretty please, review the book on the store's website. Amazon. Borders. Target. Powells. Whatever you used.

Connect me.
I am very very bad at pimping. So if you know someone who is someone (or knows someone who knows someone) then please consider connecting us. Especially people outside our analytics / search circle. Authors. CEO's. Journalists. Influencers. TV anchors (or weather man/woman). Oprah (I can dream, can't I?).

Our world is separated by six degrees of separation, I am sure you know someone who just might consider helping me with my cause.

Share a picture.
I love getting to know my audience, and while your emails and tweets are pretty fun there is nothing like a picture.

I had a "Web Analytics: An Hour A Day Fan Mail" flickr group that has some incredible pictures from around the world, bringing my audience closer to me.

I would love to do the same again for my "Web Analytics 2.0: Fan Mail". Be as creative as you want to be. Babies. Cats. Posters. Cars. Places. Or the best, you. All would be welcome.

web analytcs 2.0 fan mail

I will only post the pictures with your permission. Please send them to blog at kaushik dot net. Thanks!

A Lovely Contest:

[The contest is closed now. Winning entry details.]

Steve Cunningham invited me to be a part of a little "contest" he is running. The prize is a delight, you get to win a pack of seven books on online marketing & social media: Six Pixels of Separation, The New Community Rules, The Whuffie Factor, Trust Agents, Crush It!, Duct Tape Marketing, and Web Analytics 2.0.

How to win you ask? Two ways.

1. Answer this question in comments below: If you were to measure the success of a company's social media efforts how would you do it?

Pick any social media channel, or all. Only a short answer is required. The most innovative / interesting answer wins. No answer is too small or too simple.

[If you have my book already then my answers in the book to this question will win you major brownie points, but perhaps not the contest! :)]

2. You can get four more chances to win, if you want. Simply visit these blogs and answer a different question on each: Steve Cunningham, Beth Kanter, Tara Hunt, and John Jantsch.

Good luck!

A Word of Thanks:

This is from my book's acknowledgment page…

I would like to express my deep appreciation to the readers of my blog, Occam’s Razor. In approximately three and a half years I have written 411,725 words in my 204 blog posts, and the readers of my blog have written 615,192 words in comments! Their engagement means the world to me and motivates me to make each blog post better than the last. It is impossible to thank each person, so on their behalf let me thank three: Ned Kumar, Rick Curtis, and Joe Teixeira.

A very solid case can be made for the fact that neither one of my books would exist without you and your engagement and encouragement.

Gracias. Arigato. Ngiyabonga. Xie xie. Obrigado. Shukriya.