Trying to find meaning in the realm of interactive agencies, online strategy and the cold Canadian winter

Hey folks, my name is Briac and I am happy to answer Philippe’s invitation to use his blog for writing an extensive review of eMetrics Toronto 2010 – Canada’s Online Marketing Optimization Summit. That’s my first shot at blogging so I hope you’ll like it, please feel free to comment

Held along with the Search Marketing Expo from April 6th to 9th, eMetrics is the Canadian rendez-vous for all things web analytics.The Summit is organized by Andrea Hadley of eMetrics Canada in partnership with Rising Media and it is the official conference of the Web Analytics Association (WAA). I was participating in the event as a volunteer and it was a chance for me to meet incredibly interesting people and attend numerous insightful presentations. If you’ve been there, you probably remember me as “That French Guy”, as my business card was saying

Wow that’s a long scrollbar you have here. Why should I read all of that?

Well, here is what to expect in this post:

• An introduction on Web Analytics and the purpose of eMetrics
• An illustrated review of the best presentations I attended
• My thoughts about the WAA Certification Test
• A discussion on the past and future of Web Analytics
• Tons of links to speakers, resources, other reviews and my twitter account of course.

There’s more than fried nom nom, here is a quick wordle representing what’s in that post:

Wait, what is that Web Analytics thing?

If you attended the event or are a WAA member, skip this part. If you think Web Analytics = Google Analytics, please read it. Twice.

For those of you wondering what is Web Analytics, well first of all, it’s not simply “Google Analytics”. Not at all. Here is some neat official definition, courtesy of Wikipedia:

Web analytics is the measurement, collection, analysis and reporting of internet data for purposes of understanding and optimizing web usage

Hence yes, that’s what Google Analytics does. But the vendors of analytics solution are many out there (Omniture, Clicktale, Coremetrics…), it’s a whole ecosystem, and it’s important to go beyond the tool. It’s like saying, design is photoshop. As you may understand, Web Analytics could be understood as a broader term, ranging from Business Intelligence, Marketing Research… and I can throw more fancy consultish words in here: Customer Intelligence, Conversion Optimization, Online Accountability… Funniest thing is, the more it evolves, the less “Web” it is, increasingly focusing on a holistic approach of optimal decision-making support for businesses. Eat that pie chart!

So to understand that this is a field, an industry, with professional practitioners and a continuous concern of defining what data means online is crucial for perceiving the value of a summit like eMetrics.

Should you have any interest in this topic, the ultimate reference is Avinash Kaushik’s bible of Web Analytics

Duh! Now, What it’s in it for me?

Avinash Kaushik – Social and Search: Rock Both Worlds With Data!

Let’s start by the opening presentation on Thursday by Avinash himself, a brilliant piece of public speaking. Really entertaining and insightful, here is an good post about it. Few take-away:

• Funny definition of bounce rate: “I came, I puke, I left”
• Social analytics: sentiment rating stinks (positive/negative), instead, use attitudinal frameworks and analyse words
  • I have to add that on that point, there is an exciting new world to build. Being a marketing student, I learned some of the construction of the concepts behind “consumer behaviour”, and let me tell you that it’s a huge discipline by itself. Attitudes, behaviour, mental models, ethnographics… There are many way to quantify and qualify the conversation happening on Social Media and I agree with Avinash, starring at graphs that shows “positive sentiment toward a brand” is just plain bullshit. Get your psychology 101 book, go beyond the traditional Maslow’s pyramid and start working on your own decision matrix. Be smarter than tools. Or wait a few month, it’s coming…
• Consider “aggregate marginal gains” of long tail search and microconversions. Yes the internet is full of weird people messing with your website, and no that’s not a bummer. It’s a world of opportunities!

Theresa Locklear - How NHL.com Measures Fan Equity Online

On Wednesday, we had the chance to have the full Web Analytics team from NHL.com presenting their work. A complete team of analysts! Seems NHL.com is a serious playground… One of the best presentation of the event, and mostly because they were amongst the few to actually share numbers. Sharing is really important for this practice to progress, but as you guess it’s sensible since we deal with competitive intelligence here.

Main Dashboard. I told you, Web Analytics is more than Google Analytics. I don't know about you, but that thing arouses me.

Mobile Dashboard. More location-based arousing.

Interesting fact, most of the dashboards built by the team were done with Excel. Data was extracted from different tools (i.e social media analytics tool, mobile analytics tool, log analysis etc.) and compiled in good old worksheets. It seems that automating reports and setting up custom dashboard is not an easy task yet for our Web Analyst friends…

Matrix Framework for Voice of Customer (survey) report, progress tracking and decision-making

The other point that got my attention was how NHL.com used survey report and Voice of Customer Analysis (a frequent term used by Web Analysts). Online customers opinions are regularly investigated with the help of survey tools (from the main vendors Foresee and iPerception), and the NHL team use 4 way matrix to decide on what action to take and track progress of the part of the website they change and improve. The survey results are often showing a significant impact when doing design change, and this information is crucial for completing the data given by clickstream analysis (which is obviously not self-sufficient).

Social Media Dashboard. Yes there are a lot of metrics, so stop counting Retweets!

John Lovett – Organizing for Analytics

John is an eminent member of the WAA and an old-timer of web analytics. His management-oriented presentation aimed at explaining how to structure an organization around web analytics, who are the analysts and how should they interact within the business. It all started with one statement which everyone agrees on:

“Businesses can create sustainable and strategic competitive advantages by investing in analytics.”

After explaining why Analytics shouldn’t be a siloed function in the business, but be global and managed amongst all areas, why it’s about people and processes, not just technology, and the importance of reporting and informing, not just collecting data, we understood that the game-changing attitude lies in one thing:

“Businesses can create sustainable and strategic competitive advantages by investing in organizing for analytics.”

The Real Problem: Web Analytics is Hard

Web Analytics is Hard, but it’s ok, because John has the solution:

1. Architect Your Web Analytics Technology
2. Manage Your Web Analytics Talent
3. Focus Your Analytical Efforts

A little more on Manage Your Web Analytics Talent:
Lovett explains how Analytics is a team effort, and describes the 3 profiles required:

• Analytics Executive: the senior and business oriented analyst, team manager
• Analytics Professionals: the trained analyst
• Analytics Amateurs: one foot in analytics, he knows the tools and the practices, he his the marketer, the developer who will interact with the Analytics team.

What is a profile of an Analytics pro:

• Business-minded, strong interest in marketing
• Tech-savvy, knowledge of code, bridging the gap between business and IT
• History of working with cross-functional teams, able to message the value of the data
• Senior enough to be heard, report to VP
• Expect to pay about $100K or more (rare profile + highly skilled)

Now, what the presentation doesn’t say, it’s how much of a technical background in data analysis this professional should have. So I went to talk to John at the end of the presentation, and he confirmed: ideally, this individual has a background in statistical analysis, not necessarily a PhD, but a sufficient training in making sense of the data. While other individual like Patrick Glinski (I talk more about him later in the text) may argue that this is not enough, that ideally you’re a UX practitionner, an information vizualization enthusiast and what not, I think John has a decent and realistic framework for defining the Analytics pro.

Talking about paying for your analytics crew, here is an extract from the 2010 Ad Agency Salary Guide done by the IAB Canada:

So, despite what John annouced, we can see that one can still afford the “reporting bitch” for cheap, now moving on to higher positions, and specific to web analysis, we’re moving toward serious salaries.

One slide got my attention on the topic of profile, the Analytics output. Above the red line, you can see what is ideal but not there yet. And that include Predictive Analytics, Forecasting and Statistical models. So it would make sense that your analytics pro is a statistician, not just a tech-savvy marketer.

Web Analytics Output

Last take-away:

• Managers want Recommendations!
• Analytics is equally technology, people and process.

Brian Flanagan – How Expedia Listens to Millions of Customers as Individuals

Big business, big insights. See for yourself:

And they have tools!

• Web Analytics
• SEM tools
• Feedback and survey tools
• Email tools
• Customer Experience Management tools
• System Monitoring tools
• Data Warehousing and BI tools

But the tools give little insight on customer experience, while listening to customer only gives a part of the picture, so how do Expedia manage to be an excellent business?

First, don’t rely on the IT + Financial report, your customer might encounter problem for month without seeing those indicators fail. The secret is combining Voice of Customer and Web Analytics. It gives context for feedback and lead you to identify the source of experience problems.

The right tool combination:

Notable point, Expedia is well organized for Analytics, the reports are shared and send across different line of business and everyone is accountable for great customer experience.

After numerous example given by Brian, we could feel how Customer Experience is improved by successfully tracking and monitoring intent and action, not just planning or looking at simple metrics. The fallacy is, you may want to rely on a sole indicator, a judgmental one, such as usability testing or UX design for example, with a given set of scenario to validate. In the end, you can’t predict all intent and attempt on a huge website like expedia (or should I say, the many expedia points of access, and that also include Call Centers). You need to listen to all kind of feedback, and then investigate to understand what has happened, whether technical issue, information architecture issue, pricing issue and what not.

Eric Pierni – Built to Click

Eric Pierni, a former BComm graduate from the John Molson School of Business (\o/ yeah the school that trains champions), is the of Head of Advertising at Kijiji, the Canadian branch of Ebay classified.

As you may guess, Kijiji is a huge, traffic driven, content rich, multi-niche website. Its revenues are generated from on-site advertising, and the competition is tough. Eric explained how he manages more than a forever-going PPC campaign of 2 millions keywords for maximizing the acquisition of traffic. Kijiji is an aggregation of niche, and it’s a universe of perpetual microconversions. It was captivating to hear him explain how he optimized conversions and minimize acquistion cost, and how he is always listening to customers for forever improving the efficiency of Kijiji.ca. On the keyword acquisition thing, Eric was explaining that at the time of Kijiji’s launch, they had no idea how much to spend on a keyword. All they wanted was traffic, and matching cost to revenues was impossible – the revenues had yet to come… My feeling is that this confident analytics-driven approach had made the financial success of the site, and we can understand how the profitability of Kijiji is dependent on a perpetual loop of acquisition-optimization.

Patrick Glinski & Andrea Peckham – How Social Media Measures Up When Giving Away Half a Million Dollars

This one was refreshing! We had Andrea, who is the client, and Patrick, the agency, teaming up on stage. The case was how to help Aviva increase its aided and unaided brand awareness through social media (brand recognition and brand recall, for the jargon haters). Patrick was pushing number & experience sharing, and explaining why it was crucial for the industry to share. He gave us a full understanding of why and how this campaign was successful. Basically, Aviva gave away ½ a $m to the best charity idea, thanks to a dedicated portal where people could submit ideas and vote. The campaign was aligned with marketing goals (marketing research highlighted the need for community support) and budget cut due to the recession (be more creative! TV spend is not efficient enough!)

It was a huge success in Anglophone Canada, the metrics improved big time, the exposure was more than satisfying, but above everything, user engagement was the highest surprise. As Patrick explained, they had to micromanage tons of interactions between the brand and the consumers, and sometime really unexpected. It could happen through any social networks (not only the dedicated portal) and active listening was not an easy task.

Now, what happened to Quebec? Well it didn’t work very well. Few interactions, little exposure and not that much of a change in brand awareness in the end. I talked to Andrea at the end of the conference, and she confirmed my thoughts: launching a national campaign from Toronto requires you to have the manpower to manage and understand the specificity of Quebec’s market. They’ve been overflown with user engagement, and what would have been likely to help, is a community manager dedicated to interacting with Quebecois

Talking about community management, that was the lesson from this experience, when aiming big, and when you want engagement, you have to be prepared, because it’s a world of time-consuming micro-management, and having your customer engaging in a conversation with the brand means a gazillion of micro-conversations!

It’s overwhelmingly fun!

Adviso & NVI on Multi-lingual PPC/SEO

Because I was curious to hear my fellow Quebecois speak, I had a sneak peak at an SMX presentation held by Marc-Antoine Lacroix from Adviso and Guillaume Bouchard from NVI, also including Jon Guljord from expedia.com and Sulemaan Ahmed from Harlequin, unfortunately I missed the last two.

Multi-lingual strategy is tricky, it requires preparation, from a technical standpoint (choosing domains & structuring the architecture of your website), as well as a strong content strategy. What I remember from Guillaume was the importance of “long-tail” non-english search. The English web is saturated, but there is room for translation. As he pointed out, he had the experience of the same page translated from English, generating Adsense revenues of $3/day in English, $30 in Spanish, and $50 in French. At the same time, traffic acquisition cost is lower (less bidding on non-English keywords/key phrases). That tells you a lot about volume!

The rest

I’ve seen many other really interesting presentation, but let’s be honest, this post is long enough… and my memory fails me anyway.

So, like, people do that as a job?

Yes, but consider this, some people would do accounting for a job. It’s alright to love bar charts and compute ratios, you do that because you know it makes the world better. One report at a time.

The almighty WAA is aware of that, so they created the Certification Test. After all, why would Accountants have their certification and not Web Analytics Practitioners? eMetrics Toronto was the first occasion to “Test the Test”. Thanks to Jim Novo, I had the chance to spend an hour being tested. Alright, it should have been an hour, I took 1 and ½ hour but that’s ok, I just pretend to be a slow reader because I’m French.

The test consist of multiple choice questions only, the first part is mainly knowledge restitution, and the second part consists of case studies. Honestly, it’s tough! Mainly because it primarily consists of all right answers, and the questions tend to be “what is the best…” or “most appropriate…” solution. As Jim put it, you don’t expect the analyst to be a textbook, but an analytical mind. What you have to do is consider a business situation, and select an appropriate way to solve/improve it. Thus, I would expect to have a surprisingly decent score, even if I lack the particular technical knowledge and I’m only halfway through Avinash’s book, my BComm background with business case study experience added to my experience in web design/development was definitely helping.

In the end, the test was of great help for me to understand what defines a Web Analytics practitioner, and get a picture of what I needed to improve. Namely, a bit of technical knowledge, and a lot of experience! I’m confident that it’s what I want to do, at least part of what I want to do, and acquiring skills in that field is of primary concern when you want to work in online marketing and e-commerce (ok, vague definition, but you don’t need the full buzzword tag cloud to get it). There is no excuse in 2010 to manage and produce online initiatives without a proper way to measure and improve. Faith-based marketing is smoke and mirror sold by overpaid lazy creatives who just want to get laid on your bucks. Or at least, the truth lies in the middle.

Don’t you have to go to school before being tested?

Yes and no. I applied for UBC’s Award of Achivement in Web Analytics because I wanted some formal education in the field + a signal to others. So once I’m done, I’ll probably take the test, the real one. But I’ll wait a year or two, I need to get experience, otherwise formal knowledge won’t help much. It’s about getting your hands dirty.

Yes, but does it gives you dates?

No, not really, but I made a couple of friends at eMetrics. It’s a friendly crowd, you’ll see passion and dedication. People are approachable and eager to share their experience.

Now, let’s give a broader conclusion on WA. First of all, if you read Avinash, you know the Trinity Framework. It’s the base of Web Analytics thinking, and how it fits in the process. Let’s get a picture of what WA people think their practice is.

On a more verticaly integrated way, from Avinash’s 2nd book, here is in what the value of WA lies:

After you’ve meditated on that, here is my thought. Do you see the obsession for revenue-generating measurement? Do you see how it sweats pure business thinking? Something is missing here. WA people, in general, critically demonstrate a lack of interest/knowledge for the field of User Experience Design (IxD, IA, UCD, UX, Usability, you name it). They somehow don’t want to hear about qualitative research and design thinking. I was surprised to hear over an over the term Voice of Customer and see that it was related to Surveys. And when panels where held about the quantitative/qualitative mix, still, little talk about the world of qualitative data and user research. I mean, really, online surveying as a mean to answer the Why question? That’s such a narrow and limited channel. And about the field of design, there so much to share. As I was telling John, I wish he included a sense of UX in the skillset of the perfect Web Analyst. It’s not necessary, and you can’t expect people to do and know everything, but why are those discipline ignoring each others?

Last month, I attended UXTalk, a (young) conference/discussion gathering UX practitionners in Montreal. A roundtable of 24 persons talking about “How UX fits in the process”. So naturally, after a long discussion about designers vs developers vs clients, selling usability, agile environment, and benefits of UX for helping businesses, well I asked the question, “Who/How/When integrate Web Analytics in the process”. I had a mixed reaction of confusion/disinterest, on a defensive tone. I heard “data can tell anything”, or “the clients does not understand, it’s not helping” and what not. Despite some interesting thoughts, I noticed how much UX people feel either disinterested or threatened by WA practices, and this does not make any sense.

It’s like everyone knows better what customers want and shows no trust in the other one’s voodoo.

So that was my biggest disappointment, until the end of the summit, when I met Patrick Glinski at the lobby bar, whom I had the best conversation with. He is the Ad agency guy, the creative, UX designer, social innovator, talented marketer but… web analyst! And that gave me hope. Hope for a possible world where the two can cohabit and live in harmony, perform better, achieve optimal results and make a world of happy customer-users. You know, Drucker said:

Because the purpose of business is to create a customer, the business enterprise has two—and only two—basic functions: marketing and innovation. Marketing and innovation produce results; all the rest are costs. Marketing is the distinguishing, unique function of the business

It’s darn valid and we are at the crossroad. Now, Patrick had a nice way to put it, we need the industry to adopt design thinking.

And I was happy until Patrick told me he was done with the WAA. As he put on his blog:

On the “dread” side, I can’t help but feel this conference is going to be the end of my formal association with the digital measurement community.I’ve been a practicing Web Analytics analyst for about 6 years now – an old guy in digital terms. But even in that space, I always felt a bit like an outsider as a user experience strategist (researcher) first and an analytics practitioner second – a divide that is becoming an argument of validation versus prediction.

Meanwhile, many of the dominant conversations are distant and foreign. As Web Analysts, we focus our efforts on what will derivate the largest impact. Usually that means optimizing the user experience around the highest revenue-generating opportunities. Rather than focus our efforts on any one individual, we analyze patterns because it’s a better use of our time. But it seems as though we’ve lost our own way – now focusing in on technologies and methods designed to derive actionable results around strategies that most clients can’t afford to build in the first place.

And while you think everything happens at the lobby bar, sometimes you find answers on blogs. Christopher Berry, whose first encounter I’ll never forget, has a nice thought development on WA, UX and design thinking to answer Patrick. Christopher is the ceiling cat of Synapse, he is really brilliant (or just sound so, beware of marketers), and that’s a shame he overuses internet memes for all purposes because it severely hurts his credibility I met him at the registration desk where I was volunteering, first thing he told me was, in order to get the official WAA stamp on his badge, “hi, can I haz my golden star?”. It was utterly frustrating to see him leave the lobby bar very soon, after dropping some thought on why Microsoft sucks, which is the least interesting conversation you want to hear from such a bright funny dude. Shame on you Chris. As you put it this way:

I would hate for the perception of the newcommers to be that we’re all crusty, exclusive, and inaccessible. After all, I was a newcommer once too. And I benefited incredibly from having access to those experts.

Well the newcomer I am wanted to have a conversation but you left that damn lobby bar! (fail)

Anyways, one other thing Patrick was interested in, and brought to light during the last panel discussion, was information visualization. He is passionated about it, and he has a good reason for that: for him, dashboards and report are a way to tell a story, like anything else. I think he is sort of the marketer of Web Analytics, he’s using persuasion and story telling to make management understand by themselves what they have to do. Ok, sounds sexy, I’m waiting for examples of that now

On the topic of data/info visualization, beyond all the hype, I recommend anyone to read Stephen Few, an old-timer statistician and BI expert, his article “Information Visualization, Design, and the Arts: Collision or Collaboration?” is a pure wonder.

Talking about BI, wait, there is more!

Some BI to kill the BS

As nicely put in this tweet from Philippe Denis, Senior Web Analytics Consultant at Bell Solutions Web:

Interesting: pure business intelligence people start showing up at conferences like #eMetrics. We are slowly starting to WA / BI gap.

Indeed, I had the pleasure to meet / hear some of those people, and I was curious to see how the fields are merging. Why not, after all? Perhaps a question of maturity, or openess, on either sides.

Stéphane Hamel, whom I was really glad to meet, and I recommend anyone to follow on twitter, just wrote an excellent post about the topic of BI+WA

Alright! What do I do now?

• Join the WAA, become a Web Analytics Practitioner, change your organization, adopt a data-driven culture, be a rock star!
• Get educated at UBC and/or get yourself the right books
• Like Guy Kawasaki said, “Beware of bozos”. It’s new and exciting, try to filter the noise, beware of those who want to sell you tools, value education and curiosity, question the gurus, and get some field experience.
• Get in touch with me through twitter, facebook or linkedin (I love making new friends!)
  • Social networks are fun, but if you’re in Montreal, let’s have a beer!
• Follow the eMetrics speakers on twitter, full list here
• Check what other people are saying on twitter by looking at the #eMetrics and #measure hashtags
• Read other reviews here:
  • Rommil Santiago – My thoughts of eMetrics and the future of web analytics
  • Christopher Berry – eMetrics Toronto Summary, Segmentation and Future
  • Patrick Glinski – Hopes and Fears from eMetrics Toronto
  • Marc-Antoine Lacroix – SMX Toronto: tools, search and dollar bills
  • Gil Reich – Best of SMX Toronto
  • Garry Przyklenk – eMetrics 2010 Toronto Recap: 3 themes, multiple evangelists
• Comment that post, I’m eager to hear what you have to say!

More About eMetrics Toronto

• Brought by Andrea Hadley to Canada in 2007, first produced in Toronto in 2008.

• The Web Analytics Association is a Lifetime Sponsor of eMetrics, and eMetrics is also a sponsor of the WAA

• The WAA often host an industry meeting at eMetrics the evening before the conference, essentially the opening event. This was the case on April 7^th in Toronto.
• It is the 3rd Canadian edition of eMetrics, organized with the support of the Advisory board consisting of
  • Stephane Hamel
  • Alex Langshur
  • June Li
  • Patrick Glinski
  • Mike Sukmanowsky
  • Kelly Kubrick
  • Sionne Roberts
  • Jacques Warren
  • Marco Bailetti