Analytics Success Lies in Digesting Related Segments


Our 10th analytics class focused on how to best analyze and understand data.

The top-level message I came away with is a data metric shouldn’t be evaluated in isolation, which is meaningless.  For example, knowing only that users spent an average of five minutes on a page offers no insight to validate or adjust your strategy. It may mean only users from a paid referral on one day of the year spent this much time on the site and on other days or referrals from organic/social/direct bounced after 10 seconds.  This would call for course corrections to improve results but without looking at the ‘big picture,’ these issues might go undetected.

Specifically, you should assess data metrics:

  • In context with the organization’s goals, what competitors are doing, industry information, internal initiatives, external events/trends…
  • Ideally in related segments, such as: referral sources (paid/social/organic/direct), days of the week (weekends and weekdays), times of the day or platforms used to access a site.

Sofia cautioned against comparing unrelated data metrics, such as tablet use with social referrals or creating compound or ‘super’ combinations like Alexa page rank with inbound links.

I found it interesting to note that higher numeric data (e.g. 10,000 contact page exits on every 100,00 visits), might deliver exactly the same statistical significance as smaller data (e.g. 1,000 contact page exits out of 10,000 visits) but is more effective when expressed as a clear outcome, such as 10%, versus a ‘muddy’ ratio like 10,000:100,000 or 1,000:10,000. However, I think higher numbers do strengthen a metric’s value.

For the class exercise, we re-examined the metrics set for our senior project, identified segments to be assessed and ways they might be visually depicted.

For my project, here are some KPI segmentation and visualization options:

  • Percentage of conversions by each referral source, such as social (fb, Twitter, Reddit), organic, paid and direct in a specific month, compared to non-conversions by referral sources in the same month.  (Visual depiction – 2 pie charts or a bar graph)
  • Specific Senior Care Share modules engaged in a specific month by referral sources. (Visual depiction – segmented bar chart)
  • Page views for a specific month, segmented by weekdays versus weekends. (Visual depiction – segmented bar chart, pie chart or even infographic with other metrics)
  • Page views in a specific month, segmented by referral sources. (Visual depiction – segmented bar chart)

However, as with all analysis, you should:

  • Find out what’s happening across the organization, such as other initiatives; business changes or help desk calls.
  • Consider external events that might impact data, such as holidays, market trends or even weather/power outages.
  • Possibly do surveys, click density analysis or other research….

….to get the full picture and extract maximum value from your data.

Illustration Source: Haml via Morgue File.

Telling the Visual Story of Data

Screen Shot 2015-03-19 at 4.45.19 PMIn our ninth analytics class, we focused on data visualization, which is used to visually depict a project’s results or analytics.  Seeing a visual representation of data helps people to understand its meaning. It also makes the information more memorable as it helps to tell a story.

Data visualization is particularly effective for conveying a project’s ongoing results, particularly KPIs, to stakeholders and showing them where they need to pay attention or course correct.

There is a huge variety of data visualization formats you can use to tell the visual story of data — from conventional bar and line graphs to golden ratio depleting charts and intricate infographics.  Fortunately, there are a range of tools available for creating them. We took some time to experiment with Tableau. I think this is a really versatile tool but it comes with a steep learning curve.

I think it would be really useful for showing KPIs for my senior project, such as conversion rates from various social and organic referrals, possibly in a pie chart, to show which are most effective.  I might also use a bar graphic to compare the time users spent on each of the niche social media network’s pages.

However, in the interim, I wanted to try this tool on existing data. Last summer, I managed a youth shelter’s eight week fundraising campaign that was heavily promoted on social media. I used a tool called ‘SumAll’ to track Twitter results for this campaign and exported them into an excel sheet. I imported this into Tableau and experimented with the tool.

One challenge was getting the weeks to display from oldest to latest. I re-labelled the weeks from a ‘week of Mon 08 Sept’ format to ‘week 8 (08 Sept)’ format in Excel to make the sequencing clearer. Latterly, I also found that in some views, you can click on the Dimension value to reverse the order.

There is much more formatting I need to do but, the above bar graphic is a start at showing the stats from this campaign. In this example, I used a side-by-side bar chart format to show the growing momentum of three measures (tweets, mentions reach and retweets reach) over the last six weeks of the campaign leading up to a September 6 event date. I filtered out the first two weeks, as the Mentions and Retweet reaches were minimal during this early part of the campaign.

A graph like this could be used to demonstrate to a stakeholder the value of a campaign of several weeks, versus expecting comparable results with a one or two week period (which some might think adequate).

I need to learn much more — but can’t even begin to visually imagine the data I must absorb to proficiently use these tools.

Why Catch Key Words and the Long Tail?

Long Tail ImageOur eighth Analytics class focused on Search Engine Optimization (SEO), which determines how prominent (or where) your website site will be listed in unpaid/organic/natural search results via Google or other online search tools.

A key part of attaining optimal SEO for your site is understanding your target audience’s search behaviours.

We learned that people usually access websites to solve one of the following types of queries:

  • Navigational – to find a specific URL
  • Informational – to find any information from an address to an in-depth research study.
  • Transactional – to complete a task, usually a purchase

A key part of mastering SEO seems to be identifying the keywords people use to find your site. Two main types for keywords are: one word to short terms (2 – 5 words) called Popular/Lead keywords; and longer, detailed phrases, called Long-tail keywords.  You need both for success.

As expected, transactional searches are the highest value because they usually generate revenue but I found it interesting that these are also the more detailed, long-tail words or phrases.

There are also other psuedo keywords, such as ‘Trophy Term-Keywords,’ which may drive traffic to your website but this traffic doesn’t convert or behave how you want it to, likely because it’s not a good fit between your audience and your site. For example, a term like ‘call to arms’ might drive military oriented people to a site but they will quickly leave if it’s about prosthetic limbs.

Sofia provided a great list of links and tips for maximizing SEO. Even pointers like setting URLs in the country of your target audience and putting keywords in JPG titles helps.

To compile a list for my senior project, I used Google’s keyword generator tool for my site, then searched my competitors’ sites and ran the generator for their sites.  I found many popular terms, like: senior care, help for caregivers and help for elderly.  I also found the following particularly useful long-tail options:

  • practical tips from real caregivers
  • outside resources
  • here to share advice from the community
  • online community for family & professional #caregivers
  • Where Caregivers Survive & Thrive
  • Caregiver’s Survival Network
  • family caregivers taking care of a loved one

As my project progresses and the content is clearer, I will re-do and refine my searches to find more.

(Photo: Taken by Gabor from Hungary via Morguefile) 

Adding and Measuring the Impact of Mobile Apps

MobilePhoneMeasureSmallFor week 7 of analytics, we focused on how to measure the impact of an app — be it for entertainment, practical or promotion purposes.

Given the development work behind each app, I think it’s particularly important to measure their analytics so that you can course correct, as required, to maximize the investment. This means using app analytics to learn: who downloads the app,  how they discover it  and how frequently they use it (if at all).

Although app analytics measures many of the same metrics as web analytics, like number of visits and sources, I thought it was interesting to note some of the subtle differences between the two.  For example, I noticed that app analytics metrics include:

  • Screen resolutions (as well as screen size and orientation)
  • Varied types of mobile devices used
  • A stronger focus on groups of users — including user’s friends who also use the app and ‘cohorts’ (i.e., segments of customers that downloaded the app during a common time period).
  • “Usage” instances and frequency, since downloads/installs don’t correlate with engagement/use.

Like web analytics, you need to define metrics and KPIs. For example, you need to define an ‘active’ user’s level of engagement (frequency, duration).

We also looked at various tools for measuring app analytics. However, Sofia pointed out that the metrics they measure are not evenly distributed among the categories of apps available. Specifically, current tools are more focused on marketing metrics, like search engine keywords used to discover them, versus content metrics, such as the app’s longevity/life cycle and number of app/widget installs a user has.

Of the tools we looked at, I particularly liked Flurry Analytics because it provides benchmark measures and several distinct metrics, such as user personas. However, it didn’t look like it provided search engine keywords, which you needed to measure through other tools, such as App Annie, as its Key word ranks feature.

For the exercise, I selected a made-up ‘package ingredient analysis app’ promoted through an Instagram campaign. I identified KPIs and measuring tools as:

  • „Number of active users (Flurry- Event- Use)
  • „Average increase in users (over 2 week time periods) – (Flurry – Funnel Analysis)
  • Number of people using the app at 2x per week or more (Flurry – Retention – Return Rate & Event – Use)

I identified metrics and their tools as:

  • „Search engine words used to find app (App Annie – Key word ranks)
  • „Number of installs (Flurry – Funnel –Downloads)
  • „Number of users who have installed the app vs. Instagram users (app installs and Instagram audience)
  • „Personas using app (Flurry – Audience Interest – Personas)
  • „Use of this App compared to other lifestyle apps (Flurry – Audience Interest – My Apps Interest)

We also explored and discovered several other worthwhile app analytics tools, including:

  • Amplitude – Focuses on user engagement, including: segmentation, cohorts and churn.
  • AppSee – Indepth analysis of users and everything they do with app (for a steep price)
  • Ninja Metrics – Focuses on customers and is well suited to game apps
  • Localytics – Unique tool that integrates analytics with marketing to help produce push campaigns

This class opened a new world of analytics — and there may be an app for that too.

Linking to Success With Social Media

For week 5 of analytics, we dug deeper into social media with a focus on the value of Google+.  Although Google+ may seem less intuitive to use, you need to evaluate its use in terms of your goals. Specifically, if you want ROI or engagement, Google+ may be well worth the time it takes to set up because it controls ‘who’ sees what you share by controlling how high your organization comes up in Google searches.  Sofia showed us a couple of examples from her own business, when she posted on Google+ and how it boosted the company’s online prominence.  I thought this was effective and drove the point home.  With my senior project, I think it may be worthwhile looking into Google+ ‘communities’ that pertain to senior care but also I think it will be important to have a presence on  Google+ so that might key words and organization will be found.

We also learned than ‘not all links are created equal.’  A key way to optimize search engine results to your site (SEO) is by having links from other website’s connect to yours. However, we learned the strength of these links varies according to various criteria, such as:

  • Authority or credibility of each site linking to you. (It’s better to have links from established, high profile sites like Buzzfeed or the Huffington Post than a blog run by one person with a few subscribers)
  • Number of other sites each site links to. (If it links to a high number of other sites, its link is of less value.)
  • Where your link appears (The Jackpot to have your link in the title or anchor text but this is rare.)
  • Relevance of the sites linking to you. (If you pay for links from an essentially ‘content-free’ site or link farm, there is little value)
  • Authority and Page Rank of the linking page

I also learned the word ‘Social Signals,’ which is when someone likes, recommends or shares your post.  So with my senior project, I ideally want many ‘high value’ inbound links, ideally from relevant healthcare/senior care sites and to generate many ‘social signals.’

Drilling Down to Social Strategies

Our fourth Analytics class focused on the first of two parts on analytics for social media. This first part highlighted facebook, twitter and LinkedIn in terms of what users can do on these sites and “how” you can use these platforms to measure online, as well as offline, initiatives.

As with all analytics, you need to identify your SMART goals, KPIs, metrics and measurement methods in advance to ensure you capture all the relevant data and don’t waste time or lose important stats.

I think it’s interesting to note that if you are running a campaign that includes an offline tactic (e.g. a coupon or redeemable voucher), you can measure its impact online — IF you plan in advance.  The reverse also applies. For example, you can measure the offline response of a Facebook ad by using “offer claims.”

For this week’s in class exercise, we had to develop a social media strategy for promoting Parks Canada’s “Unplugged” campaign. For this, we had to identify SMART Goals and tactics for achieving them, as well as KPIs and metrics to measure progress.  I found this exercise a little confusing because we had just learned that traditionally KPIs are metrics that relate directly to your goals and generally impact the business.  Specifically, KPIs traditionally apply to revenue-related metrics (e.g. cost per lead, return on invest) or direct conversions that impact the business (new memberships).  However, I learned that the KPIs for a social media campaign are an exception to this. That is, they are social, as are the SMART Goals that support the campaign (e.g. generating X number of tweets with a specific hashtag).  This is good to note, as ‘when’ I launch my senior project, I will likely do it with a social campaign to drive traffic to the site and ideally generate conversions (member sign-ups).

In the exercise, we had to create a strategy for Twitter, Facebook and LinkedIn. The first two were straightforward, as the campaign was consumer-facing, which both Facebook and Twitter can be.  However, LinkedIn forced us to think ‘outside the box’ as it’s more of a B2B channel.  This was a challenge but good because sometimes clients decide they want to use a specific social media channel and you need to find a way to make it relevant to your marketing needs.

Many Web Analytics Metrics and Ways to Measure

Screen Shot 2015-01-29 at 4.06.21 PMOur third Analytics class focused on the various types of website traffic data points and ways to measure them, both qualitatively and quantitatively.

I learned that you can measure a vast quantity of web data points or metrics. I think it helps to try to remember them by categories, which include: conversions (rate, by source..), traffic ‘referral’ sources, geographic location, visits, visitors (new, return, unique…), time on page, time on site, bounce rate, exit rate, engagement and purchase habits (cart abandonment, days/visits to purchase…).

It was also worthwhile learning that analytics aren’t infallible and that loopholes exist. For example, a visitor may spend five minutes on a page but if they don’t go to a new page, their visit is tracked as a ‘bounce,’ which is inaccurately perceived as negative.

I think a good point to remember from the class is also that the impact or positive/negative attribute of a metric varies according to the type of site and its purpose.  For example, for a service business site, like Sofia’s, you want first time visitors who scan the site, exit on the contact page and follow through to enquire about purchasing a service.  Repeat visitors who never follow through offer negligible value.  In contrast, for a social network (like the like the caregivers one I’m proposing for my senior project), repeat visitors are critical for the site’s longevity; you also want them to come to the site, ‘engage’ (by posting or responding to a post) and leave, most likely without going to the contact page. Ideally, you want repeat visits from those who’ve signed on as members (i.e., converted) but even those who check the site a few times before committing are ok, particularly as the site evolves.

As for ways to measure web analytics, we learned about various methods. It sounds like one of the key ways to quantitatively measure analytics is using JavaScript tags. I’ve used this method to measure another blog by embedding the tag via a CloudFare workaround. Unfortunately, I haven’t blogged much since I set this up in Oct. 2013. However, my page views for Nov. 2013 were 1.47, which doesn’t sound as bad when I consider that page views are not as relevant on a blog because people usually visit the most recent post and bypass the archives. We also learned about various qualitative measurement methods, including heuristic evaluations, site visits, usability testing, surveys, web enabled research, experience testing and collecting competitive intelligence data (e.g. panel based measurement, similar to AC Nielsen for  TV).

The we did an in class exercise to clarify our understanding and from this I learned more about methods.  My takeaways from this assignment, (which I ended up presenting) were:

  • When trying to determine a website’s goal, always search for ‘how’ it is making revenue (unless it’s clearly run by a not-for-profit or clearly funded by another source).
  • Be very specific when citing KPIs or metrics. For example, don’t just say “measure return on investment” but clarify ‘what’ you are measuring (e.g. orders), against what (e.g. operational costs) and how frequently (e.g. weekly).
  • Heuristic Evaluations, such as Site Visits and Remote Testing, do not usually have a sufficient number of participants to be statistically significant, even if you are testing a metric that can be quantified.