Measuring ROI and Success with Analytics

SONY DSCOur 11th Analytics class was about ‘how’ to measure an online strategy’s return on investment (ROI) — which sounds much simpler than it often is.

Some of the many challenges are:

  • Assessing SEO  – It’s hard to estimate the SEO’s ROI because you can’t estimate the full impact of the long tail. A workaround is to multiply the search volume for top key words by 3.3 (or 30% of the possible clicks) but this is just an estimate.
  • Defining Social Media ROI – You can’t always definitively tie a social conversion to a specific financial metric.  Subsequently, you need to assign a financial value for each social engagement metric by testing and validating social activity. For example, if you generate a $1,000 sale for every 10,000 Twitter re-tweets (RT), each RT is worth $10
  • Attribution Analysis Quandary – There is no definitive answer on which brand/campaign touchpoint ‘wins a visitor over’ and prompts them to complete the desired action (e.g. purchase, sign-up or send an email inquiry). It may be the fourth visit to the website or seeing the item in the store after after reading about it once on the website. As an alternative, Sofia outlined a blend of Media Mix Modeling and Marginal Attribution Analysis. Specifically, this means measuring your baseline, allocating part of the budget to one marketing channel, run tests with/without it and track impact until the value declines. Repeat for each channel until you find the most cost-effective combination with the highest returns.
  • Offline Impacts – Offline promotions and external events (e.g. new competition, scandal, market crash) can skew results. You need to take extra steps to track offline promotions, such as setting-up vanity URLs to track coupons, and find rationale for traffic patterns.
  • App ROI – Apps aren’t cheap to make or run. To measure their ROI, you need to consider the Total Cost of Ownership (TCO), including development and ongoing operation costs, which can fluctuate over time.

So while measurement is a finite science, measuring online ROI is part art because it requires some subjective decisions.

Sofia provided comprehensive best practices and a number of formulas, which I think will give me long-term value — or ROI on the cost of her college course. However each campaign and scenario is different and there is no ‘cookie cutter’ solution.

What does this mean to me?

For example, the business model for my senior project generates revenue by selling online advertising. If I need 10,000 page views per month to sell a $1,000 ad but I currently only get 9,000 views per month, I might use a keyword upgrade to drive traffic to my site. The resulting formulas might look like:

For SEO Revenue:

18,000 (people searching for my keywords) X 10% (average success rate, where I define success as clickthroughs with page views) = 1,800 new page views

$1,000/1,800 = $0.50 value per page view

For ROI:

$1,000 (July SEO Ad Revenue)/$200 (Contractor to identify keywords and incorporate them into site) = ROI of 5

Illustration Source: Lisa Solonynko /Haml via Morgue File.
Advertisements

Analytics Success Lies in Digesting Related Segments

Pie

Our 10th analytics class focused on how to best analyze and understand data.

The top-level message I came away with is a data metric shouldn’t be evaluated in isolation, which is meaningless.  For example, knowing only that users spent an average of five minutes on a page offers no insight to validate or adjust your strategy. It may mean only users from a paid referral on one day of the year spent this much time on the site and on other days or referrals from organic/social/direct bounced after 10 seconds.  This would call for course corrections to improve results but without looking at the ‘big picture,’ these issues might go undetected.

Specifically, you should assess data metrics:

  • In context with the organization’s goals, what competitors are doing, industry information, internal initiatives, external events/trends…
  • Ideally in related segments, such as: referral sources (paid/social/organic/direct), days of the week (weekends and weekdays), times of the day or platforms used to access a site.

Sofia cautioned against comparing unrelated data metrics, such as tablet use with social referrals or creating compound or ‘super’ combinations like Alexa page rank with inbound links.

I found it interesting to note that higher numeric data (e.g. 10,000 contact page exits on every 100,00 visits), might deliver exactly the same statistical significance as smaller data (e.g. 1,000 contact page exits out of 10,000 visits) but is more effective when expressed as a clear outcome, such as 10%, versus a ‘muddy’ ratio like 10,000:100,000 or 1,000:10,000. However, I think higher numbers do strengthen a metric’s value.

For the class exercise, we re-examined the metrics set for our senior project, identified segments to be assessed and ways they might be visually depicted.

For my project, here are some KPI segmentation and visualization options:

  • Percentage of conversions by each referral source, such as social (fb, Twitter, Reddit), organic, paid and direct in a specific month, compared to non-conversions by referral sources in the same month.  (Visual depiction – 2 pie charts or a bar graph)
  • Specific Senior Care Share modules engaged in a specific month by referral sources. (Visual depiction – segmented bar chart)
  • Page views for a specific month, segmented by weekdays versus weekends. (Visual depiction – segmented bar chart, pie chart or even infographic with other metrics)
  • Page views in a specific month, segmented by referral sources. (Visual depiction – segmented bar chart)

However, as with all analysis, you should:

  • Find out what’s happening across the organization, such as other initiatives; business changes or help desk calls.
  • Consider external events that might impact data, such as holidays, market trends or even weather/power outages.
  • Possibly do surveys, click density analysis or other research….

….to get the full picture and extract maximum value from your data.

Illustration Source: Haml via Morgue File.

Telling the Visual Story of Data

Screen Shot 2015-03-19 at 4.45.19 PMIn our ninth analytics class, we focused on data visualization, which is used to visually depict a project’s results or analytics.  Seeing a visual representation of data helps people to understand its meaning. It also makes the information more memorable as it helps to tell a story.

Data visualization is particularly effective for conveying a project’s ongoing results, particularly KPIs, to stakeholders and showing them where they need to pay attention or course correct.

There is a huge variety of data visualization formats you can use to tell the visual story of data — from conventional bar and line graphs to golden ratio depleting charts and intricate infographics.  Fortunately, there are a range of tools available for creating them. We took some time to experiment with Tableau. I think this is a really versatile tool but it comes with a steep learning curve.

I think it would be really useful for showing KPIs for my senior project, such as conversion rates from various social and organic referrals, possibly in a pie chart, to show which are most effective.  I might also use a bar graphic to compare the time users spent on each of the niche social media network’s pages.

However, in the interim, I wanted to try this tool on existing data. Last summer, I managed a youth shelter’s eight week fundraising campaign that was heavily promoted on social media. I used a tool called ‘SumAll’ to track Twitter results for this campaign and exported them into an excel sheet. I imported this into Tableau and experimented with the tool.

One challenge was getting the weeks to display from oldest to latest. I re-labelled the weeks from a ‘week of Mon 08 Sept’ format to ‘week 8 (08 Sept)’ format in Excel to make the sequencing clearer. Latterly, I also found that in some views, you can click on the Dimension value to reverse the order.

There is much more formatting I need to do but, the above bar graphic is a start at showing the stats from this campaign. In this example, I used a side-by-side bar chart format to show the growing momentum of three measures (tweets, mentions reach and retweets reach) over the last six weeks of the campaign leading up to a September 6 event date. I filtered out the first two weeks, as the Mentions and Retweet reaches were minimal during this early part of the campaign.

A graph like this could be used to demonstrate to a stakeholder the value of a campaign of several weeks, versus expecting comparable results with a one or two week period (which some might think adequate).

I need to learn much more — but can’t even begin to visually imagine the data I must absorb to proficiently use these tools.

Why Catch Key Words and the Long Tail?

Long Tail ImageOur eighth Analytics class focused on Search Engine Optimization (SEO), which determines how prominent (or where) your website site will be listed in unpaid/organic/natural search results via Google or other online search tools.

A key part of attaining optimal SEO for your site is understanding your target audience’s search behaviours.

We learned that people usually access websites to solve one of the following types of queries:

  • Navigational – to find a specific URL
  • Informational – to find any information from an address to an in-depth research study.
  • Transactional – to complete a task, usually a purchase

A key part of mastering SEO seems to be identifying the keywords people use to find your site. Two main types for keywords are: one word to short terms (2 – 5 words) called Popular/Lead keywords; and longer, detailed phrases, called Long-tail keywords.  You need both for success.

As expected, transactional searches are the highest value because they usually generate revenue but I found it interesting that these are also the more detailed, long-tail words or phrases.

There are also other psuedo keywords, such as ‘Trophy Term-Keywords,’ which may drive traffic to your website but this traffic doesn’t convert or behave how you want it to, likely because it’s not a good fit between your audience and your site. For example, a term like ‘call to arms’ might drive military oriented people to a site but they will quickly leave if it’s about prosthetic limbs.

Sofia provided a great list of links and tips for maximizing SEO. Even pointers like setting URLs in the country of your target audience and putting keywords in JPG titles helps.

To compile a list for my senior project, I used Google’s keyword generator tool for my site, then searched my competitors’ sites and ran the generator for their sites.  I found many popular terms, like: senior care, help for caregivers and help for elderly.  I also found the following particularly useful long-tail options:

  • practical tips from real caregivers
  • outside resources
  • here to share advice from the community
  • online community for family & professional #caregivers
  • Where Caregivers Survive & Thrive
  • Caregiver’s Survival Network
  • family caregivers taking care of a loved one

As my project progresses and the content is clearer, I will re-do and refine my searches to find more.

(Photo: Taken by Gabor from Hungary via Morguefile) 

Adding and Measuring the Impact of Mobile Apps

MobilePhoneMeasureSmallFor week 7 of analytics, we focused on how to measure the impact of an app — be it for entertainment, practical or promotion purposes.

Given the development work behind each app, I think it’s particularly important to measure their analytics so that you can course correct, as required, to maximize the investment. This means using app analytics to learn: who downloads the app,  how they discover it  and how frequently they use it (if at all).

Although app analytics measures many of the same metrics as web analytics, like number of visits and sources, I thought it was interesting to note some of the subtle differences between the two.  For example, I noticed that app analytics metrics include:

  • Screen resolutions (as well as screen size and orientation)
  • Varied types of mobile devices used
  • A stronger focus on groups of users — including user’s friends who also use the app and ‘cohorts’ (i.e., segments of customers that downloaded the app during a common time period).
  • “Usage” instances and frequency, since downloads/installs don’t correlate with engagement/use.

Like web analytics, you need to define metrics and KPIs. For example, you need to define an ‘active’ user’s level of engagement (frequency, duration).

We also looked at various tools for measuring app analytics. However, Sofia pointed out that the metrics they measure are not evenly distributed among the categories of apps available. Specifically, current tools are more focused on marketing metrics, like search engine keywords used to discover them, versus content metrics, such as the app’s longevity/life cycle and number of app/widget installs a user has.

Of the tools we looked at, I particularly liked Flurry Analytics because it provides benchmark measures and several distinct metrics, such as user personas. However, it didn’t look like it provided search engine keywords, which you needed to measure through other tools, such as App Annie, as its Key word ranks feature.

For the exercise, I selected a made-up ‘package ingredient analysis app’ promoted through an Instagram campaign. I identified KPIs and measuring tools as:

  • „Number of active users (Flurry- Event- Use)
  • „Average increase in users (over 2 week time periods) – (Flurry – Funnel Analysis)
  • Number of people using the app at 2x per week or more (Flurry – Retention – Return Rate & Event – Use)

I identified metrics and their tools as:

  • „Search engine words used to find app (App Annie – Key word ranks)
  • „Number of installs (Flurry – Funnel –Downloads)
  • „Number of users who have installed the app vs. Instagram users (app installs and Instagram audience)
  • „Personas using app (Flurry – Audience Interest – Personas)
  • „Use of this App compared to other lifestyle apps (Flurry – Audience Interest – My Apps Interest)

We also explored and discovered several other worthwhile app analytics tools, including:

  • Amplitude – Focuses on user engagement, including: segmentation, cohorts and churn.
  • AppSee – Indepth analysis of users and everything they do with app (for a steep price)
  • Ninja Metrics – Focuses on customers and is well suited to game apps
  • Localytics – Unique tool that integrates analytics with marketing to help produce push campaigns

This class opened a new world of analytics — and there may be an app for that too.

Linking to Success With Social Media

For week 5 of analytics, we dug deeper into social media with a focus on the value of Google+.  Although Google+ may seem less intuitive to use, you need to evaluate its use in terms of your goals. Specifically, if you want ROI or engagement, Google+ may be well worth the time it takes to set up because it controls ‘who’ sees what you share by controlling how high your organization comes up in Google searches.  Sofia showed us a couple of examples from her own business, when she posted on Google+ and how it boosted the company’s online prominence.  I thought this was effective and drove the point home.  With my senior project, I think it may be worthwhile looking into Google+ ‘communities’ that pertain to senior care but also I think it will be important to have a presence on  Google+ so that might key words and organization will be found.

We also learned than ‘not all links are created equal.’  A key way to optimize search engine results to your site (SEO) is by having links from other website’s connect to yours. However, we learned the strength of these links varies according to various criteria, such as:

  • Authority or credibility of each site linking to you. (It’s better to have links from established, high profile sites like Buzzfeed or the Huffington Post than a blog run by one person with a few subscribers)
  • Number of other sites each site links to. (If it links to a high number of other sites, its link is of less value.)
  • Where your link appears (The Jackpot to have your link in the title or anchor text but this is rare.)
  • Relevance of the sites linking to you. (If you pay for links from an essentially ‘content-free’ site or link farm, there is little value)
  • Authority and Page Rank of the linking page

I also learned the word ‘Social Signals,’ which is when someone likes, recommends or shares your post.  So with my senior project, I ideally want many ‘high value’ inbound links, ideally from relevant healthcare/senior care sites and to generate many ‘social signals.’

Drilling Down to Social Strategies

Our fourth Analytics class focused on the first of two parts on analytics for social media. This first part highlighted facebook, twitter and LinkedIn in terms of what users can do on these sites and “how” you can use these platforms to measure online, as well as offline, initiatives.

As with all analytics, you need to identify your SMART goals, KPIs, metrics and measurement methods in advance to ensure you capture all the relevant data and don’t waste time or lose important stats.

I think it’s interesting to note that if you are running a campaign that includes an offline tactic (e.g. a coupon or redeemable voucher), you can measure its impact online — IF you plan in advance.  The reverse also applies. For example, you can measure the offline response of a Facebook ad by using “offer claims.”

For this week’s in class exercise, we had to develop a social media strategy for promoting Parks Canada’s “Unplugged” campaign. For this, we had to identify SMART Goals and tactics for achieving them, as well as KPIs and metrics to measure progress.  I found this exercise a little confusing because we had just learned that traditionally KPIs are metrics that relate directly to your goals and generally impact the business.  Specifically, KPIs traditionally apply to revenue-related metrics (e.g. cost per lead, return on invest) or direct conversions that impact the business (new memberships).  However, I learned that the KPIs for a social media campaign are an exception to this. That is, they are social, as are the SMART Goals that support the campaign (e.g. generating X number of tweets with a specific hashtag).  This is good to note, as ‘when’ I launch my senior project, I will likely do it with a social campaign to drive traffic to the site and ideally generate conversions (member sign-ups).

In the exercise, we had to create a strategy for Twitter, Facebook and LinkedIn. The first two were straightforward, as the campaign was consumer-facing, which both Facebook and Twitter can be.  However, LinkedIn forced us to think ‘outside the box’ as it’s more of a B2B channel.  This was a challenge but good because sometimes clients decide they want to use a specific social media channel and you need to find a way to make it relevant to your marketing needs.