Analytics and the Reporting Life Cycle

In this post I'd like to provide an introduction to the reporting life cycle and discuss some approaches to setting up an acceptable reporting process. This will include discussion of set up for both existing sites, and the opportunities for sites being developed.

I've written this with XEN clients in mind, so when I refer to 'you' I'm talking to you as a XEN client, and as such this is mainly an education piece I've written. If you're an SEO or agency reading this, please keep this in mind (also: welcome).

When it comes to your reporting requirements there's a number of criteria that need to be considered. Reports need to be prepared with your intended audiences in mind, including:

• What the stakeholders want to see
• What the marketing team(s) need to see
• What the the SEO team needs to see
• What the financial approvers needs to see

Tailoring reports can be a complex task, but when prepared properly gives considerable insight and advantage to the company or business.

Reporting  can be broadly divided into two types:

1. General reporting - covers what activity is happening on the site (Eg how many visitors, their engagement).
2. Deeper reporting -  aims to understand why certain activity is happening on the site (eg which segments are changing).

We'll come to this second part - the Why - later. First, let's take a look at the reporting life cycle - also known as a maturity model - of how web reporting usually progresses.

In the following discussions we'll use the term 'reportee' to refer to the person viewing the reports eg it could be you (eg  you might be the marketing manager or sales manager), or perhaps it's someone you report to (eg if you report to the CEO).

The Reporting Life Cycle

Depending on the reportee's main interest they typically focus somewhere along the following reporting life cycle spectrum:

1. Rankings > 2. Traffic > 3. Engagement > 4. Conversions > 5. Financial

1. Rankings

This used to be where everyone focussed their attention. Rankings were the name of the game: What the site is ranking for, how rankings improved/dropped, what additional terms to rank for, those kinds of things.

2. Traffic

Next came the focus on traffic, due to the realisation that high ranking positions don't necessarily contribute to more traffic - due to a bunch of reasons (including personalisation, ads appearing before organic results) - and that reporting on the traffic trends was much more important.

3. Engagement

Then came the understanding that it was important to keep people engaged on the site. If people came and left quickly it defeated the purpose of attracting them in the first place. So metrics such as pages per visit, time on site, return visits, social sharing and commenting became the focus.

4. Conversions

As engagement indicators became more specific, the focus on conversions become more important. People started thinking about conversions as important 'engagement' actions on the site, and at the same time the tools to track them improved (Eg Google Analytics increasing to allow 20 goals, adding events and custom variables).

Web managers started assigning conversion statuses to a range of items (eg signups to tools and resources, paths taken through the site) in addition to the traditional goals of sites (Eg purchases, lead generation form signups, contact us enquiries). Conversion funnels and customer segmentation became much more mainstream.

5. Financial

Ultimately though, reporting matures to focus on the financial impact the site has to a organisation, enterprise or department. It matches site activity with financial goals, whether they be traditional (Eg sales targets) or more strategic (Eg justification of a budget for an awareness campaign).

Although financial metrics aren't often initially understood as part of the general web analytics reporting framework, it doesn't take too much extrapolation to realise how they apply. No matter which industry or sector, most sites end up having a range of dollar-related objectives in mind. A few examples:

• for the typical B2B company, the site is a sales tools and aims to maximise the profit per customer as well as minimising the cost per aquisition
• for a service provider (eg an orthodontist, plumber or consultant) the site is a lead generation tool aimed to increase customer numbers
• for a non-profit organisation the site can be a mix of awareness driver (to justify budgets) and donation channel (to increase revenues)
• for a typical B2C company, the site is usually a brand awareness driver, as well as an ecommerce channel (to increase revenue)
• for a Government department, the site is often aimed at increasing awareness (to justify budgets) as well as long term economic outcomes (eg reducing health service costs, increasing educational intakes, improving environmental efficiencies)
• for a media company (eg a news site), the site is often advertising focussed with an aims to increase revenue per visitor
• for a SaaS provider, the site is often interaction focussed with the goal of increasing revenue generating micro-transactions (eg storage increments or computational cycles)

The key here is to understand what the site reporting needs to do in order to continue being of value to the company/enterprise/department.

And here's where the reporting life cycle comes in. Whereas previously simply reporting on traffic trends was enough to justify further investment (Eg traffic has increased X% each month), increasingly the site needs to demonstrate financial justification (eg our X% investment generated Y% return over Z years). This of course can get tricky - and the purpose of this post is not to go into the process for identifying all the specific financial items. The point here is to simply raise the need to be thinking longer term about financial items when it comes to reporting.

Each Has Its Place

Note though that each of the five items in the life cycle have their place. They are neither better or worse, and whilst ultimately all companies should be increasing their focus on financial end points in their web reporting, there still needs to be a balance that's appropriate for the report audience.

Thus, although ranking reports might seem old-school these days, it's not uncommon for some of our XEN clients to focus solely on rankings - despite numerous conversations around the need to focus further along the life cycle... It's about working out what's appropriate for the audience and providing that first, and then gently pushing them a little further down the lifecycle.

Warning: Do you have enough data?

Note: before jumping to reporting metrics, the first question to answer is whether you have enough data to make informed decisions. This can be a trap otherwise. For the new site with only a few thousands visits it can often be premature to make strategic decisions. Focusing on financial reporting wouldn't be sensible. And thus, it would be entirely appropriate to focus much of the reporting purely on rankings and traffic. Then, once the traffic has increased to a sufficient size, the engagement and conversion reports become more important to focus on.

Here's another example: when it comes to click-throughs from ads we generally like to ensure there's been at least 300 or 400 hundred clicks from an ad  group before spending too much time analysing results. If the majority of your site traffic has come from a big spread of ad related clicks, with only a few clicks from each ad then there's not enough data to make informed decisions. Instead there's the danger of making ill-informed decisions based on incomplete data - often a more costly problem than the opportunity cost of postponing decisions until later.

Understanding 'Why' as well as 'What'

Let's return to the Why question. Most reports will outline the 'What' really well. Example: Mobile traffic increased 7% this quarter.

But in order to take advantage of this, the 'Why' question needs to be understood. ie gain an understanding of whether that mobile traffic is viewing a particular section of content, coming via a particular channel,  and, ultimately, understanding if it contributes to a financial metric.

Delving into these questions also highlights potential issues with the site eg an increase in mobile traffic that leaves at a certain section of the site likely highlights an implementation issue with the site. Which in turn, if matched to a financial metric, may be the justification required for a web development budget in the coming months. A budget that would have been otherwise difficult to request.

Why the 'Why' is Important

It's easy to think that understanding the What is the main importance and Why is just a bonus, but that's rarely the case longer term. Here's why:

• Results improve: let's say you deliver a report to the Executive Board outlining an increase in engagement across the site. Great! They are happy. Soon though, they'll expect to improve that further. How do you do that? Obviously you need to understand Why the engagement has improved. Is it specific campaigns, it is a certain segment of traffic, is it seasonal...
• Result decline: but let's say the reverse happens and results go down. This is probably the most obvious case of wanting to understand why. Let's say engagement metrics drop by 20% across the site in a single quarter. How do you explain why this has happened? And, what if it is possibly a good thing eg a segment of traffic was removed that wasn't actually good traffic - eg they spent a lot of time on the site, but then just left negative troll comments on blog posts. Understanding this and explaining it is important: those negative results (the ones that show in red on reports) might actually be positives.
• Results stay the same: this is rarer than the other two cases (since it hasn't caused problems, nor has it got anyone excited). However, these are often the really important areas to analyse because they highlight inherent issues with the site that just need tweaking: understanding the Why (eg it turns out a whole new section of the site wasn't easily discoverable and hence nobody found it) is useful information to feed into ongoing improvements.
  Continual incremental improvement is often the secret weapon of web sites. However, their incremental nature can easily be missed in reports. Over the long term they drive significant financial improvements.

Let's Analyse The Why.

There are a number of ways to to do this - the common being:

Understanding Audience Segments

Part of answering the Why question is understanding the different types of traffic the site attracts, and then narrowing this to focus on the segments of traffic that the site aims to increase.

For example: It's no use having a growing audience of bargain shoppers (eg referrals from 'deals' sites, searches on 'coupon' related terms) if your site sells premium items. Sure, traffic stats might look healthy, but financially the metrics aren't improving. And perhaps they are actually being adversely affected due to hosting costs and increased customer support queries.

Audience segments are broken into common themes, usually around understanding the visitor 'intent' and matching to personas and general demographics eg looking for information (research intent) versus looking to purchase (buyer intent). They also look to match engagement  and conversions with visitor loyalty eg do existing customers engage and convert more (conventional wisdom says 'yes', but data often contradicts this depending on the site offering).

Aside: The Problem with Sales Funnels

A decade ago most company outbound sales processes worked on a simple numbers game: If we make 1000 calls (via a telemarketing campaign) we will likely get 30 leads (let's say). If we follow these up via a series of further calls, we'll 5 meetings (if we're lucky) which will end up with 2 proposals and ultimately 1 sale. Therefore, the logic goes, if we want to increase sales, we just need to tip more telemarketing calls into the start of the funnel.

Of course, these sales processes are much less common these days, and whilst still effective for some companies, they are mostly on the decline. More on that in another post. However, the point of mentioning it here is to highlight that this 'funnel' thinking has (unfortunately) permeated many web reporting requirements. It's why when a sales manager see that the web traffic has increased 15% they get excited, assuming that it will magically translate into similar lead and sales increases.

But this would miss the point - with so many sources of traffic to a site, just getting an increase is no indication of quality.

Fortunately though, there's a key difference between old-style sales funnel analysis and new-style web analytics analysis. With web analytics we can understand Why visitors convert. Previously the telemarketing calls had no way of understanding why people responded - sure they could extrapolate some metrics (Eg the time of day, the person calling), but they couldn't really identify which particular words within the pitch were of interest.

Preparing For Better Reporting

OK, so far we've looked at the reporting life cycle and considered some scenarios and areas to think about when it comes to the reporting goals.

In this final section we'll quickly look at how to approach moving along the reporting life cycle.

In an ideal world every business would have whole teams working full time on reporting. And there'd be a continual improvement process that applied to reporting and metrics. Sadly, and perhaps ironically, only the large already very successful web based companies can afford to do this. Which means that every reporting setup process has time and implementation constraints imposed. So the process of setting up reporting needs to be as efficient and targeted as possible.

At the start of this post we touched on the various report viewers, each with their separate needs. This is crucial. First, who's footing the bill for the web implementation? Once that's known then they are the first port of call when it comes to establishing reporting metrics. Typically you, or a project manager or a marketing manager will help us facilitate this understanding, so we generally work with you to determine this.

Next comes the site life. For an existing site the process is going to be different to a new site development.

For Existing Sites

For existing sites, the focus should be on improving existing reports and generally moving them a little further down the reporting life cycle. The aim is to augment and incrementally improve reporting.

Typically enabling additional goals in analytics packages is a source of quick wins, as is updating the social sharing plugins to provide analytics integrated  functionality.

For New Sites

The best time to be preparing the reporting approach is as part of a new site development, as it gives the greatest opportunities to tailor the reporting implementation. It also allows all included parties to take a step back and reconsider the reporting approach. New site developments generally come with high level goals that can be translated into reporting requirements. However, the development phase of a new site build affords additional opportunities to implement report tracking, often at minimal cost.

From a technical point of view - especially on large sites - there's a number of considerations for the site implementation including analytics tracking eg:

• adding Google analytics events calls into site functionality calls eg building into internal search functionality
• adding Google analytics custom variables into specific sections eg tracking logged in users, tracking language settings, tracking custom user agent settings, and other segmentation items that are included in global reports

Note that these often need to be built into backend code - they can't just be added as an implementation item later - so it's imperative that they be understood during the design and development stage and not postponed until later (when they may be more difficult and costly to include).

The key point here is to include the longer term reporting goals into present development considerations. Even though the current reporting requirements might not yet be focusing on financial items, preparing for them now as part of a web build is highly recommended.

One Caveat: The Danger of Over-analysis

However, there's always the danger that reporting becomes so detailed that it loses touch with original objectives.

Yes, this happens. Not often (because frankly, we're so time poor that it's not that likely even if we wanted to) but it can happen. Usually this is when there's a mismatch between the level of reporting that the you want, and the level we are focusing on. This can happen in a number of ways:

• you're focussed on a different stage of the reporting life cycle than we are are (Eg you're focussed on engagement, we're focussed on conversions)
• you're focussed on the What, and we're talking about the Why

When this happens it causes confusion, and sometimes frustration (on both sides). This of course is our (ie my) fault - for not explaining the approach well enough or perhaps not understanding your needs well enough. In either case the onus is on us to properly direct the discussion.

This happened to me recently (and in some ways prompted this post), but it's a valuable reminder: always be questioning the value of the reporting. If it doesn't provide value (even long term) then it should be removed from the discussion.

The only thing worse than not enough reporting insight, is being overwhelmed with unhelpful or inappropriate reports.

Whilst it's incredibly important to be focusing on improving reports, this shouldn't cross into inappropriate or useless reporting simply for the sake of it.

Summary

So what have we covered.

My aim with this post has been to:

• highlight the reporting life cycle (or introduce it if it is new)
• encourage you to be considering more of your business or department reporting in terms of financial metrics
• discuss the benefits of having reports that can answer the Why questions (not just the What activities)
• highlight that providing proper reporting often requires careful planning
• recommend using new site developments as an ideal time to implement reporting framework improvements

Hopefully this has been of help. Let me know.