Drilling down into (not provided) organic search traffic

29 August 2013

Want to know a bit more about organic search traffic coming through to your site with the dreaded “(not provided)” flag instead of keyword data?..

This blog post by KissMetrics describes some methods you could use to investigate the traffic coming through “(not provided)”, but there are some weaknesses in the methods they describe.

Using profile filters

Using method 1 in their approach, which utilises profile filters means you should really create a new Analytics profile (now called a “view” in Analytics – thanks for changing the language, Google!) otherwise you create dirty data in the existing Google Analytics view.

Whether you do create a new view (you should!) or not for this, you would only have filtered data from the date you create and activated the filter onward. Therefore, there is no way, at least for a while to get some context about how behaviour has changed for traffic coming through not provided.  It also isn’t so straightforward to determine it as a percentage of all traffic, or of all search traffic.

Furthermore, their methods 2 and 4 are essentially the same, as the queries report in Google Analytics is pulled from Webmaster Tools data anyway.  Interestingly, the numbers do not always match between the reports, but it is also fair to say that it is difficult to trust their accuracy at the best of times. They do provide some useful input to understanding keywords though.

Method 3, using Adwords data is a good way to understand what keywords could bring traffic through organic search, but it is limited by the fact that organic search and paid search are not the same. It takes a thorough knowledge of the PPC activity to understand the limitations of this approach. For example, how could one draw conclusion about potential organic search traffic from a keyword where the ad has a different average position in the ad space than the organic listing has in the organic real search real estate, or even if these are the same?

I’ve picked some holes in their approaches, but to be fair it is impossible to retrieve the keywords data from “(not provided)”. We need to look for other ways of interpreting the data to get the best idea of how we can optimise the site best for organic search.

First of all, it is helpful to disconnect thought slightly from the traditional mind-set of high ranking at all cost. High ranking and well-optimised websites are not the same thing. Ranking rewards well-optimised websites with traffic, but ranking is only part of the journey. Optimisation is about improving the rate at which traffic from search engines becomes customers. Therefore, ranking might increase volume, but relevance and conversion rate and utility are all important factors. Google uses human raters to judge utility of webpages to feed back into its algorithm how useful websites that are ranked well are for users. You are entitled to your own view about how good a job they make of that, of course.

Taking this view, one can begin to use the different data sets to input into search strategy in a different, and more refined way.

Average position, click through rates and the like come from the Webmaster Tools data, albeit with a generous pinch of salt.

Keyword exploration and efficiency studies can come from Adwords. I’d always recommend keeping aside a small click budget for running some long-tail keyword experiments on Adwords, but that is for another blog post.

Enabling an understanding of the incoming traffic is still a job for Google Analytics. However, I would not recommend using profile filters to do this. Better to use Advanced segments in my view, which enable you to compare much more than just the keyword reports.

Finding advanced segments

To set up the segments, log into the correct Google Analytics profile and click on “Advanced Segments” at the top left of the page. Name your new segment “(not provided) Organic Search”, and set the filter options to “Include > Keyword > Exactly matching > (not provided)” and then add an AND statement: “Include > Medium > Exactly matching > organic”.

Advanced Segment set-up

Save your segment, then set an opposite one up, named “Organic Search minus (not provided)”, which is identical except for the first of our two filters, which is now “Exclude > Keyword > Exactly matching > (not provided)”

This allows you to compare the behaviour, landing pages, browsers/OS, conversion rate and so on of traffic coming through as “(not provided)”. Pay particular attention to landing pages, as these can help you identify what the original user intent might have been – compare to known keywords that users land on these pages for.

There is another way to use Google Analytics to investigate the role of (not provided). The top conversion paths report (Conversion > Multichannel funnels > Top Conversion Paths) is customisable in a similar way to creating Advanced Segments.

Click on Channel Groupings, and select “Create a custom Channel Grouping”.

MCF set-up

Name your grouping something like “Not provided study” and create rules identical to the segments described above.

Setting up a Custom Grouping

When you save the grouping, the resulting report identifies the conversion path containing (not provided) organic traffic distinct from other organic traffic. By expanding your channel grouping report, you can examine how (not provided) interacts with other traffic sources, such as PPC, social, and referrals.

MCF Report

Clearly, this is all a very inexact science, but such is the nature of missing data. However, by focusing on the ability of landing pages to attract organic search, and how well they subsequently convert, the process of search engine optimisation focuses more on its real aim – to create customers, not vanity rankings.

If you have more ideas about how to interrogate (not provided) organic search traffic, please share them in the comments!




Subscribe to Email Updates

  • Interesting and I’ve actually gotten away from being so concerned about what keywords are ranking. You said it best “Optimisation is about improving the rate at which traffic from search engines becomes customers.”

  • as these methods are worked here but some things like np show visits without the keywords.

We are a digital transformation consultancy. We help our clients succeed.

View Services