October 11, 2011 Google decided to implement Secure Sockets Layer technology for their authenticated (Google, Gmail, YouTube, etc...) search users. In practice, this effectively meant that the referral string that is otherwise passed by Web servers to browsers indicating the referring page URL is truncated. Specifically, while the search engines domain is still included in the referral string, now the actual search queries are excluded. Of course, they certainly track on their end what their users clicked on - this is valuable insight for PageRank..
While often mistaken as an analytics problem, this is actually the case for all site analytics systems. Moreover, this is an organic search problem only; as long as you are paying for Google's paid search advertising you can have the full referral data inclusive of search queries.
40% and Growing
Back in late 2011, a drone originally calculated the impact as single-digit percentage. Sounds manageable but as more and more stories emerge about much higher percentages this lack of search results transparency becomes more troubling. For your reference, TOTSB decided to take a look at our own site (in both Google Analytics and Piwik) and were shocked to learn as much as 4x the expected amount...and it is trending higher. Below you can see the "Not Provided" percentage volume increasing since early-2011.
An indepdent SEO firm's prepared a study that looked at many Web sites - but the same problem persists.
It is pretty clear that if you are interested in optimizing your organic search presence, the hand that giveth has taketh away. With Google's dominant position in the search market, it essentially means that about half of your organic search keyword results cannot be understood right now. Worse, this remarkably this could continue higher.
The net effect of this move is that Google is denying site owners (the providers of free content to the search engine) their referral information. It is absolutely outrageous that otherwise discerning digital marketers allow this to happen - perhaps class action legal action will emerge. Maybe the W3C or the IAB should get involved and speak out about this perversion of data control.
What to Do?
It is easy enough to track your own Web site's numbers but beyond that advertisers should start playing hardball and complain - especially those buying paid search. Some are already making noise, including:
- An online petition where people can take action for more organic referral transparency; you can even buy a (Not Provided) T-Shirt.
- The organization fairsearch.org will hopefully include "Not Provided" in the scope of their work
- Notprovidedcount.com: a site tracking "Not Provided" results across 60 different Web sites